How not to do pip requirements

If you’re working on a Python application, you’re probably using pip, and thus will need a requirements.txt file along the way to handle your project dependencies (when deploying your application for example).

The most common way to generate a requirements.txt file is to simply output the pip freeze command in the file.

pip freeze > requirements.txt

This will list not only the packages you manually installed (top-level dependencies), but also all of their dependencies (low-level dependencies).

amqp==2.1.1
anyjson==0.3.3
beautifulsoup4==4.5.1
billiard==3.5.0.2
celery==4.0.0
Django==1.10.2
django-unixdatetimefield==0.1.4
djangorestframework==3.5.1
djangorestframework-bulk==0.2.1
kombu==4.0.0
pytz==2016.7
redis==2.10.5
requests==2.11.1
vine==1.1.3
wheel==0.24.0
gunicorn==19.6.0

While it is considered best practice to use this complete list of all dependencies, it can be a handful when trying to globally update your application packages, especially if you’re not familiar with the codebase and don’t know if a dependency is top-level or not. This can result in a headache of package compatibility issues.

A solution to that problem is to work with a second requirements file, where we’ll keep track of our application top-level dependencies: requirements-top-level.txt

Django==1.11.3
djangorestframework==3.5.1
celery
beautifulsoup4==4.5.1
redis==2.10.5
requests
django-debug-toolbar==1.6
djangorestframework-bulk

In this file, you’ll list your top-level dependencies, with or without a specific version.

Usage

# Installing the top-level dependencies, upgrading them if necessary
pip install –r requirements-top-level.txt --upgrade 

# Updating requirements.txt
pip freeze –r requirements-top-level.txt > requirements.txt

requirements.txt will contains the output of pip freeze after the top-level dependencies have been installed, with the top-level and low-level dependencies nicely separated.

Django==1.11.3
djangorestframework==3.5.1
celery==4.0.0
beautifulsoup4==4.5.1
redis==2.10.5
requests==2.11.1
django-debug-toolbar==1.6
djangorestframework-bulk==0.2.1
## The following requirements were added by pip freeze:
amqp==2.2.1
anyjson==0.3.3
billiard==3.5.0.3
django-unixdatetimefield==0.1.4
kombu==4.1.0
pytz==2017.2
sqlparse==0.2.3
vine==1.1.4
wheel==0.29.0

For a fresh install, you can safely use pip install -r requirements.txt