When Django 4.00rc-1, pipenv, djstripe and docker decide to pull the rug from under you

Sometimes you just need to rebuild your docker containers and that's where the problem started. When my containers came back, it wouldn't start because of a TypeError being thrown. 

  File "/usr/local/lib/python3.8/site-packages/djstripe/models/__init__.py", line 25, in <module>  
   from .core import (  
  File "/usr/local/lib/python3.8/site-packages/djstripe/models/core.py", line 30, in <module>  
   from ..signals import WEBHOOK_SIGNALS  
  File "/usr/local/lib/python3.8/site-packages/djstripe/signals.py", line 8, in <module>  
   webhook_processing_error = Signal(providing_args=["data", "exception"])  
 TypeError: __init__() got an unexpected keyword argument 'providing_args'  

Hmmm...

So I reviewed what was running on my docker containers. I had 4 containers: django with pipenv, celery, rabbitmq, and postgres. I ruled out postgres and rabbitmq out of the gate since the error is clearly a Python error. 

Focusing on the django container, I eventually found the error - took me the whole afternoon. At first I thought it was a docker bug but it wasn't. It was because of how pipenv was getting and installing the packages. Let me explain.

Our pipenv file that had django was set to install the latest version so it was written as django = "*". When I rebuilt the container and the dockerfile executed it downloaded the latest version of django which was 4.00rc-1.

The fix was to set the version: django = "~=3.2".

The moral of the story is don't blindly set your pipenv file to "*" and instead be explicit with the versions. Luckily pipenv does support semantic versioning meaning it will install minor versions of django 3.2 like 3.2.2 but not Django 4.0.

Downtime Investigations: Pipenv vs Poetry

Python projects start around a virtual environment. This helps organizing project dependencies and builds are deterministic. In this area there are two maturing tools Pipenv and Poetry. I've been a long time user of pipenv but it's never a bad idea to see what the other side has to offer. 

Handling Packages

Both Pipenv and Poetry organize project dependencies with a separate file for production and development. ie: requirements.txt, requirements-dev.txt

Pipenv uses a Toml file called Pipfile while Poetry uses a similar Toml file called pyproject.toml. Interesting side note, unlike Pipfile, pyproject.toml follows PEP 518. 

The difference here, I discovered, is that Pipfile is smaller in scope vs pyproject.toml. The pyproject.toml can be contain configs for supports tools like flake8 or publishing info if you're going to put the project up to pypi. You need a separate file, like say setup.py, in Pipenv to publish your project/package. 

Adding dependencies

Pipevn's Install command installs all dependencies if you don't specify a package. Poetry decided to have separate commands for add a new dependency and installing existing ones. In effect, Poetry ask the dev to be more explicit on what he's try to do. 

Poetry also has more info in the terminal vs Pipenv which is fairly spartan when installing something.

Pipenv installing pytest

Poetry installing pytest

Uninstalling dependencies

A thing that I discovered, Poetry uninstalls sub-dependencies. Pipenv does not. With pipenv you have to do something like pipenv uninstall [some package] && pipenv clean to do what Poetry does from a single poetry remove [some package]

Wrap up

Both Pipenv and Poetry goal's are to make dependency management easier and building projects more consistent. While pipenv has more broader support but I think that's just because it's older than poetry. Poetry has some interesting features like having configs in it's project Toml and uninstalls sub-dependencies. 

At some point in the feature, I think I'd like poetry in my professional projects.