Years ago at PyCon Ireland there was an enthusiastic workshop on Test-Driven Development (TDD) by Harry Percival. (He's written a book on it, now in its second edition). The idea is that you write the tests first, show they fail, then write the minimum of code to fix them and then the tests pass. This way everything you write has a test to back it up. I really like the idea, and although I have to force myself to do it, I do like the warm fuzzy feeling of a proper set of tests covering every line I write before I even write it.
Anyway, a minor corner of my mostly-sysadmin job is web development. I'd written a Django project for internal use that used Django's built-in admin pages as a database front end. I wrote it while Django 1.10 was current, but shortly after I finished it 1.11 was released. I could have left my project as 1.10, but its support life is pretty short, and 1.11 is "LTS", long-term support, with support for three years.
I bumped my requirements.txt to pull slightly newer versions of three packages, including Django 1.11 instead of 1.10.*
I ran git push to our private GitLab instance, which triggers GitLab-CI to run my tests, and a minute or two later, boom, tests failed. Lots of them. Most of them, actually. I was vaguely reassured that so many had failed, because somewhat counter-intuitively when something is fundamentally broken it's often quite easy to fix. I read release notes, changelogs, tried various combinations of settings.py. I thought, hang on, GitLab is really actively developed (there's a new version every week almost and we're pretty good and keeping it up to date) so maybe something changed there.
Nothing worked. My buildfix git branch was getting increasingly desperate. The errors from the tests weren't particularly helpful:
ProgrammingError? Really? Thanks.
It took a while (in my defence this was neither urgent nor the main part of my part-time day job), but I finally properly read the error above the exception:
E django.db.utils.ProgrammingError: column project_indexname.history_change_reason does not exist
I had thought until now this was a problem with py.test, or my PostgreSQL service definition in GitLab-CI. Finally I ran:
Well, F*!#, if it didn't spit out a heap of database changes. Remember I'd updated some of the packages above? One of them was django-simple-history, a nice little package for adding change histories to any Django model. Updated from version 1.8.2 to 1.9, and it seems that changed its database tables. git add/commit/push, tests pass.
- TDD was correct. My project really was broken, and the automatic tests meant it never got pushed into production.
- Small version changes sometimes aren't.
- Treat all version changes like refactoring. Change one package version at a time.
* I'll note here that I somewhat disagree with people who say to use the exact versions given by pip freeze, and instead use ranges for things like this:
This will pull the latest of the 1.11 series (1.11.7 today), so you get the latest bug fixes and security fixes, but it should all still work. Certainly in my case I'm doing nothing adventurous enough to cause trouble. And anyway, I've got tests!