Reviving Gamelog, a project to track video game playtime and purchases

Because I love putting my entire life into spreadsheets, I started tracking the video games I play over a decade ago. At first, everything was contained in a Google Sheet, and this was actually a pretty good solution for a while. I even managed to come up with an in-sheet visualization that was satisfying:

The problem with a Google Sheet is that it sort of acts like a relational database but clearly is not. Sure, you can put together enough hacky code to get your sheet to recognize that, say, the “Final Fantasy X” entry in one sheet is actually the same game as “Final Fantasy X” mentioned elsewhere, but it’s not exactly built for that purpose. At some point I wanted to start keeping track of how much money I was spending on games, as well as how much time I spent playing them. And ideally I would be able to look up whether I was actually playing the games I was buying or not, and how long. (This actually turns out to be the kind of stat I don’t look at too often, but it was an early consideration of mine.) I was also playing around with Django at the time; I’d built several sites using Django for its original purpose as a journalism-focused CMS, but in theory the web framework could be used for a lot. And so in 2012, I decided to move out of the Google Sheet and into a Django app.

The system worked great, for the most part. There were some challenges, mostly to do with some of the more complex entity relationships and how they interacted with Django’s form system, but after a month or two development I started using it to track all my video game purchases and playtime. And then I left it alone. For a decade.

A few years ago I realized that Django had long since outgrown its v1.x origins, and getting to far behind the latest version had potential downsides even for a personal project that wasn’t actually exposed to the open internet. So I began a project to upgrade the codebase from Django 1.5.x to the then-latest version of 2.2.1. This is where the sheer age of the codebase came back to bite me: it was developed well before I knew anything about unit testing or test frameworks, which of course would help immensely in ensuring the functionality of the app didn’t break during the upgrade. This led me down a lot of rabbit holes as I decided what to do next: should I build a test suite in that years-old version of Django? Should I start from scratch? And how was I going to handle some of the chronic issues I’d uncovered in almost a decade of using the app I’d built? Between all these opportunities for indecision and the increasing mental workload in my career giving me a certain allergy to doing any hobby coding after-hours, I set the upgrade project aside and continued using the original app, warts and all.

Fast forward to now. Django has moved on, not just to a new version but a new versioning system altogether; now it’s at version 5.1, quite a ways away from 1.5.x. I’ve also moved on; I have more free time, and the warts I’d mentioned earlier have become increasingly annoying. I’ll explain these issues in a later post; for now, it’s just worth noting that there was enough incentive to get this onto a more modern version of the framework and potentially to continue improving and refactoring what I’d already built.

I’d like to pretend that I had an elegant process for upgrading the app to Django 5.x, one that ensured no significant downtime. But let’s face it: without that test suite, I was fully expecting a lot of things to break. I think there are ways to automatically upgrade a Django application codebase automatically, but I’ve encountered trouble with such tools in the past, and in any case there was some custom code that didn’t have any obvious analogue in Django 5.x; completely different solutions would have to be found, as that code couldn’t have been ported over anyways. So I did everything the brute-force way: checked out the repo in a new Python environment, upgraded Django to 5.x, and launched the testserver to see what broke.

The first main challenge: MySQL, surprisingly. It turns out that Django stopped supporting MySQL 5.x at Django 4.2. Migrating data to MySQL 8 is supposedly mostly painless, but not necessarily entirely. Since this was somewhat outside my initial scope and could potentially be handled later, I ended up retargeting the upgrade to Django 4.1.x; at the very least it would be much, much closer to being fully up to date, and I could save the task of moving all my data or upgrading MySQL in place for another time.

After that, I’m surprised to report that the process was actually relatively smooth. Part of this may be that some of the heavy lifting was done in my previous attempt to upgrade to 2.2.1, and that the versioning change meant that 4.1 was not as far away from 2.2.1 as the numbers would suggest. Still, you always expect a bunch of stuff to break, and for the most part it didn’t; the vast majority of the issues I ran into were basic things like certain functions being deprecated in favour of new ones (render_to_response() was a big one, as were some of the urlconf pattern functions). The next biggest issue ended up being the form to add or edit gaming purchases, which relied on a custom <select> widget decorated with additional HTML attributes to trigger JavaScript changes in the form. Previously, there were some undocumented ways to modify how <option> elements were created by the form framework; these have apparently been eliminated, meaning I had to find a completely different approach. Luckily, this wasn’t too difficult, although it is a bit clunky: I just render the entire widget manually in a Django template.

After running both versions side-by-side to make sure there were no major issues, I finally switched over to the Django 4.2 codebase a few weeks ago, and aside from one or two lingering bugs from the transition, it’s been painless. So now I have a codebase that runs on a relatively modern version of the Django framework. That doesn’t mean, of course, that the application couldn’t use a good refactoring; there are lots of quirks that result partially from the rat’s nest nature of the code, and partially from the complete lack of unit tests. And of course, there are all those warts still to deal with. I’ll go into more detail on the biggest one next time.