Developing Upwards

I am announcing the release of Django Better Cache 0.5 today. This release includes a move to sphinx as a documentation tool and a new component, the bettercache.objects module, which provides a lite ORM-like interface for caching data. Please read the full, but short documentation over at Read The Docs for details on the bettercache {% cache %} tag and the bettercache.objects ORM, and have a much easier time with your caching needs. Here is just a quick example of the new cache models, from the docs: class User ( CacheModel ): username = Key () email = Field () full_name = Field () user = User ( username = 'bob' , email = 'bob@hotmail.com' , full_name = 'Bob T Fredrick' , ) user . save () ... user = User . get ( username = 'bob' ) user . email == 'bob@hotmail.com' user . full_name == 'Bob T Fredrick'

This situation comes up, from time to time, when we need to get something to happen after a many-to-many field is changed. The novice will connect a post_save signal and scratch his head when it doesn't fire on the addition or removal of items in the ManyToManyField. We all learn that it takes a slightly more complicated signal, the m2m_changed signal, and its many actions , which tell us exactly what has changed in the particular field sending it (the signal comes from the field's through table, to be exact). Well, a slightly more complicated case arose in a design today and I was scratching my head and feeling like a novice all over again. You see, I needed to know when new things had been adding to such a field, but I had more than one. In fact, I had four of them. I needed a specific function called on the instance when all of these fields were finished being cleared or added to or subtracted from. This was in a form in the Django admin. Thankfully I had an assumpti...

I had a problem with a Django project that took forever to run its unit tests. The test database took an enormous amount of time to run, upwards of ten to fifteen minutes each. I didn't have a lot of ways around this, because I had to use a base Model that pulled in lots of cascading requirements and I couldn't avoid the dozens of applications it needed to build tables for. This was really hindering my ability to develop, as I rely heavily on constantly running tests in my own pathetic attempt at Continuous Integration. After some poking around the PG forums, I eventually worked out this script, which I now run on startup. #!/usr/bin/env bash service postgresql stop mount -t tmpfs -o size=500m tmpfs /mnt/pg_data_mem/ cp -R /var/lib/postgresql/8.4/main/ /mnt/pg_data_mem/ mount --bind /var/lib/postgresql/8.4/main/pg_xlog /mnt/pg_data_mem/main/pg_xlog chown -R postgres:postgres /mnt/pg_data_mem/ sudo -u postgres /usr/lib/postgresql/8.4/bin/pg_resetxlog -f /mnt/pg_data_mem/main/ser...

This is probably taken as the concept for a horror novel by many, but bare with me as I am going somewhere productive with this line of thought. There are a number of valuable ideas in Django, even if they aren’t original. I think some of the concepts could be interesting if applied to Twisted. We can’t start over, but maybe we could bridge the gap. The Project A “Project” is a very vague concept that we all start when we’re doing something new, but in a system like Django it is a codified piece of the framework. What a project is, how it is structured, and where its boundaries lie are all documented carefully. This provides an obvious starting point and creates a wonderful sense of consistency from one project to the next. The Application Within a Django project one or more Django applications are combined, and each application provides a unit of reusable functionality. This might consist of DB models and views to aid in tagging objects, providing search indexes, or generating thumbn...

How many ways can django import a module? Grep is hard for this. In .py files "import (.*)\..*" "from (.*)\..* import .*" "patterns\(['"](.*)['"]" "url(r?['"].*, ['"](.*)" In INSTALLED_APPS and other settings. Am I missing any? Better question: Why do I have to wonder if I'm missing any?

When I complained about the problem, I promptly outlined some ideas about solving it, vaguely. Now, I want to narrow that outline into systems I actually use. I do most of my work with Django, some hobby time is spent with App Engine and Twisted, and I enjoy Amazon Web Services, so I'm thinking from these perspectives when I approach this. Parts one and two were broad, but some of this might only apply to fewer of you. Either ignore those or adapt to whatever you use. Django's cache layer sucks. Simply stated and simply true. Any time I decide I can cache something, I should ask myself if I could have built it before I even had the request in the first place. Doing that with the template caches simply isn't possible. It should be possible and it should be the first path you take, instead of forcing us to go out of our way to do the better thing. Anything I might want to cache, I also might want to be sure I'm not doing in more place than once, and forcing them inline i...

In Part 1 I wrote about my method of testing the actual tag function for a custom Django template tag. I said I would follow it with a Part 2 on testing the rendering of the resulting Node . This is that follow up post for any of you who were waiting for it. Testing the rendering poses some more problems than our little tag function. Rendering is going to do a bit more, including loading the templates, resolving any variables in question, doing any processing on those results (like looking up a record from the database based on the variable value), and finally populating a new context to render the tag's template with. How do we test all of that, without actually doing any of that? This is the goal we're reaching here, for unittests. We want each test to be so specific that we test what something will do, without actually relying on those things it does. We aren't testing any of those things, just that our render() method does them. What can we mock easily? get_te...

I'm involved in a project that has gone for a long time without tests and everyone involved knows tests are rilly rilly important . There is a point where acknowledged best practices simply meets the reality of the development mind and it doesn't always work out like you'd hope. We know tests are important, but we need to resolve this ticket right freaking now . You understand. The point was reached that this just couldn't continue and the costs of fixing the same bugs over and over were way too obvious to ignore. Tests are now popping up for things we're fixing and new things we're writing. As it happens, I came across my first real need to create a custom template tag. Of course, I wanted to test it. So how do you test something that is so entrenched in the Django processing pipeline as a template tag? Incidentally, I'm just going to assume you either know all about testing and Django template tags or you can follow along just fine. Testing breaks ...