Setting up a Django Development Environment

In this screencast I show how I typically set up my Django development environment. It goes through installing Django by checking out the latest development version and linking it to the correct places on your system. It also talks about how to install reusable applications. Finally, it covers how to update all of those projects and keep a toolbox of snippets for your personal use.

The simple pylink command that I use in the screencast is this:

#!/bin/bash
ln -s `pwd`/$1 `python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`/$1

Please let me know in the comments if you have any other tips and tricks for setting up a development environment for Django.

Using Django-Pagination

Django-pagination is an application that I wrote and released a while ago, which I use all the time, but that hasn't really seen much attention. In this screencast, I show how to take an existing project with too much data on one page, and use django-pagination to quickly and easily paginate the items on the page. There is a bit more documentation for the project that's available in the project directory if you do a subversion checkout, and docstrings throughout the source code, if you're interested in how it works.

Keep in Mind

These are my very first screencasts, ever. I'm not entirely sure what I'm doing yet, and the only way I can improve is by your feedback. If you have any advice and/or criticisms of these screencasts, please don't keep your mouth shut--speak up, and let me know in the comments. Hopefully someone finds these useful, and thanks for watching!

Feedalizer.net

Before getting too far into this post mortem, I'm going to just get it out of the way and shamelessly promote the site that Tony and I created during the dash: feedalizer.net. The idea behind the site is that it's a feed aggregator, but people vote on the feeds. The higher the feed's score, the more likely items from that feed will bubble up to the top of the list. There's also the concept of a "channel", which only aggregates feeds for a specific area. For example, there's a Humor channel, a Django channel, and a Python channel. You can also subscribe to channels to create your own "station", which aggregates the content from the channels that you care about.

The idea came to me when a friend of mine asked me "I've never used a feed reader before, but I want to get started and subscribe to programming feeds. What are some good ones for me to subscribe to?" It took me about 30 minutes to cull through my feeds and produce a list of the best. But it shouldn't have taken me any time at all--there should have been a site out there to do this for him!

OK, enough shameless self-promotion.

The Dash

52 teams registered for the dash, so watching the commit activity at the turn of the clock was pretty crazy. Unfortunately, Tony was driving from 4 hours away and he hadn't arrived yet. When he did arrive, we both wanted to spend some time catching up and talking about non-Django things. So we didn't even get started until about 3:30AM. Getting started mainly consisted of frantically checking in 3rd party projects that we thought we would use, and talking about architecture, and writing a few cron jobs. Not much code got written that night (morning?), since we still had a lot of planning to do.

The next day, all of a sudden our commits weren't working! We went to the website to see what was going on, and the website wasn't responding to our requests. Something was definitely going on, and it was slowing down our progress significantly. We tried working on our own separate parts of the project, but at this early stage there was simply too much overlap. We found out later in the night that there were problems at Webfaction's data warehouse,`The Planet`_, where a transformer quite literally exploded.

This severely slowed us down, because we ended up having to switch to git, and then once we got everything into our git repository, we had tons of merge conflicts. We got an e-mail saying that the due date would be postponed, so we decided to take the afternoon and night off to do other things.

The next day we did the brunt of our work. I had the task of designing the frontend, so I opened up my trusty text editor and hammered out the worst-looking CSS file you'll ever see in your life, producing some of the worst-looking pages you'll ever see in your life. This changed over the course of the day, but not by much as you'll see if you visit the site. This same day, Tony was working on some of the harder queries etc.

The final day (the deadline had been extended, remember) was all about integration. There was nothing really notable about this, but it took all day to get everything working properly together. I ended up writing a bunch of Javascript to make the client experience more enjoyable, and Tony had the chance to debug his views now that I had templates and we had sample data. It was a crunch to make the deadline, but we tried to do the little important extra details like write an "about" page, a README file, etc.

Conclusion

Whether we win or lose, and despite the technical difficulties that The Planet suffered, I had a blast doing the competition. I think that our idea is novel, and Tony and I got to work on something once more post-graduation. (Nothing like a programming competition to bring people together, I always say.) In fact, we'll probably continue to work on it for the months to come, especially in upgrading its graphics. It's going to be really awesome to see what everyone else produced this year. I encourage anyone who thought about participating this year, or anyone who even considers it as a possibility, to sign up and just do it next year!

Model and Field Setup

First let's just set up two basic models. The first will be our mixin, NaiveHierarchy, which has a single field, parent, which is a reference to itself. Using this, we can traverse the tree and find all sorts of fun hierarchical information. Also, we'll create the canonical example model: the blog post. Our models start out looking something like this:

from django.db import models

class NaiveHierarchy(models.Model):
    parent = models.ForeignKey('self', null=True)

    class Meta:
        abstract = True

class BlogPost(NaiveHierarchy):
    title = models.CharField(max_length = 128)
    body = models.TextField()

    def __unicode__(self):
        return self.title

Now let's test to make sure that worked. We'll create some data and test that parent exists on the instances.

>>> from mixins.models import BlogPost
>>> bp = BlogPost.objects.create(title="post1", body="First post!")
>>> bp2 = BlogPost.objects.create(title="post2", body="Second post!", parent=bp)
>>> bp3 = BlogPost.objects.create(title="post3", body="Third post!", parent=bp2)
>>> bp.parent
>>> bp2.parent
<BlogPost: post1>

Inherited Class-Level Methods

So as you can see, everything is working correctly! But that really doesn't save us much time yet, as it's fairly easy to copy and paste fields onto new models, and we still have to write methods which take advantage of those new fields. In this case, I already know that I'm going to want to get the related children and descendants of my blogposts. So why not write those methods on the abstract model? Thanks to inheritance, those methods will apply to the new model as well.

class NaiveHierarchy(models.Model):
    parent = models.ForeignKey('self', null=True)

    def get_children(self):
        return self._default_manager.filter(parent=self)

    def get_descendants(self):
        descs = set(self.get_children())
        for node in list(descs):
            descs.update(node.get_descendants())
        return descs

    class Meta:
        abstract = True

Now, getting all the children or descendents of a particular node is easy:

>>> bp.get_children()
[<BlogPost: post2>]
>>> bp.get_descendants()
set([<BlogPost: post2>, <BlogPost: post3>])

Now this NaiveHierarchy mixin is starting to become quite useful! But what happens if I want to get all of the BlogPosts that have no parents? It's really manager-level functionality. So let's write a manager which defines a get_roots function. Unfortunately, using abstract managers doesn't quite work yet (it works for non-abstract inheritance), but it probably will in future versions of Django. In fact, by applying the latest patch on either Django ticket 7252 or 7154, it will work today. Let's see how this would look:

class NaiveHierarchyManager(models.Manager):
    def get_roots(self):
        return self.get_query_set().filter(parent__isnull=True)

class NaiveHierarchy(models.Model):
    parent = models.ForeignKey('self', null=True)

    tree = NaiveHierarchyManager()

    def get_children(self):
        return self._default_manager.filter(parent=self)

    def get_descendants(self):
        descs = set(self.get_children())
        for node in list(descs):
            descs.update(node.get_descendants())
        return descs

    class Meta:
        abstract = True

class BlogPost(NaiveHierarchy):
    title = models.CharField(max_length = 128)
    body = models.TextField()

    objects = models.Manager()

    def __unicode__(self):
        return self.title

Note that we needed to explicitly define objects as the basic manager, because once a parent class specifies a manager, it gets set as the default manager on all inherited subclasses. This would play out exactly how you would expect:

>>> BlogPost.tree.get_roots()
[<BlogPost: post1>]
>>> BlogPost.tree.all()
[<BlogPost: post1>, <BlogPost: post2>, <BlogPost: post3>]

Advanced Stuff

So now I really wanted to push the limit, and write a mixin which would enhance one of the basic methods of all Model classes: save(). This would be a DateMixin which would contain date_added and date_modified, where date_modified was updated on each save. To my surprise, this Just Worked. Let's see the final result:

import datetime
from django.db import models

class DateMixin(models.Model):
    date_added = models.DateTimeField(default=datetime.datetime.now)
    date_modified = models.DateTimeField()

    def save(self):
        self.date_modified = datetime.datetime.now()
        super(DateMixin, self).save()

class NaiveHierarchyManager(models.Manager):
    def get_roots(self):
        return self.get_query_set().filter(parent__isnull=True)

class NaiveHierarchy(models.Model):
    parent = models.ForeignKey('self', null=True)

    tree = NaiveHierarchyManager()

    def get_children(self):
        return self._default_manager.filter(parent=self)

    def get_descendants(self):
        descs = set(self.get_children())
        for node in list(descs):
            descs.update(node.get_descendants())
        return descs

    class Meta:
        abstract = True

class BlogPost(NaiveHierarchy, DateMixin):
    title = models.CharField(max_length = 128)
    body = models.TextField()

    objects = models.Manager()

    def __unicode__(self):
        return self.title

Conclusions

Mixins can be powerful tools, but there are some hazards in using mixins, which all boil down to the same basic problem: unexpected consequences. In the case of the DateMixin, if any other class has defined a save() method, our custom save() method simply won't be called unless called explicitly. Perhaps this is a documentation problem, but perhaps it's a fault in the idea of a date mixin altogether.

So all that being said, I'm not suggesting to go off and start using any of the mixins that I have provided here, but rather to illustrate how a mixin can be constructed with Django's new Model Inheritance. I do hope that a reusable app emerges with some great mixins that are useful for a large variety of tasks. Because mixins are powerful, and new shiny things that Django can do, and new shiny things are worth being explored!

Examples of Composition and Inheritance

Before talking about the consequences of inheritance vs. composition, some simple examples of both are needed. Here's a simplistic example of object composition (using Python, of course, as our demonstration language):

class UserDetails(object):
    email = "floguy@gmail.com"
    homepage = "http://www.eflorenzano.com"

class User(object):
    first_name = "Eric"
    last_name = "Florenzano"
    details = UserDetails()

Obviously these are not very useful classes, but the essential point is that we have created a namespace for each User object, "details", which contains the extra information about that particular user.

An example of the same objects, modified to use object inheritance might look as follows:

class User(object):
    first_name = "Eric"
    last_name = "Florenzano"

class UserDetails(User):
    email = "floguy@gmail.com"
    homepage = "http://www.eflorenzano.com"

Now we have a flat namespace, which contains all of the attributes from both of the objects. In the case of any collisions, Python will take the attribute from UserDetails.

Consequences

From a pure programming language complexity standpoint, object composition is the simpler of the two methods. In fact, the word "object" may not even apply here, as it's possible to achieve this type of composition using structs in C, which are clearly not objects in the sense that we think of them today.

Another immediate thing to notice is that with composition, there's no possibility of namespace clashes. There's no need to determine which attribute should "win", between the object and the composed object, as each attribute remains readily available.

The composed object, more often than not, has no knowledge about its containing class, so it can completely encapsulate its particular functionality. This also means that it cannot make any assumptions about its containing class, and the entire scheme can be considered less brittle. Change an attribute or method on User? That's fine, since UserDetails doesn't know or care about User at all.

That being said, object inheritance is arguably more straightforward. After all, an e-mail address isn't a logical property of some real-world object called a "UserDetails". No--it's a property of a user--so it makes more sense to make it an attribute on our virtual equivalent, the User class.

Object inheritance is also a more commonly-understood idea. Asking a typical developer about object composition will most likely result in some mumbling and deflection, whereas the same question about object inheritance will probably reveal a whole host of opinions and experience. That's not to say that composition is some sort of dark art, but simply that it's less commonly talked about in so many words.

As more of a sidenote than anything else, inheritance can be speedier in some compiled languages due to some compile-time optimizations vs. the dynamic lookup that composition requires. Of course, in Java you can't escape the dynamic method lookup, and in Python it's all a moot point.

My Preferences

In general, I find object composition to be desirable. I've seen too many projects get incredibly (and unnecessarily) confusing due to complicated inheritance hierarchies. However, there are some cases where inheritance simply makes more sense logically and programmatically. These are typically the cases where an object has been broken into so many subcomponents that it doesn't make sense any more as an object itself.

The Django web framework has an interesting way of dealing with model inheritance, and I think that more projects should follow its example. It uses composition behind the scenes, and then flattens the namespace according to typical inheritance rules. However, that composition still exists under the covers, so that that method may be used instead.

The answer is not going to be "composition always" or "inheritance always" or even any combination of the two, "always". Each has its own drawbacks and advantages and those should be considered before choosing an approach. More research needs to be done on the hybrid approaches, as well, because things like what Django is doing will provide more answers to more people than traditional approaches. Cheers to continued thought about these problems and to challenging conventional thought!