Even the newest django developers are probably familiar with the Django idiom admin.site.register(ModelClass, ModelAdminClass). When we run admin.autodiscover() in our urls.py, and include admin.site.urls, we know that it will find all the admin.py files for our INSTALLED_APPS and that we will have access to the CRUD functionality for our models through a standard interface. This is a very useful pattern for developing easily extendable functionality. A specific interface / functionality is defined (in the case of the admin, this would be the ability to do CRUD operations on our models) and then it is up to the programmer to define the models and behaviors they want to make available. Haystack search uses such a system, with the convention of a 'search_indexes.py' file in the app directory. So how does it work?

There are a couple things going on. Taking a look at django.contrib.admin.sites, we find a class called AdminSite which is instantiated at the bottom of the file. The first lines of the init method reveals that at the heart of this class, there's an attribute called '_registry', which is a dictionary of Model classes and ModelAdmin instances.

def __init__(self, name=None, app_name='admin'):
    self._registry = {} # model_class class -> admin_class instance

When we import admin and run admin.site.register(), the register() method on AdminSite is called, which performs some validation and then adds the model/modeladmin to its internal dictionary:

# Instantiate the admin class to save in the registry
self._registry[model] = admin_class(model, self)

When we include admin.site.urls in the urlconf, the "urls" property refers to the method get_urls(), at the bottom of which is this code:

# Add in each model's views.
for model, model_admin in self._registry.iteritems():
    urlpatterns += patterns('',
        url(r'^%s/%s/' % (model._meta.app_label, model._meta.module_name),
            include(model_admin.urls))
    )

In this way, URL patterns are created for all the models we've registered. Inside the ModelAdmin, there is a property called urls which similarly refers to a get_urls() method - this method exposes the CRUD views. Thus we have a system whereby any number of models can register admin classes with the admin system. We don't have to create url patterns for all the models we wish to use, and we can lean on the ModelAdmin class to provide the functionality we need, extending it as much or as little as we want without having to change the way the AdminSite works.

The last bit of the admin I want to talk about is its autodiscover() method, which is similarly implemented in Haystack. The autodiscover() method lives in django.contrib.admin.init. On line 4, we can see that it is importing the AdminSite class and the instance:

from django.contrib.admin.sites import AdminSite, site

Here is the code that does the importing. It figures out the path of each app in INSTALLED_APPS, looks for an 'admin' module inside each package, and finally imports it. When the admin module is imported, all those calls to admin.site.register() get executed and the AdminSite's internal registry gets populated.

import imp
from django.conf import settings

for app in settings.INSTALLED_APPS:
    # For each app, we need to look for an admin.py inside that app's
    # package. We can't use os.path here -- recall that modules may be
    # imported different ways (think zip files) -- so we need to get
    # the app's __path__ and look for admin.py on that path.

    # Step 1: find out the app's __path__ Import errors here will (and
    # should) bubble up, but a missing __path__ (which is legal, but weird)
    # fails silently -- apps that do weird things with __path__ might
    # need to roll their own admin registration.
    try:
        app_path = import_module(app).__path__
    except AttributeError:
        continue

    # Step 2: use imp.find_module to find the app's admin.py. For some
    # reason imp.find_module raises ImportError if the app can't be found
    # but doesn't actually try to import the module. So skip this app if
    # its admin.py doesn't exist
    try:
        imp.find_module('admin', app_path)
    except ImportError:
        continue

    # Step 3: import the app's admin file. If this has errors we want them
    # to bubble up.
    import_module("%s.admin" % app)
# autodiscover was successful, reset loading flag.
LOADING = False

Another approach: Metaclasses and Django's Models

Django's models are a pretty awesome feat of programming. They, too, implement a registry pattern. Taking a look at django.db.models.base, at the top of the file there's a class definition for ModelBase, which subclasses 'type'. It overrides the new method of type, adding custom behavior to all models (which specify ModelBase as their metaclass) when they are created. At the bottom of the new method, on line 190, there is the following:

register_models(new_class._meta.app_label, new_class)
#...
return get_model(new_class._meta.app_label, name, False)

Following the code to django.db.models.loading, we see a class definition for an object called AppCache and at the bottom of the file that class being instantiated (cache = AppCache()) - this is similar to the way AdminSite works. When register_models() is called in ModelBase, it populates AppCache's internal registry (the "app_models" attribute) with a key/value of app_label -> SortedDict(), which in turn is a dictionary of model name / model class pairs. The new method of ModelBase calls its parent-class' new method to create a class, adds a ton of functionality to that newly-created class, and then registers it with AppCache. If you look at Model's class definition, which follows ModelBase in django.db.models.base, you will see:

class Model(object):
    __metaclass__ = ModelBase

Whenever a subclass of Model is created, it uses ModelBase to create itself, allowing all this functionality to happen automatically, behind-the-scenes. To see how this stuff gets used, take a look at syncdb's code (django.core.management.commands.syncdb). It calls get_apps() on the AppCache, which is a SortedDict of app_label -> SortedDict(), which in turn is a dictionary of model name / model classes:

# Create the tables for each model
for app in models.get_apps():
    app_name = app.__name__.split('.')[-2]
    model_list = models.get_models(app)
    for model in model_list:
        # Create the model's database table, if it doesn't already exist.
        # ...

Line 4 of django.db.models.init imports AppCache functions, so they are available directly from django.db.models:

from django.db.models.loading import get_apps, get_app, get_models, get_model, register_models

This is a very powerful pattern, and it is talked about in greater detail in Marty Alchin's book Pro Django.

The plug-in approach to metaclasses

A similar approach to using Metaclasses is discussed in Pro Django as a way of creating a plug-in system. A real-life example of this technique can be found in the code for oohEmbed.

Looking at line 7 of base.py, there is the following Metaclass definition:

class ProviderMount(type):
    def __init__(cls, name, bases, attrs):
        if not hasattr(cls, 'plugins'):
            cls.plugins = []
        else:
            cls.plugins.append(cls)

    def get_providers(self, *args, **kwargs):
        return [p(*args, **kwargs) for p in self.plugins]

It does not override the new method, as ModelBase does, but it does do some interesting stuff in the init. It creates a list attribute on the class called 'plugins' when Provider is defined, then for all subclasses of Provider the metaclass adds the class being created to this list. When get_providers is called, it iterates over the plugins, instantiating them and returing them as a list.

class Provider:
    __metaclass__ = ProviderMount

The Provider class specifies ProviderMount as its metaclass, and taking a look at videoprovider.py, on line 11 a YouTubeProvider is defined, inheriting from Provider. In the app's main.py, Provider is imported and all the provider subclasses are retrieved and instantiated:

from provider import *

class EndPoint(webapp.RequestHandler):
    providers = Provider.get_providers()

Whenever a request comes in, the get handler iterates over the providers, searching for the right one.

for p in self.providers:

Implementing it yourself

The way you choose to implement this pattern depends on your needs. The Admin method has an explicit registration step, as well as the ability to programmatically unregister a model. I've seen this technique used to unregister a default ModelAdmin (for example one included in contrib) and register your own version that adds some functionality. The metaclass method removes the explicit step of registration/discovery and handles autodiscovery at class-creation. However you end up doing it, I hope this post was informative! Any corrections / points of clarification are appreciated.

Further Reading

• Pro Django
• django.contrib.admin.sites
• django.contrib.admin.init
• django.db.models.base
• django.db.models.loading
• oohEmbed source