Looking at registration patterns in Django

January 10, 2010 14:37 / django django-patterns python / 0 comments

Most developers who have written a Django application are familiar with the admin interface. In this post I'll talk about the way the admin module uses a registration pattern to allow tools like admin.autodiscover() and admin.site.urls to do their magic.

Registration patterns are useful when developing flexible and extensible libraries. By specifying an interface and allowing you to register your custom implementations, the library code remains decoupled from your own custom code.

To get an idea of how these patterns work, let's take a look at the django.contrib.admin.sites module, we find a class called AdminSite which is instantiated at the bottom of the file (essentially a singleton that is used by default across your apps). The first lines of the __init__ method reveal that at the heart of this class, there's an attribute called _registry, which is a dictionary of Model classes and ModelAdmin instances.

def __init__(self, name=None, app_name='admin'):
    self._registry = {} # model_class class -> admin_class instance

When we import admin and run admin.site.register(), the register method on AdminSite is called, which performs some validation and then adds the model/modeladmin to its internal dictionary:

# Instantiate the admin class to save in the registry
self._registry[model] = admin_class(model, self)

When we include admin.site.urls in the urlconf, the urls property refers to the method get_urls(), at the bottom of which is this code:

# Add in each model's views.
for model, model_admin in self._registry.iteritems():
    urlpatterns += patterns('',
        url(r'^%s/%s/' % (model._meta.app_label, model._meta.module_name),
            include(model_admin.urls))
    )

In this way, URL patterns are created for all the models we've registered. Inside the ModelAdmin, there is a property called urls which similarly refers to a get_urls() method - this method exposes the CRUD views. Thus we have a system whereby any number of models can register admin classes with the admin system. We don't have to create url patterns for all the models we wish to use, and we can lean on the ModelAdmin class to provide the functionality we need, extending it as much or as little as we want without having to change the way the AdminSite works.

The last bit of the admin I want to talk about is its autodiscover() method, which lives in contrib.admin.__init__. On line 4, we can see that it is importing the AdminSite class and the singleton site instance:

from django.contrib.admin.sites import AdminSite, site

Here is the code that does the importing. It figures out the path of each app in INSTALLED_APPS, looks for an admin module inside each package, and finally imports it. When the admin module is imported, all those calls to admin.site.register() get executed and the AdminSite's internal registry gets populated.

import imp
from django.conf import settings

for app in settings.INSTALLED_APPS:
    # For each app, we need to look for an admin.py inside that app's
    # package. We can't use os.path here -- recall that modules may be
    # imported different ways (think zip files) -- so we need to get
    # the app's __path__ and look for admin.py on that path.

    # Step 1: find out the app's __path__ Import errors here will (and
    # should) bubble up, but a missing __path__ (which is legal, but weird)
    # fails silently -- apps that do weird things with __path__ might
    # need to roll their own admin registration.
    try:
        app_path = import_module(app).__path__
    except AttributeError:
        continue

    # Step 2: use imp.find_module to find the app's admin.py. For some
    # reason imp.find_module raises ImportError if the app can't be found
    # but doesn't actually try to import the module. So skip this app if
    # its admin.py doesn't exist
    try:
        imp.find_module('admin', app_path)
    except ImportError:
        continue

    # Step 3: import the app's admin file. If this has errors we want them
    # to bubble up.
    import_module("%s.admin" % app)
# autodiscover was successful, reset loading flag.
LOADING = False

Another approach: Metaclasses and Django's Models

Django's models are a pretty awesome feat of programming. They, too, implement a registry pattern. Taking a look at django.db.models.base, at the top of the file there's a class definition for ModelBase, which subclasses 'type'. It overrides the new method of type, adding custom behavior to all models (which specify ModelBase as their metaclass) when they are created. At the bottom of the __new__ method, on line 190, there is the following:

register_models(new_class._meta.app_label, new_class)
#...
return get_model(new_class._meta.app_label, name, False)

Following the code to db.models.loading, we see a class definition for an object called AppCache and at the bottom of the file that class being instantiated (cache = AppCache()) - this is similar to the way AdminSite works. When register_models() is called in ModelBase, it populates AppCache's internal registry (the "app_models" attribute) with a key/value of app_label -> SortedDict(), which in turn is a dictionary of model name / model class pairs. The __new__ method of ModelBase calls its parent-class' __new__ method to create a class, adds a ton of functionality to that newly-created class, and then registers it with AppCache. If you look at Model's class definition, which follows ModelBase in db.models.base, you will see:

class Model(object):
    __metaclass__ = ModelBase

Whenever a subclass of Model is created, it uses ModelBase to create itself, allowing all this functionality to happen automatically, behind-the-scenes. To see how this stuff gets used, take a look at syncdb's code (core.management.commands.syncdb). It calls get_apps() on the AppCache, which is a SortedDict of app_label -> SortedDict(), which in turn is a dictionary of model name / model classes:

# Create the tables for each model
for app in models.get_apps():
    app_name = app.__name__.split('.')[-2]
    model_list = models.get_models(app)
    for model in model_list:
        # Create the model's database table, if it doesn't already exist.
        # ...

Line 4 of django.db.models.init imports AppCache functions, so they are available directly from django.db.models:

from django.db.models.loading import get_apps, get_app, get_models, get_model, register_models

This is a very powerful pattern, and it is talked about in greater detail in Marty Alchin's book Pro Django.

The plug-in approach to metaclasses

A similar approach to using Metaclasses is discussed in Pro Django as a way of creating a plug-in system. A real-life example of this technique can be found in the code for oohEmbed.

Looking at line 7 of base.py, there is the following Metaclass definition:

class ProviderMount(type):
    def __init__(cls, name, bases, attrs):
        if not hasattr(cls, 'plugins'):
            cls.plugins = []
        else:
            cls.plugins.append(cls)

    def get_providers(self, *args, **kwargs):
        return [p(*args, **kwargs) for p in self.plugins]

It does not override the __new__ method, as ModelBase does, but it does do some interesting stuff in the __init__. It creates a list attribute on the class called plugins when Provider is defined, then for all subclasses of Provider the metaclass adds the class being created to this list. When get_providers is called, it iterates over the plugins, instantiating them and returing them as a list.

class Provider:
    __metaclass__ = ProviderMount

The Provider class specifies ProviderMount as its metaclass, and taking a look at videoprovider.py, on line 11 a YouTubeProvider is defined, inheriting from Provider. In the app's main.py, Provider is imported and all the provider subclasses are retrieved and instantiated:

from provider import *

class EndPoint(webapp.RequestHandler):
    providers = Provider.get_providers()

Whenever a request comes in, the get handler iterates over the providers, searching for the right one.

for p in self.providers:

Implementing it yourself

The way you choose to implement this pattern depends on your needs. The Admin method has an explicit registration step, as well as the ability to programmatically unregister a model. I've seen this technique used to unregister a default ModelAdmin (for example one included in contrib) and register your own version that adds some functionality. The metaclass method removes the explicit step of registration/discovery and handles autodiscovery at class-creation, but still relies on import-time side-effects.

However you end up doing it, I hope this post was informative! Any corrections / points of clarification are appreciated.

Comments (0)

Commenting has been closed.

Looking at registration patterns in Django

Another approach: Metaclasses and Django's Models

The plug-in approach to metaclasses

Implementing it yourself

Further Reading

Comments (0)