"djpeewee": use the peewee ORM with your Django models

November 19, 2013 22:40 / 5 comments

I sat down and started working on a new library shortly after posting about Django's missing API for generating SQL. djpeewee is the result, and provides a simple translate() function that will recursively translate a Django model graph into a set of "peewee equivalents". The peewee versions can then be used to construct queries which can be passed back into Django as a "raw query".

Here are a couple scenarios when this might be useful:

  • Joining on fields that are not related by foreign key (for example UUID fields).
  • Performing filters on calculated values.
  • Performing aggregate queries on calculated values.
  • Using SQL statements that Django does not support such as CASE.
  • Utilizing SQL functions that Django does not support, such as SUBSTR.
  • Replacing nearly-identical SQL queries with reusable, composable data-structures.

I've included this module in peewee's playhouse, which is bundled with peewee.


The search for the missing link: what lies between SQL and Django's ORM?

November 12, 2013 11:54 / 7 comments

I had the opportunity this week to write some fairly interesting SQL queries. I don't write "raw" SQL too often, so it was fun to use that part of my brain (by the way, does it bother anyone else when people call SQL "raw"?). At Counsyl we use Django for pretty much everything so naturally we also use the ORM. Every place I've worked there's a strong bias against using SQL when you've got an ORM on board, which makes sense -- if you choose a tool you should standardize on it if for no other reason than it makes maintenance easier.

So as I was saying, I had some pretty interesting queries to write and I struggled to think how to shoehorn them into Django's ORM. I've already written about some of the shortcomings of Django's ORM so I won't rehash those points. I'll just say that Django fell short and I found myself writing SQL. The queries I was working on joined models from very disparate parts of our codebase. The joins were on values that weren't necessarily foreign keys (think UUIDs) and this is something that Django just doesn't cope with. Additionally I was interested in aggregates on calculated values, and it seems like Django can only do aggregates on a single column.

As I was prototyping, I found several mistakes in my queries and decided to run them in the postgres shell before translating them into my code. I started to think that some of these errors could have been avoided if I could find an abstraction that sat between the ORM and a string of SQL. By leveraging the python interpreter, the obvious syntax errors could have been caught at module import time. By using composable data structures, methods I wrote that used similar table structures could have been more DRY. When I write less code, I think I generally write less bugs as well.

That got me started on my search for the "missing link" between SQL (represented as a string) and Django's ORM.


Becoming a Django Apologist

July 24, 2013 08:33 / 1 comments

It's been roughly four years since my introduction to the Django framework and I thought I'd write a little post to commemorate this. In my mind nothing had as big an impact on my career as my decision to work at the Journal World. When I started there I knew basically nothing about software engineering or open-source, and it is entirely thanks to my excellent (and patient) coworkers there that I was able to learn about these things.


Structuring flask apps, a how-to for those coming from Django

April 27, 2013 13:21 / 0 comments

The other day a friend of mine was trying out flask-peewee and he had some questions about the best way to structure his app to avoid triggering circular imports. For someone new to flask, this can be a bit of a puzzler, especially if you're coming from django which automatically imports your modules. In this post I'll walk through how I like to structure my flask apps to avoid circular imports. In my examples I'll be showing how to use "flask-peewee", but the same technique should be applicable for other flask plugins.

I'll walk through the modules I commonly use in my apps, then show how to tie them all together and provide a single entrypoint into your app.


The missing library: ad-hoc queries for your models

September 05, 2012 11:41 / 5 comments

I think it would be great if more sites allowed users (or consumers of their APIs) to produce and execute ad-hoc queries against their data. In this post I'll talk a little bit about some ways sites are currently doing this, some of the challenges involved, my experience trying to build something "reusable", and finally invite you to share your thoughts.


Working around Django's ORM to do interesting things with GFKs

May 03, 2012 00:05 / 0 comments

In this post I want to discuss how to work around some of the shortcomings of djangos ORM when dealing with Generic Foreign Keys (GFKs).

At the end of the post I'll show how to work around django's lack of correctly CAST-ing when the generic foreign key is of a different column type than the objects it may point to.


Micawber, a python library for extracting rich content from URLs

April 19, 2012 11:13 / 0 comments

A while ago I wrote about an awesome API for retrieving metadata about URLs called oembed. I'm writing to announce a new project I've been working on called micawber, which is very similar but with a cleaner API and not restricted to django projects.


So long, djangosnippets, and thanks for all the fish

March 06, 2012 09:38 / 0 comments

After two years of maintaining djangosnippets.org, I am pleased to announce that the guys from django-de are going to be taking over and you can expect to see some real improvements.


Integrating the flask microframework with the peewee ORM

September 27, 2011 10:52 / 5 comments

I'd like to write a post about a project I've been working on for the past month or so. I've had a great time working on it and am excited to start putting it to use. The project is called flask-peewee -- it is a set of utilities that bridges the python microframework flask and the lightweight ORM peewee. It is packaged as a flask extension and comes with the following batteries included:


Suggesting tags with django-taggit and jQuery UI

June 29, 2011 12:20 / 2 comments

One of the problems mentioned by a couple people when I asked for suggestions on improving djangosnippets.org was the proliferation of tags. This is a well-known problem on sites that allow users to enter their own tags, where misspellings are frequent and its sometimes unclear whether a tag should be plural or singular.

To try and reduce the amount of different tags on djangosnippets I ended up using the jQuery UI autocomplete tools to provide users with hints when they enter tags for their snippets.