How do you use peewee?

February 22, 2014 13:11 / 5 comments

When I first wrote peewee I set out to accomplish a simple task: make it easy to execute queries in my Flask apps. I was a bit familiar with SQLAlchemy, but wanted something lightweight and thought it would be a quick project. While the first version only took a couple days to write, over the past two or three years peewee has been my favorite project to work on. I've been very surprised to see that it's user base has grown, and would like to ask anyone who is using peewee:

How do you use peewee?

I'd like to add a "testimonials" section to the documentation that describes the interesting projects people have written using peewee. If you don't mind sharing, I'd love to hear about your project.


Window functions, case statements, and savepoints in peewee

February 21, 2014 10:44 / 0 comments

In case you've missed the last few releases, I've been busy adding some fun new features to peewee. While the changelog and the docs explain the new features and describe their usage, I thought I'd write a blog post to provide a bit more context.

Most of these features were requested by peewee users. I depend heavily on users like you to help me improve peewee, so thank you very much! Not only have your feature requests helped make peewee a better library, they've helped me become a better programmer.

So what's new in peewee? Here is something of an overview:

Hopefully some of those things sound interesting. In this post I will not be discussing everything, but will hit some of the highlights.


"djpeewee": use the peewee ORM with your Django models

November 19, 2013 22:40 / 5 comments

I sat down and started working on a new library shortly after posting about Django's missing API for generating SQL. djpeewee is the result, and provides a simple translate() function that will recursively translate a Django model graph into a set of "peewee equivalents". The peewee versions can then be used to construct queries which can be passed back into Django as a "raw query".

Here are a couple scenarios when this might be useful:

  • Joining on fields that are not related by foreign key (for example UUID fields).
  • Performing filters on calculated values.
  • Performing aggregate queries on calculated values.
  • Using SQL statements that Django does not support such as CASE.
  • Utilizing SQL functions that Django does not support, such as SUBSTR.
  • Replacing nearly-identical SQL queries with reusable, composable data-structures.

I've included this module in peewee's playhouse, which is bundled with peewee.


The search for the missing link: what lies between SQL and Django's ORM?

November 12, 2013 11:54 / 7 comments

I had the opportunity this week to write some fairly interesting SQL queries. I don't write "raw" SQL too often, so it was fun to use that part of my brain (by the way, does it bother anyone else when people call SQL "raw"?). At Counsyl we use Django for pretty much everything so naturally we also use the ORM. Every place I've worked there's a strong bias against using SQL when you've got an ORM on board, which makes sense -- if you choose a tool you should standardize on it if for no other reason than it makes maintenance easier.

So as I was saying, I had some pretty interesting queries to write and I struggled to think how to shoehorn them into Django's ORM. I've already written about some of the shortcomings of Django's ORM so I won't rehash those points. I'll just say that Django fell short and I found myself writing SQL. The queries I was working on joined models from very disparate parts of our codebase. The joins were on values that weren't necessarily foreign keys (think UUIDs) and this is something that Django just doesn't cope with. Additionally I was interested in aggregates on calculated values, and it seems like Django can only do aggregates on a single column.

As I was prototyping, I found several mistakes in my queries and decided to run them in the postgres shell before translating them into my code. I started to think that some of these errors could have been avoided if I could find an abstraction that sat between the ORM and a string of SQL. By leveraging the python interpreter, the obvious syntax errors could have been caught at module import time. By using composable data structures, methods I wrote that used similar table structures could have been more DRY. When I write less code, I think I generally write less bugs as well.

That got me started on my search for the "missing link" between SQL (represented as a string) and Django's ORM.


Using peewee to explore CSV files

November 07, 2013 06:19 / 3 comments

I recently heard a talk from a coworker wherein one of the things he discussed was automatically converting CSV data for use with a SQLite database. I thought this would be a great thing to add to peewee, especially as lately I've found myself on several occasions working with CSV and battling with it in a spreadsheet. It would be much easier to load it into a database and then query it using a tool I'm familiar with.

Which brings me to playhouse.csv_loader, a new module I've added to the playhouse package of extras. It's hopefully really easy to use. Here is an example of how you might use it:

>>> from playhouse.csv_loader import *
>>> db = SqliteDatabase(':memory:')  # Create an in-memory sqlite database

# Load the CSV file into the in-memory database and return a Model suitable
# for querying the data.
>>> ZipToTZ = load_csv(db, 'zipcode_to_timezone.csv')

# Get the timezone for a zipcode.
>>> ZipToTZ.get(ZipToTZ.zip == 66047).timezone
'US/Central'

# Get all the zipcodes for my town.
>>> [row.zip for row in ZipToTZ.select().where(
...     (ZipToTZ.city == 'Lawrence') && (ZipToTZ.state == 'KS'))]
[66044, 66045, 66046, 66047, 66049]

Structuring flask apps, a how-to for those coming from Django

April 27, 2013 13:21 / 0 comments

The other day a friend of mine was trying out flask-peewee and he had some questions about the best way to structure his app to avoid triggering circular imports. For someone new to flask, this can be a bit of a puzzler, especially if you're coming from django which automatically imports your modules. In this post I'll walk through how I like to structure my flask apps to avoid circular imports. In my examples I'll be showing how to use "flask-peewee", but the same technique should be applicable for other flask plugins.

I'll walk through the modules I commonly use in my apps, then show how to tie them all together and provide a single entrypoint into your app.


Peewee was baroque, so I rewrote it

October 08, 2012 09:25 / 7 comments

Today I merged in the "unstable/2.0" branch of peewee. I'm very excited about the changes and I hope you will be, too.

I have written a documentation page on upgrading which gives the rationale behind the rewrite and some examples of the new querying API. Please feel free to take a look but much of the information presented in this post is lifted directly from the docs.

Goals for the rewrite

  • consistent: there is one way of doing things
  • expressive: things can be done that I never thought of

What changed?

The biggest changes between 1.0 and 2.0 are in the syntax used for constructing queries. The first iteration of peewee I threw up on github was about 600 lines. I was passing around strings and dictionaries and as time went on and I added features, those strings turned into tuples and objects. This meant, though, that I needed code to handle all the possible ways of expressing something. Look at the code for parse_select.

I learned a valuable lesson: keep data in datastructures until the absolute last second.

With the benefit of hindsight and experience, I decided to rewrite and unify the API a bit. The result is a tradeoff. The newer syntax may be a bit more verbose at times, but at least it will be consistent.


Using advanced database features with Peewee, a python ORM

July 01, 2012 01:30 / 0 comments

I've developed an interest in some of the more advanced features of SQLite 3.7 after reading the O'Reilly title Using SQLite (Small. Fast. Reliable. Choose Any Three). For personal projects I like using SQLite for prototyping or for simple applications, and when I need something more powerful I turn to Postgresql. Because peewee supports both of these databases (as well as MySQL), it is limited to a lowest-common-denominator feature set. While this encompasses a broad range of features, each database engine has its own extensions and I've been interested in adding some pythonic support for the cooler extensions.

Currently, I've got support for:

This post will show the usage of the hstore and full-text search extensions. I will also show how I went about writing these extension modules so if you're interested in writing your own you will have a good foundation.

All of these extensions live in the playhouse package, included with the current master branch of peewee.

To follow along at home, feel free to install peewee:

pip install -e git+https://github.com/coleifer/peewee.git#egg=peewee

Model code generation with peewee

March 20, 2012 15:03 / 0 comments

For fun I put together a small script that is capable of introspecting databases and generating peewee models. I borrowed the crucial bits from django's codebase, which has methods for introspecting column types and foreign key constraints.

The code is hopefully rather straightforward - it simply grabs the list of tables, column type information which is then mapped to peewee field types, then finally resolves foreign keys. The generated models are then dumped to standard out, along with a database declaration.


Redesign of flask-peewee admin

October 28, 2011 15:44 / 0 comments

Recently I stumbled across the twitter bootstrap project, which is a set of cross-browser compliant stylesheets and scripts. I liked them so much that I've ported the admin templates to use bootstrap. Here's a little screenshot of the design refresh taken from the example app:

http://media.charlesleifer.com/images/photos/flask-peewee-admin.jpg

I hope this will make the admin easier to work with in the long-run!