Playing with Python Magic Methods to make a nicer Regex API

july 19, 2014 11:24am / python regex / 3 comments


A co-worker of mine mentioned that he missed Ruby's syntactic sugar for regular expressions. I haven't used Ruby's regular expressions, but I'm familiar enough with Python's to know that the API is a bit wanting in syntactic sweetness.

In this post I'll show how you might use python's magic methods to make a nicer API for working with regular expressions.


SQLite: Small. Fast. Reliable. Choose any three.

july 14, 2014 12:37pm / berkeleydb peewee python sqlite / 10 comments

Sqlite Logo

SQLite is a fantastic database and in this post I'd like to explain why I think that, for many scenarios, SQLite is actually a great choice. I hope to also clear up some common misconceptions about SQLite.


Completely un-scientific benchmarks of some embedded databases with Python

june 29, 2014 01:31am / nosql python / 5 comments

I've spent some time over the past couple weeks playing with the embedded NoSQL databases Vedis and UnQLite. Vedis, as its name might indicate, is an embedded data-structure database modeled after Redis. UnQLite is a JSON document store (like MongoDB, I guess??). Beneath the higher-level APIs, both Vedis and UnQLite are key/value stores, which puts them in the same category as BerkeleyDB, KyotoCabinet and LevelDB. The Python standard library also includes some dbm-style databases, including gdbm.

For fun, I thought I would put together a completely un-scientific benchmark showing the relative speeds of these various databases for storing and retrieving simple keys and values.

Here are the databases and drivers that I used for the test:

I'm running these tests with:

For the test, I simply recorded the time it took to store 100K simple key/value pairs (no collisions). Then I recorded the time it took to read back all these values. The results are in seconds elapsed:



Python bindings for UnQLite, an embedded NoSQL database/JSON document store

june 26, 2014 11:46am / nosql python unqlite / 1 comments

unqlite python logo

I'm happy to write that I've just released some python bindings for UnQLite, an embedded NoSQL database and JSON document store. UnQLite might be characterized as the SQLite of NoSQL databases, though it's JSON document-store and Jx9 scripting language make it a pretty unique offering. UnQLite is created by Symisc Systems, who are also responsible for Vedis, an embedded Redis-like database (I also wrote some python bindings for vedis). Here is a quick overview of some of UnQLite's features, as described on the project homepage:

In the rest of this post I will show some basic usage of the unqlite-python library. If you'd like to follow along, you can use pip to install unqlite:

pip install unqlite

You can find the project source code hosted on GitHub and the documentation is available on readthedocs.

Read on for the details!


Python bindings for Vedis, the Embedded NoSQL Database

june 19, 2014 10:22am / nosql python vedis / 0 comments

vedis-python logo

Over the past week I've been writing some python bindings to the embedded NoSQL database Vedis, a transactional data-store modeled after Redis. Like Redis, Vedis could be characterized as an advanced key-value store that supports hash, set and list data-structures. Vedis has over 70 available commands for working with the various data types. Unlike Redis, which is run as a separate server process, Vedis is embedded in the host process like SQLite. Vedis works with either in-memory databases or on-disk databases. Vedis is transactional (ACID) and also thread-safe. If you'd like more information, check out the Vedis FAQ.


Vedis-python allows you to use Vedis in your Python apps. Vedis-python supports all the Vedis data-types, and also allows you extend Vedis by writing your own commands in Python. As I mentioned, this project is very new so while I have written pretty extensive unit tests, the library has certainly not been battle-tested yet.

If you'd like to give it a try, you can use pip to install vedis-python. At the time of writing the current version is 0.1.5.

$ pip install vedis-python

Just a word of caution, I've tested the installation on various flavors of Linux (including on my raspberry pi), and Mac OSX, but have not tested on Windows.

Read the rest of the post for the details.


jist: a command-line utility for managing multi-file, multi-directory private gists

june 04, 2014 02:48pm / gist python / 2 comments

Gist logo

I'd like to share a little command-line utility I wrote for managing multi-file and multi-directory private gists on GitHub. If you're not familiar with GitHub Gist, it's basically a git-backed pastebin. One of the benefits of Gist is that it supports private gists for free, allowing you to create private repos for your code snippets. To prevent abuse, GitHub does not allow you to create gists containing subdirectories.

I like to keep my list of public GitHub repositories very tidy, so I frequently use Gists for smaller projects. Last week I wanted to share the code for the note-taking app I blogged about. I didn't want to put the code into a GitHub repo, so I decided to create a gist. Unfortunately, the project contained templates and javascript that needed to go in subdirectories. To work around Gist's subdirectory restriction I used a naming convention to indicate that these files belonged in subdirectories, e.g.:

Then I had a lightbulb moment -- why not write a script to do all this automatically?

Read the rest of the post for the details.


Saturday morning hacks: Revisiting the notes app

may 30, 2014 04:17pm / flask javascript peewee python saturday-morning-hacks / 1 comments

Saturday morning hacks

My post from last month, Saturday Morning Hack, a Little Note-Taking App with Flask, was pretty well-received. Since I've made a number of improvements to the app, I thought I would write one more post to share some of the updates I've made to this project, in the hopes that they may be of interest to you.

A live demo is up and running on Python Anywhere, so feel free to check that out before continuing on with the post:

To briefly recap the previous post, I discussed how I built a lightweight note-taking app which I could use from my phone or desktop. It has a nice ajax-ey interface and some simple markdown helpers written with javascript. In addition to supporting markdown, it also supports oembed for automatically embedding YouTube videos and the like. Here is what it looked like when we left off a few weeks ago:

Notes on Desktop

And this is how it looks now!

New and improved notes app

So what's new? Well, I've made a couple changes under-the-hood, and added some entirely new features to the UI.

This was super fun to hack on so I thought I'd share the new code and describe how I added these features. Honestly, I didn't really end up adding much in terms of implementation. Huey handles scheduling and sending the email reminders, even automatically retrying messages that fail to send. Similarly, Flask-Peewee's REST API provides search and pagination out-of-the-box, so all I had to do was write the JavaScript to communicate with it. Thanks to these libraries, I was able to focus on the things that made this project unique, and hopefully you enjoy reading about the code.

Read the rest of the post for the details.


That one time I used a coroutine

may 26, 2014 12:55pm / coroutine python / 0 comments

Coroutines are an interesting feature of the Python language that I have, in practice, found very little occasion to use. Then one day I was stuck working on a problem and I realized suddenly that coroutines were exactly the thing I needed. In this post I'll describe how coroutines helped me to solve a tricky problem.


David Beazley has written an excellent guide to coroutines in Python titled A Curious Course on Coroutines and Concurrency. I strongly recommend checking it out if this is your first brush with coroutines, or if it's just been a while and you'd like a refresher.


Saturday morning hack: Automatically import your app's models when starting IPython

may 18, 2014 02:45pm / python saturday-morning-hacks / 0 comments

Saturday morning hacks

I use Flask and peewee for all my personal projects, and wanted an easy way to automatically open an IPython shell with all my project's models in the namespace. If you've used the excellent django-extensions project, you may be familiar with the shell_plus command, which does the same thing. If your project has multiple models spread across several modules, this kind of hack can save you a lot of keystrokes.


Using SQLite Full-Text Search with Python

may 12, 2014 07:12pm / peewee python search sqlite / 0 comments

Full-text search with SQLite

In this post I will show how to use SQLite full-text search with Python (and a lot of help from peewee ORM). We will see how to index content for searching, and how to order search results using two ranking algorithms.

Last week I migrated my site from Postgresql to SQLite. I had been using Redis to power my site's search, but since SQLite has an awesome full-text search extension, I decided to give it a try. I am really pleased with the results, and being able to specify boolean search queries is an added plus. Here is a brief overview of the types of search queries SQLite supports:

Check out the full post for details on adding full-text search to your project.