Saturday morning hacks: DataSet for Peewee

october 17, 2014 05:39pm / peewee python saturday-morning-hacks / 0 comments

Saturday morning hacks

I recently became acquainted with the dataset project while browsing a curated list of awesome SQLAlchemy resources. I was intrigued by the project's simplicity, and apparently I'm not the only one as the project has quite a few followers on GitHub. Since peewee has the ingredients required to provide a similar API (reflection, schema migrations), I decided it might be a fun project to add a DataSet-like module to the playhouse extension collection.

Taking a look at DataSet

I feel like the best way to explain DataSet is to show it in action, so let's take a look at what it can do.

>>> from playhouse.dataset import DataSet

>>> db = DataSet('sqlite:///:memory:')

>>> people = db['people']  # This will create the people table.

>>> people.insert(name='charlie', gender='M')
1
>>> people.insert(name='huey', gender='M', favorite_color='blue')
2

>>> list(people)
[{'favorite_color': None, 'gender': u'M', 'id': 1, 'name': u'charlie'},
 {'favorite_color': u'blue', 'gender': u'M', 'id': 2, 'name': u'huey'}]

>>> people.update(favorite_color='green', name='charlie', columns=['name'])
1

>>> people.find_one(name='charlie')['favorite_color']
u'green'

Read more...


Saturday morning hacks: Adding full-text search to the flask note-taking app

october 08, 2014 11:57am / flask peewee python saturday-morning-hacks / 0 comments

Saturday morning hacks

In preparation for the fourth and final installment in the "Flask Note-taking app" series, I found it necessary to improve the search feature of the note-taking app. In this post we will use SQLite's full-text search extension to improve the search feature.

To recap, the note-taking app provides a lightweight interface for storing markdown-formatted notes. Because I frequently find myself wanting to take notes on the spur of the moment, the note-taking app needed to be very mobile-friendly. By using twitter bootstrap and a hefty dose of JavaScript, we made an app that matches our spec and manages to look good doing it!

In part 2, we added email reminders and check-able task lists to the note-taking app. We also converted the backend to use flask-peewee's REST API extension, which made it easy to add pagination and search. And that is how I've left it for the last three months or so.

Below is a screenshot of the latest version of the notes app. The UI is much cleaner thanks to a stylesheet from bootswatch. The bootswatch stylesheet works as a drop-in replacement for the default bootstrap CSS file.

photos/p1412692599.23.png

All together, the note-taking app has the following features:

You can browse or download the finished code from part 2 in this gist. If you're in a hurry, you can find all the code from this post in this gist.

In case you were curious, I've been using the notes app for things like:

Read more...


Saturday morning hacks: Building an Analytics App with Flask

september 30, 2014 10:40am / flask peewee python saturday-morning-hacks / 5 comments

Saturday morning hacks

A couple years back I wrote about building an Analytics service with Cassandra. As fun as that project was to build, the reality was that Cassandra was completely unsuitable for my actual needs, so I decided to switch to something simpler. I'm happy to say the replacement app has been running without a hitch for the past 5 months taking up only about 20 MB of RAM! In this post I'll show how to build a lightweight Analytics service using Flask.

Read more...


Postgresql HStore, JSON data-type and Arrays with Peewee ORM

september 12, 2014 11:13pm / peewee postgresql python / 2 comments

I've developed an interest in some of the more advanced features of SQLite after reading the O'Reilly title Using SQLite (Small. Fast. Reliable. Choose Any Three). For personal projects I like using SQLite, but when I need something more powerful I turn to Postgresql. Because peewee supports both of these databases (as well as MySQL), it is limited to a lowest-common-denominator feature set. While this encompasses a broad range of features, each database engine has its own extensions and I've been interested in adding some pythonic support for the cooler extensions.

Here are some of the fun things you can find in peewee's playhouse (collection of extensions):

This post will showcase the peewee support for HStore, JSON document store, and arrays. I've written elsewhere about SQLite's full-text search, so if you're a SQLite user you may want to check out that post.

To follow along at home, feel free to install peewee:

pip install peewee

Read more...


Playing with Python Magic Methods to make a nicer Regex API

july 19, 2014 11:24am / python regex / 5 comments

p1405787444.31.png

A co-worker of mine mentioned that he missed Ruby's syntactic sugar for regular expressions. I haven't used Ruby's regular expressions, but I'm familiar enough with Python's to know that the API is a bit wanting in syntactic sweetness.

In this post I'll show how you might use python's magic methods to make a nicer API for working with regular expressions.

Read more...


SQLite: Small. Fast. Reliable. Choose any three.

july 14, 2014 12:37pm / berkeleydb peewee python sqlite / 10 comments

Sqlite Logo

SQLite is a fantastic database and in this post I'd like to explain why I think that, for many scenarios, SQLite is actually a great choice. I hope to also clear up some common misconceptions about SQLite.

Read more...


Completely un-scientific benchmarks of some embedded databases with Python

june 29, 2014 01:31am / nosql python / 5 comments

I've spent some time over the past couple weeks playing with the embedded NoSQL databases Vedis and UnQLite. Vedis, as its name might indicate, is an embedded data-structure database modeled after Redis. UnQLite is a JSON document store (like MongoDB, I guess??). Beneath the higher-level APIs, both Vedis and UnQLite are key/value stores, which puts them in the same category as BerkeleyDB, KyotoCabinet and LevelDB. The Python standard library also includes some dbm-style databases, including gdbm.

For fun, I thought I would put together a completely un-scientific benchmark showing the relative speeds of these various databases for storing and retrieving simple keys and values.

Here are the databases and drivers that I used for the test:

I'm running these tests with:

For the test, I simply recorded the time it took to store 100K simple key/value pairs (no collisions). Then I recorded the time it took to read back all these values. The results are in seconds elapsed:

p1404105100.86.png

Read more...


Python bindings for UnQLite, an embedded NoSQL database/JSON document store

june 26, 2014 11:46am / nosql python unqlite / 1 comments

unqlite python logo

I'm happy to write that I've just released some python bindings for UnQLite, an embedded NoSQL database and JSON document store. UnQLite might be characterized as the SQLite of NoSQL databases, though it's JSON document-store and Jx9 scripting language make it a pretty unique offering. UnQLite is created by Symisc Systems, who are also responsible for Vedis, an embedded Redis-like database (I also wrote some python bindings for vedis). Here is a quick overview of some of UnQLite's features, as described on the project homepage:

In the rest of this post I will show some basic usage of the unqlite-python library. If you'd like to follow along, you can use pip to install unqlite:

pip install unqlite

You can find the project source code hosted on GitHub and the documentation is available on readthedocs.

Read on for the details!

Read more...


Python bindings for Vedis, the Embedded NoSQL Database

june 19, 2014 10:22am / nosql python vedis / 0 comments

vedis-python logo

Over the past week I've been writing some python bindings to the embedded NoSQL database Vedis, a transactional data-store modeled after Redis. Like Redis, Vedis could be characterized as an advanced key-value store that supports hash, set and list data-structures. Vedis has over 70 available commands for working with the various data types. Unlike Redis, which is run as a separate server process, Vedis is embedded in the host process like SQLite. Vedis works with either in-memory databases or on-disk databases. Vedis is transactional (ACID) and also thread-safe. If you'd like more information, check out the Vedis FAQ.

Vedis-python

Vedis-python allows you to use Vedis in your Python apps. Vedis-python supports all the Vedis data-types, and also allows you extend Vedis by writing your own commands in Python. As I mentioned, this project is very new so while I have written pretty extensive unit tests, the library has certainly not been battle-tested yet.

If you'd like to give it a try, you can use pip to install vedis-python. At the time of writing the current version is 0.1.5.

$ pip install vedis-python

Just a word of caution, I've tested the installation on various flavors of Linux (including on my raspberry pi), and Mac OSX, but have not tested on Windows.

Read the rest of the post for the details.

Read more...


jist: a command-line utility for managing multi-file, multi-directory private gists

june 04, 2014 02:48pm / gist python / 2 comments

Gist logo

I'd like to share a little command-line utility I wrote for managing multi-file and multi-directory private gists on GitHub. If you're not familiar with GitHub Gist, it's basically a git-backed pastebin. One of the benefits of Gist is that it supports private gists for free, allowing you to create private repos for your code snippets. To prevent abuse, GitHub does not allow you to create gists containing subdirectories.

I like to keep my list of public GitHub repositories very tidy, so I frequently use Gists for smaller projects. Last week I wanted to share the code for the note-taking app I blogged about. I didn't want to put the code into a GitHub repo, so I decided to create a gist. Unfortunately, the project contained templates and javascript that needed to go in subdirectories. To work around Gist's subdirectory restriction I used a naming convention to indicate that these files belonged in subdirectories, e.g.:

Then I had a lightbulb moment -- why not write a script to do all this automatically?

Read the rest of the post for the details.

Read more...