A couple years back I wrote about building an Analytics service with Cassandra. As fun as that project was to build, the reality was that Cassandra was completely unsuitable for my actual needs, so I decided to switch to something simpler. I'm happy to say the replacement app has been running without a hitch for the past 5 months taking up only about 20 MB of RAM! In this post I'll show how to build a lightweight Analytics service using Flask.
SQLAlchemy is one of those tools that is so powerful and so well-established that it's hard to imagine a time when it didn't exist. Mike Bayer, the project's primary author, after what I'm sure was a lot of real-world experience, hit on the right amount of abstraction and API that pleases both die-hard DBA types as well as more casual users.
So why would anyone use peewee?
I've developed an interest in some of the more advanced features of SQLite after reading the O'Reilly title Using SQLite (Small. Fast. Reliable. Choose Any Three). For personal projects I like using SQLite, but when I need something more powerful I turn to Postgresql. Because peewee supports both of these databases (as well as MySQL), it is limited to a lowest-common-denominator feature set. While this encompasses a broad range of features, each database engine has its own extensions and I've been interested in adding some pythonic support for the cooler extensions.
Here are some of the fun things you can find in peewee's playhouse (collection of extensions):
This post will showcase the peewee support for HStore, JSON document store, and arrays. I've written elsewhere about SQLite's full-text search, so if you're a SQLite user you may want to check out that post.
To follow along at home, feel free to install peewee:
pip install peewee
SQLite is a fantastic database and in this post I'd like to explain why I think that, for many scenarios, SQLite is actually a great choice. I hope to also clear up some common misconceptions about SQLite.
My post from last month, Saturday Morning Hack, a Little Note-Taking App with Flask, was pretty well-received. Since I've made a number of improvements to the app, I thought I would write one more post to share some of the updates I've made to this project, in the hopes that they may be of interest to you.
A live demo is up and running on Python Anywhere, so feel free to check that out before continuing on with the post: http://beetlejuicer.pythonanywhere.com/
And this is how it looks now!
So what's new? Well, I've made a couple changes under-the-hood, and added some entirely new features to the UI.
Notemodel. Thanks to flask-peewee everything comes "for free".
Read the rest of the post for the details.
In this post I will show how to use SQLite full-text search with Python (and a lot of help from peewee ORM). We will see how to index content for searching, and how to order search results using two ranking algorithms.
Last week I migrated my site from Postgresql to SQLite. I had been using Redis to power my site's search, but since SQLite has an awesome full-text search extension, I decided to give it a try. I am really pleased with the results, and being able to specify boolean search queries is an added plus. Here is a brief overview of the types of search queries SQLite supports:
NEAR: peewee NEAR sqlite would return docs containing the words peewee and sqlite with no more than
10intervening words. You can also specify the max number of intervening words, e.g. peewee NEAR/3 sqlite.
NOT: sqlite OR postgresql AND NOT mysql would return docs about high-quality databases (just trollin).
Check out the full post for details on adding full-text search to your project.
Because I had so much fun writing my last Saturday morning hack, I thought I would share another little hack. I was thinking that I really enjoy my subscription to Python weekly and wouldn't it be great if I had a personal email digest containing just the types of things that interest me? I regularly cruise reddit and
hacker hater news but in my opinion there's a pretty low signal-to-noise ratio. Occasionally I stumble on fascinating content and that's what keeps me coming back.
I wanted to write an app that would search the sites I read and automatically create an email digest based on search terms that I specified. I recently swapped my blog over to SQLite and I love that the SQLite full-text search extension lets you specify boolean queries. With that in mind, I decided that I would have a curated list of boolean search queries which would be used to filter content from the various sites I read. Any articles that match my search would then be emailed to me.
Here are some of my search terms, which I am viewing in the flask-peewee admin interface:
If you're interested in learning how to build your own version of this project, check out the rest of the post.
I made the decision this week to migrate my personal sites and several other sites I host onto SQLite. Previously almost everything I hosted had been using Postgresql. The move was motivated by a couple factors:
At times it has seemed to me that there is a tacit agreement within the Flask / Django communities that if you're using SQL you should be using Postgresql. Postgresql is an amazing piece of engineering. I have spent the last five years of my career working exclusively with it, and I am continually impressed by its performance and the constant stream of great new features.
So why change things?
Well, as my list indicates, there are a handful of reasons. But the primary reason was that I wanted something lightweight. I'm running a fairly low-traffic, read-heavy site, so Postgresql was definitely overkill. My blog is deployed on a VPS with very limited resources, so every MB of RAM counts. Additionally, I wasn't using any special Postgresql features so there was nothing holding me back.
A couple Saturdays ago I spent the morning hacking together a note-taking app. I'm really pleased with the result, so I thought I'd share the code in case anyone else might find it useful.
The note-taking project idea came about out of necessity -- I wanted something that worked well from my phone. While I have a personal wiki site I've used for things like software installation notes or salsa recipes, I've also noticed that because it's so cumbersome to use from my phone, I often end up emailing things to myself. Plus a wiki implies a kind of permanence to the content, making it not a great fit for these impromptu notes. I also like to use markdown to format notes, but markdown isn't too easy on a phone because of the special characters or the need to indent blocks of text. With these considerations in mind, I set out to build a note-taking app that would be easy to use from my phone.
Here is how the app appears on a narrow screen like my phone:
And here it is on my laptop:
Because markdown is a bit difficult to use when you're not in a nice text editor like vim, I've added some simple toolbar buttons to the editor:
Read the full post for all the details!
I'm pleased to announce that peewee now has more robust support for schema migrations. Basic migration support for Postgres has existed for some time, but peewee now supports SQLite and MySQL as well. In addition, there were a couple of issues with the original migrations API. Schema migrations have been one of the most-requested features, so I'm hopeful that with the addition of this feature, peewee users will have one more reason to enjoy using the library!
Check out the rest of the post for more details.