Task queues are frequently deployed alongside websites to do background processing outside the normal request/response cycle. In the past I've used them for things like sending emails, generating thumbnails, warming caches, or periodically fetching remote resources. By pushing that work out of the request/response cycle, you can increase the throughput (and responsiveness) of your web application.
Depending on your workload, though, it may be possible to move your task processing into the same process as your web server. In this post I'll describe how I did just that using gevent, though the technique would probably work well with a number of different WSGI servers.
In this post I'd like to share with you some techniques for effectively working with SQLite using Python. SQLite is a capable library, providing an in-process relational database for efficient storage of small-to-medium-sized data sets. It supports most of the common features of SQL with few exceptions. Best of all, most Python users do not need to install anything to get started working with SQLite, as the standard library in most distributions ships with the sqlite3 module.
I thought I had arrived. One of my open-source projects started gaining a small following on GitHub. No more nagging self-doubt, the thousand or so star-gazers of my project provided all the validation I needed. Here was something I could conjure up in moments of doubt, reminding myself that I truly was all those things I wanted to believe about myself.
* * *
All this got turned on it's head, though, by one of those very people who I set so much store by. I'm thinking in particular of one person who was using my project to manage the backend data-storage for his company's platform. It was a critical function, and as the CTO of his organization, he was responsible for ensuring it was technically sound. He was very invested, professionally, in the direction of my project. This was a sharp contrast to most people I'd talked to, who were using my project for side-projects and hobbies of their own.
The combination of his expectations of me, as a maintainer, and my beliefs about my own motivations for sharing my code led to a pretty unbelievable series of events.
Python3 is a mess. How did this happen? So many of the changes seem to me to fly in the face of the whole Zen of Python aesthetic. The two biggest offenders, in my opinion, are asyncio and type hints.
SQLite's write lock and pysqlite's clunky transaction state-machine are a toxic combination for multi-threaded applications. Unless you are very diligent about keeping your write transactions as short as possible, you can easily wind up with one thread accidentally holding a write transaction open for an unnecessarily long time. Threads that are waiting to write will then have a much greater likelihood of timing out while waiting for the lock, giving the illusion of poor performance.
In this post I'd like to share a very effective technique for performing writes to a SQLite database from multiple threads.
In this post I'll share a simple code snippet you can use to perform optimistic locking when updating model instances. I've intentionally avoided providing an implementation for this in peewee, because I don't believe it will be easy to find a one-size-fits-all approach to versioning and conflict resolution. I've updated the documentation to include the sample implementation provided here, however.
For the past six months or so, I've been experimenting with a variety of monospace fonts in a quest to find the perfect coding font. While I haven't found a clear winner, I have found a dozen nice-looking fonts and learned a lot about typefaces in general. I've also learned quite a bit about font rendering on Linux, which I hope to summarize in a separate post soon.
In this post I'd like to share some screenshots (or "swatches") of my favorite fonts.
It's been over 2 years since I wrote about the tooling I use to theme my desktop, so I thought I'd post about my current scripts...