Shortcomings in the Django ORM and a look at Peewee, a lightweight alternative December 15, 2012 14:27 / 13 comments
In this post I'd like to talk about some of the shortcomings of the Django ORM, the ways peewee approaches things differently, and how this resulted in peewee having an API that is both more consistent and more expressive.
Sharing Screenshots with Dropbox and Imgur November 28, 2012 11:09 / 2 comments
I saw a post on hackernews this morning where a guy had built a little screenshot uploader for dropbox. Unfortunately, his script is for Mac OS and I use linux.
So, for linux folks out there, here is a little wrapper around scrot, a linux screenshot utility. It will allow you to capture the full screen, the current window, or free-select a region, then take the resulting image and put it in your dropbox folder or upload it to Imgur:
#!/usr/bin/env python
import base64
import json
import optparse
import os
import subprocess
import sys
import time
import urllib
import urllib2
BINARY = 'scrot'
HOME = os.environ['HOME']
# Imgur API -- register your app and paste the client id and secret:
CLIENT_ID = ''
CLIENT_SECRET = ''
# Location of your dropbox folder and your dropbox user id:
DROPBOX_DIR = os.path.join(HOME, 'Dropbox/Public/screens/')
DROPBOX_URL_TEMPLATE = 'http://dl.dropbox.com/u/%s/screens/%s'
DROPBOX_UID = ''
def upload_file(filename):
with open(filename, 'rb') as fh:
contents = fh.read()
payload = urllib.urlencode((
('image', base64.b64encode(contents)),
('key', CLIENT_SECRET),
))
request = urllib2.Request('https://api.imgur.com/3/image', payload)
request.add_header('Authorization', 'Client-ID ' + CLIENT_ID)
try:
resp = urllib2.urlopen(request)
except urllib2.HTTPError, exc:
return False, 'Returned status: %s' % exc.code
except urllib2.URLError, exc:
return False, exc.reason
resp_data = resp.read()
try:
resp_json = json.loads(resp_data)
except ValueError:
return False, 'Error decoding response: %s' % resp_data
if resp_json['success']:
return True, resp_json['data']['link']
return False, 'Imgur failure: %s' % resp_data
def get_parser():
parser = optparse.OptionParser('Screenshot helper')
parser.add_option('-s', '--select', action='store_true', default=True,
dest='select', help='Select region to capture')
parser.add_option('-f', '--full', action='store_true', dest='full',
help='Capture entire screen')
parser.add_option('-c', '--current', action='store_true', dest='current',
help='Capture currently selected window')
parser.add_option('-d', '--delay', default=0, dest='delay', type='int',
help='Seconds to wait before capture')
parser.add_option('-p', '--public', action='store_true', dest='dropbox',
help='Store in dropbox public folder')
parser.add_option('-x', '--no-upload', action='store_false', default=True,
dest='upload', help='Do not upload to imgur')
parser.add_option('-k', '--keep-local', action='store_true', default=False,
dest='keep', help='Keep local copy after upload')
return parser
def get_scrot_command(filename, options):
args = [BINARY]
if options.current:
args.append('-u')
elif not options.full:
args.append('-s')
if options.delay:
args.append('-d %s' % options.delay)
args.append(dest)
return args
if __name__ == '__main__':
parser = get_parser()
options, args = parser.parse_args()
filename = 's%s.png' % time.time()
if options.dropbox:
dest = os.path.join(DROPBOX_DIR, filename)
else:
dest = os.path.join(HOME, 'tmp', filename)
if not options.current and not options.full:
print 'Select a region to capture...'
scrot_args = get_scrot_command(dest, options)
p = subprocess.Popen(scrot_args)
p.wait()
if options.dropbox:
print DROPBOX_URL_TEMPLATE % (DROPBOX_UID, filename)
if options.upload:
success, res = upload_file(dest)
if not success:
print 'Error uploading image: %s' % res
print 'Image stored in: %s' % dest
sys.exit(1)
else:
if not options.keep:
os.unlink(dest)
print res
else:
print dest
Using python and k-means to find the dominant colors in images October 23, 2012 17:23 / 17 comments
I'm working on a little photography website for my Dad and thought it would be neat to extract color information from photographs. I tried a couple of different approaches before finding one that works pretty well. This approach uses k-means clustering to cluster the pixels in groups based on their color. The center of those resulting clusters are then the "dominant" colors. k-means is a great fit for this problem because it is (usually) fast.
Peewee was baroque, so I rewrote it October 08, 2012 09:25 / 7 comments
Today I merged in the "unstable/2.0" branch of peewee. I'm very excited about the changes and I hope you will be, too.
I have written a documentation page on upgrading which gives the rationale behind the rewrite and some examples of the new querying API. Please feel free to take a look but much of the information presented in this post is lifted directly from the docs.
Goals for the rewrite
- consistent: there is one way of doing things
- expressive: things can be done that I never thought of
What changed?
The biggest changes between 1.0 and 2.0 are in the syntax used for constructing queries. The first iteration of peewee I threw up on github was about 600 lines. I was passing around strings and dictionaries and as time went on and I added features, those strings turned into tuples and objects. This meant, though, that I needed code to handle all the possible ways of expressing something. Look at the code for parse_select.
I learned a valuable lesson: keep data in datastructures until the absolute last second.
With the benefit of hindsight and experience, I decided to rewrite and unify the API a bit. The result is a tradeoff. The newer syntax may be a bit more verbose at times, but at least it will be consistent.
My 2004 Yamaha R6 September 12, 2012 14:47 / 2 comments
About two months ago I became the proud owner of a 2004 Yamaha R6! Previously I had been riding an '02 Honda Shadow (pic) and the change has been a revelation.
Web-based encrypted file storage using Flask and AWS September 12, 2012 10:34 / 6 comments
The other day I noticed I had a couple thumbdrives kicking around with various versions of my "absolutely do not lose" files...stuff like my private keys, tax documents, zips of papers I wrote in college, etc. These USB drives were all over the house, and many contained duplicate versions of the same files. I thought it would be neat to write a little app to give me a web-based interface to store and manage these files securely. In this post I'll talk about how I built a web-based file storage app using flask, pycrypto, and amazon S3.
A picture is worth a thousand words... using Google Images to spice up IRC September 11, 2012 14:07 / 1 comments
I wrote a little python IRC bot library a while back. It comes with a few silly examples like a google search bot, an ASCII art bot, even an example botnet. Today I was lurking around in a channel with a bunch of other local developers and noticed that we often are pasting links of images to "contextualize" things other folks have said.
Gedit port of vim theme "candycode" September 10, 2012 13:07 / 0 comments
I decided to port my favorite vim theme, candycode, to gedit.
The missing library: ad-hoc queries for your models September 05, 2012 11:41 / 5 comments
I think it would be great if more sites allowed users (or consumers of their APIs) to produce and execute ad-hoc queries against their data. In this post I'll talk a little bit about some ways sites are currently doing this, some of the challenges involved, my experience trying to build something "reusable", and finally invite you to share your thoughts.
Experimenting with an analytics web-service using python and cassandra August 02, 2012 10:48 / 2 comments
The other day I was poking around my google analytics account and thought it would be a fun project to see if I could collect "analytics"-type data myself. I recalled that the Apache Cassandra project was supposed to use a data model similar to Google's BigTable so I decided to use it for this project. The BigTable data model turned out to be a good fit for this project once I got over some of the intricacies of dealing with time-series data in Cassandra. In this post I'll talk about how I went about modelling, collecting, and finally analyzing basic page-view data I collected from this very blog.