Rewriting Huey for a better API May 15, 2013 11:50 / 0 comments
For a while I've been itching to rewrite Huey, and just last week released 0.4 which is an almost total rewrite. I initially started Huey for performing tasks like checking comments for spam, sending emails, generating thumbnails, and basically anything that would slow down the pagespeed on my sites. This is still what I see as the primary use-case for huey -- performing small tasks outside the request/response cycle and running jobs on a schedule (I have a site that scrapes the county sheriff's site and keeps a log of arrests in my town). The goal for the rewrite was not to change the purpose of Huey, rather it was to change the API.
Structuring flask apps, a how-to for those coming from Django April 27, 2013 13:21 / 0 comments
The other day a friend of mine was trying out flask-peewee and he had some questions about the best way to structure his app to avoid triggering circular imports. For someone new to flask, this can be a bit of a puzzler, especially if you're coming from django which automatically imports your modules. In this post I'll walk through how I like to structure my flask apps to avoid circular imports. In my examples I'll be showing how to use "flask-peewee", but the same technique should be applicable for other flask plugins.
I'll walk through the modules I commonly use in my apps, then show how to tie them all together and provide a single entrypoint into your app.
"wallfix", using python to set my wallpaper April 22, 2013 09:54 / 0 comments
I had fun writing about my "cd" helper, so I thought I'd share another productivity helper I wrote for setting my wallpaper. It's a little silly, but I insist on my wallpaper being used for my lockscreen and my login window as well -- that way the entire time I'm on my computer the background is "seamless". Before I wrote this script it used to take me probably 3 or 4 minutes to change wallpapers!
Creating a personal password manager April 14, 2013 09:26 / 1 comments
My password "system" used to be that I had three different passwords, all of which were variations on the same theme. I maintained a list of sites I had accounts on and for each site gave a hint which of the three passwords I used. What a terrible scheme.
A couple weeks ago I decided to do something about it. I wanted, above all, to only have to remember a single password. Being lately security-conscious, I also recognized the need for a unique password on every site.
In this post I'll show how I used python to create a password management system that allows me to use a single "master" password to generate unique passwords for all the sites and services I use.
"j" for switching directories - hacking "cd" with python April 12, 2013 22:05 / 7 comments
Everyone uses cd a lot, I'm no exception. Because I use virtualenvs for my
python projects, I'm often "cutting" through several layers of crap to get to
what I actually want to edit. This was a good opportunity for a helper script!
The two biggest annoyances I was trying to alleviate were:
- There are directories I use a lot, but making bash aliases for them is not maintainable. I should be able to get to them quickly.
- I have to keep a mental map of the directory tree to go from one
nested directory to another -- e.g.
cd ../../some-other-dir/foo/. It would be nice to just type the part that matters and not the whole thing.
The solution I came up with stores directories I use (the entire path), and then I can perform a search of that history using a partial path.
Raspberry Pi Mobile January 14, 2013 09:31 / 0 comments
My Raspberry Pi got a new case this weekend:
Shortcomings in the Django ORM and a look at Peewee, a lightweight alternative December 15, 2012 14:27 / 13 comments
In this post I'd like to talk about some of the shortcomings of the Django ORM, the ways peewee approaches things differently, and how this resulted in peewee having an API that is both more consistent and more expressive.
Sharing Screenshots with Dropbox and Imgur November 28, 2012 11:09 / 2 comments
I saw a post on hackernews this morning where a guy had built a little screenshot uploader for dropbox. Unfortunately, his script is for Mac OS and I use linux.
So, for linux folks out there, here is a little wrapper around scrot, a linux screenshot utility. It will allow you to capture the full screen, the current window, or free-select a region, then take the resulting image and put it in your dropbox folder or upload it to Imgur:
#!/usr/bin/env python
import base64
import json
import optparse
import os
import subprocess
import sys
import time
import urllib
import urllib2
BINARY = 'scrot'
HOME = os.environ['HOME']
# Imgur API -- register your app and paste the client id and secret:
CLIENT_ID = ''
CLIENT_SECRET = ''
# Location of your dropbox folder and your dropbox user id:
DROPBOX_DIR = os.path.join(HOME, 'Dropbox/Public/screens/')
DROPBOX_URL_TEMPLATE = 'http://dl.dropbox.com/u/%s/screens/%s'
DROPBOX_UID = ''
def upload_file(filename):
with open(filename, 'rb') as fh:
contents = fh.read()
payload = urllib.urlencode((
('image', base64.b64encode(contents)),
('key', CLIENT_SECRET),
))
request = urllib2.Request('https://api.imgur.com/3/image', payload)
request.add_header('Authorization', 'Client-ID ' + CLIENT_ID)
try:
resp = urllib2.urlopen(request)
except urllib2.HTTPError, exc:
return False, 'Returned status: %s' % exc.code
except urllib2.URLError, exc:
return False, exc.reason
resp_data = resp.read()
try:
resp_json = json.loads(resp_data)
except ValueError:
return False, 'Error decoding response: %s' % resp_data
if resp_json['success']:
return True, resp_json['data']['link']
return False, 'Imgur failure: %s' % resp_data
def get_parser():
parser = optparse.OptionParser('Screenshot helper')
parser.add_option('-s', '--select', action='store_true', default=True,
dest='select', help='Select region to capture')
parser.add_option('-f', '--full', action='store_true', dest='full',
help='Capture entire screen')
parser.add_option('-c', '--current', action='store_true', dest='current',
help='Capture currently selected window')
parser.add_option('-d', '--delay', default=0, dest='delay', type='int',
help='Seconds to wait before capture')
parser.add_option('-p', '--public', action='store_true', dest='dropbox',
help='Store in dropbox public folder')
parser.add_option('-x', '--no-upload', action='store_false', default=True,
dest='upload', help='Do not upload to imgur')
parser.add_option('-k', '--keep-local', action='store_true', default=False,
dest='keep', help='Keep local copy after upload')
return parser
def get_scrot_command(filename, options):
args = [BINARY]
if options.current:
args.append('-u')
elif not options.full:
args.append('-s')
if options.delay:
args.append('-d %s' % options.delay)
args.append(dest)
return args
if __name__ == '__main__':
parser = get_parser()
options, args = parser.parse_args()
filename = 's%s.png' % time.time()
if options.dropbox:
dest = os.path.join(DROPBOX_DIR, filename)
else:
dest = os.path.join(HOME, 'tmp', filename)
if not options.current and not options.full:
print 'Select a region to capture...'
scrot_args = get_scrot_command(dest, options)
p = subprocess.Popen(scrot_args)
p.wait()
if options.dropbox:
print DROPBOX_URL_TEMPLATE % (DROPBOX_UID, filename)
if options.upload:
success, res = upload_file(dest)
if not success:
print 'Error uploading image: %s' % res
print 'Image stored in: %s' % dest
sys.exit(1)
else:
if not options.keep:
os.unlink(dest)
print res
else:
print dest
Using python and k-means to find the dominant colors in images October 23, 2012 17:23 / 17 comments
I'm working on a little photography website for my Dad and thought it would be neat to extract color information from photographs. I tried a couple of different approaches before finding one that works pretty well. This approach uses k-means clustering to cluster the pixels in groups based on their color. The center of those resulting clusters are then the "dominant" colors. k-means is a great fit for this problem because it is (usually) fast.
Peewee was baroque, so I rewrote it October 08, 2012 09:25 / 7 comments
Today I merged in the "unstable/2.0" branch of peewee. I'm very excited about the changes and I hope you will be, too.
I have written a documentation page on upgrading which gives the rationale behind the rewrite and some examples of the new querying API. Please feel free to take a look but much of the information presented in this post is lifted directly from the docs.
Goals for the rewrite
- consistent: there is one way of doing things
- expressive: things can be done that I never thought of
What changed?
The biggest changes between 1.0 and 2.0 are in the syntax used for constructing queries. The first iteration of peewee I threw up on github was about 600 lines. I was passing around strings and dictionaries and as time went on and I added features, those strings turned into tuples and objects. This meant, though, that I needed code to handle all the possible ways of expressing something. Look at the code for parse_select.
I learned a valuable lesson: keep data in datastructures until the absolute last second.
With the benefit of hindsight and experience, I decided to rewrite and unify the API a bit. The result is a tradeoff. The newer syntax may be a bit more verbose at times, but at least it will be consistent.