June 19, 2014 10:22 / 0 comments / nosql python vedis

vedis-python logo

Over the past week I've been writing some python bindings to the embedded NoSQL database Vedis, a transactional data-store modeled after Redis. Like Redis, Vedis could be characterized as an advanced key-value store that supports hash, set and list data-structures. Vedis has over 70 available commands for working with the various data types. Unlike Redis, which is run as a separate server process, Vedis is embedded in the host process like SQLite. Vedis works with either in-memory databases or on-disk databases. Vedis is transactional (ACID) and also thread-safe. If you'd like more information, check out the Vedis FAQ.

Vedis-python

Vedis-python allows you to use Vedis in your Python apps. Vedis-python supports all the Vedis data-types, and also allows you extend Vedis by writing your own commands in Python. As I mentioned, this project is very new so while I have written pretty extensive unit tests, the library has certainly not been battle-tested yet.

If you'd like to give it a try, you can use pip to install vedis-python. At the time of writing the current version is 0.1.5.

$ pip install vedis-python

Just a word of caution, I've tested the installation on various flavors of Linux (including on my raspberry pi) and Mac OS but have not tested on Windows.

Basic Usage

The following is taken from the quick-start. You can also check out the full API documentation for a complete description vedis-python's features.

Key/value features

You can use Vedis like a dictionary for simple key/value lookups:

>>> from vedis import Vedis
>>> db = Vedis(':mem:')  # Create an in-memory database. Alternatively you could supply a filename for an on-disk database.
>>> db['k1'] = 'v1'
>>> db['k1']
'v1'

>>> db.append('k1', 'more data')  # Returns length of value after appending new data.
11
>>> db['k1']
'v1more data'

>>> del db['k1']
>>> 'k1' in db
False
>>> db['k1']
None

You can set and get multiple items at a time:

>>> db.mset(k1='v1', k2='v2', k3='v3')
True

>>> db.mget('k1', 'k2', 'missing key', 'k3')
<generator object iter_vedis_array at 0x7f37dd58be10>

>>> list(db.mget('k1', 'k2', 'missing key', 'k3'))
['v1', 'v2', None, 'v3']

In addition to storing string keys/values, you can also implement counters:

>>> db.incr('counter')
1
>>> db.incr('counter')
2

>>> db.incr_by('counter', 10)
12
>>> db.decr('counter')
11

Hashes

Vedis supports nested key/value lookups which have the additional benefit of supporting operations to retrieve all keys, values, the number of items in the hash, and so on.

>>> h = db.Hash('some key')
>>> h['k1'] = 'v1'
>>> h.update(k2='v2', k3='v3')

>>> h
<Hash: {'k3': 'v3', 'k2': 'v2', 'k1': 'v1'}>

>>> h.to_dict()
{'k3': 'v3', 'k2': 'v2', 'k1': 'v1'}

>>> h.items()
[('k1', 'v1'), ('k3', 'v3'), ('k2', 'v2')]

>>> list(h.keys())
['k1', 'k3', 'k2']

>>> del h['k2']

>>> len(h)
2

>>> 'k1' in h
True

>>> [key for key in h]
['k1', 'k3']

Sets

Vedis supports a set data-type which stores a unique collection of items.

>>> s = db.Set('some set')
>>> s.add('v1', 'v2', 'v3')
3

>>> len(s)
3

>>> 'v1' in s, 'v4' in s
(True, False)

>>> s.top()
'v1'

>>> s.peek()
'v3'

>>> s.remove('v2')
1

>>> s.add('v4', 'v5')
2

>>> s.pop()
'v5'

>>> [item for item in s]
['v1', 'v3', 'v4']

>>> s.to_set()
set(['v1', 'v3', 'v4'])

>>> s2 = db.Set('another set')
>>> s2.add('v1', 'v4', 'v5', 'v6')
4

>>> s2 & s  # Intersection.
set(['v1', 'v4'])

>>> s2 - s  # Difference.
set(['v5', 'v6'])

Lists

Vedis also supports a list data type. The list data type is fairly basic and does not support as many operations as the set and hash types. I'm not sure if this is intentional or something that will change in future versions, but thought it was worth mentioning.

>>> l = db.List('my list')
>>> l.append('v1', 'v2', 'v3', 'v1')
4

>>> len(l)
4

>>> l[1]
'v2'

>>> db.llen('my_list')
2

>>> l.pop(), l.pop()
('v1', 'v2')

>>> len(l)
2

Misc

Vedis has a somewhat quirky collection of other miscellaneous commands. Below is a sampling, but the full API documentation can be found here.

>>> db.base64('encode me')
'ZW5jb2RlIG1l'

>>> db.base64_decode('ZW5jb2RlIG1l')
'encode me'

>>> db.random_string(10)
'raurquvsnx'

>>> db.rand(1, 6)
4

>>> list(db.str_split('abcdefghijklmnop', 5))
['abcde', 'fghij', 'klmno', 'p']

>>> db['data'] = 'abcdefghijklmnop'
>>> db.strlen('data')
16

>>> db.strip_tags('<p>This <span>is</span> a <a href="#">test</a>.</p>')
'This is a test.'

Writing your own Vedis commands

It is easy to write your own Vedis commands:

db = Vedis()

@db.register('CONCAT')
def concat(context, glue, *params):
    return glue.join(params)

@db.register('TITLE')
def title(context, *params):
    return [param.title() for param in params]

Here is how you might call the custom commands:

>>> print db.execute('CONCAT | foo bar baz', result=True)
foo|bar|baz

>>> print db.execute('TITLE "testing" "this is a test" "another"', result=True)
['Testing', 'This Is A Test', 'Another']

>>> print db.TITLE('testing', 'this is a test', 'another')
['Testing', 'This Is A Test', 'Another']

To-dos

I am still working out some kinks in the explicit transaction management code, but I expect to have that working shortly. Additionally I am planning to add a nicer wrapper around the vedis_context object which is passed in to user-defined commands. If you have suggestions for improvements to the API I would love to hear them, so please leave a comment.

Reading more

Thanks for taking the time to read this post, I hope you found it interesting. Feel free to leave a comment below if you have any questions. If you find a bug in Vedis-Python, I would be grateful if you could open an issue on GitHub describing the problem.

Comments (0)


Commenting has been closed, but please feel free to contact me