Welcome to aioriak’s documentation!

Asyncio (PEP 3156) Riak client library. This project is based on official Bash python client library (https://github.com/basho/riak-python-client).

Features

Riak KV operations Yes
Riak Datatypes Yes
Riak BucketTypes Yes
Custom resolver Yes
Node list support No
Custom quorum No
Connections Pool No
Operations timout No
Security No
Riak Search No
MapReduce No
Tested python versions 3.5.0, 3.5.1
Tested Riak versions 2.1.3

Installation

The easiest way to install aioriak is by using the package on PyPi:

pip install aioriak

Requirements

  • Python >= 3.5
  • riak>=2.1.3

Contribute

Feel free to file an issue or make pull request if you find any bugs or have some suggestions for library improvement.

License

The aioriak is offered under MIT license.


Contents

Client & Connections

To connect to a Riak cluster, you must create a RiakClient object. The default configuration connects to a Riak node on localhost with the default ports. The below instantiation statements are all equivalent:

from aioriak import RiakClient


client = RiakClient()

async def go():
    client = await RiakClient.create(host='127.0.0.1', port=8087)

Note

Connections are not established until you attempt to perform an operation. If the host or port are incorrect, you will not get an error raised immediately.

Client objects

Client-level Operations

Some operations are not scoped by buckets or bucket types and can be performed on the client directly:

Accessing Bucket Types and Buckets

Most client operations are on bucket type objects, the bucket objects they contain or keys within those buckets. Use the bucket_type or bucket methods for creating bucket types and buckets that will proxy operations to the called client.

Bucket Type Operations

Bucket Operations

Key-level Operations

Serialization

The client supports automatic transformation of Riak responses into Python types if encoders and decoders are registered for the media-types. Supported by default are application/json and text/plain.

Buckets & Bucket Types

Buckets are both namespaces for the key-value pairs you store in Riak, and containers for properties that apply to that namespace. In older versions of Riak, this was the only logical organization available. Now a higher-level collection called a Bucket Type can group buckets together. They allow for efficiently setting properties on a group of buckets at the same time.

Unlike buckets, Bucket Types must be explicitly created and activated before being used:

riak-admin bucket-type create n_equals_1 '{"props":{"n_val":1}}'
riak-admin bucket-type activate n_equals_1

Bucket Type creation and activation is only supported via the riak-admin bucket-type command-line tool. Riak 2.0 does not include an API to perform these actions, but the Python client can retrieve and set bucket-type properties.

If Bucket Types are not specified, the default bucket type is used. These buckets should be created via the bucket() method on the client object, like so:

import riak

async def go():
    client = await riak.RiakClient.create()
    mybucket = client.bucket('mybucket')

Buckets with a user-specified Bucket Type can also be created via the same bucket() method with an additional parameter or explicitly via bucket_type():

othertype = client.bucket_type('othertype')
otherbucket = othertype.bucket('otherbucket')

# Alternate way to get a bucket within a bucket-type
mybucket = client.bucket('mybucket', bucket_type='mybuckettype')

For more detailed discussion, see Using Bucket Types.

Bucket objects

Bucket properties

Bucket properties are flags and defaults that apply to all keys in the bucket.

Shortcuts for common properties

Some of the most commonly-used bucket properties are exposed as object properties as well. The getters and setters simply call Bucket.get_property() and Bucket.set_property() respectively.

Working with keys

The primary purpose of buckets is to act as namespaces for keys. As such, you can use the bucket object to create, fetch and delete objects.

Serialization

Similar to RiakClient, buckets can register custom transformation functions for media-types. When undefined on the bucket, Bucket.get_encoder() and Bucket.get_decoder() will delegate to the client associated with the bucket.

Listing keys

Shortcuts for RiakClient.get_keys() are exposed on the bucket object. The same admonitions for these operation apply.

Bucket Type objects

Bucket Type properties

Bucket Type properties are flags and defaults that apply to all buckets in the Bucket Type.

BucketType.datatype

The assigned datatype for this bucket type, if present.

Listing buckets

Shortcut for RiakClient.get_buckets() is exposed on the bucket type object. This is similar to Listing keys on buckets.

Values & Objects

Keys in Riak are namespaced into buckets, and their associated values are represented by objects, not to be confused with Python “objects”. A RiakObject is a container for the key, the Vector clock, the value(s) and any metadata associated with the value(s).

Values may also be datatypes, but are not discussed here.

RiakObject

Vector clock

Vector clocks are Riak’s means of tracking the relationships between writes to a key. It is best practice to fetch the latest version of a key before attempting to modify or overwrite the value; if you do not, you may create Siblings or lose data! The content of a vector clock is essentially opaque to the user.

Persistence

Fetching, storing, and deleting keys are the bread-and-butter of Riak.

Value and Metadata

Unless you have enabled Siblings via the allow_mult bucket property, you can inspect and manipulate the value and metadata of an object directly using these properties and methods:

Siblings

Because Riak’s consistency model is “eventual” (and not linearizable), there is no way for it to disambiguate writes that happen concurrently. The Vector clock helps establish a “happens after” relationships so that concurrent writes can be detected, but with the exception of Data Types, Riak has no way to determine which write has the correct value.

Instead, when allow_mult is True, Riak keeps all writes that appear to be concurrent. Thus, the contents of a key’s value may, in fact, be multiple values, which are called “siblings”. Siblings are modeled in RiakContent objects, which contain all of the same Persistence methods and attributes as the parent object.

You do not typically have to create RiakContent objects yourself, but they will be created for you when fetching objects from Riak.

Note

The Persistence accessors on RiakObject are actually proxied to the first sibling when the object has only one.

Conflicts and Resolvers

When an object is not in conflict, it has only one sibling. When it is in conflict, you will have to resolve the conflict before it can be written again. How you choose to resolve the conflict is up to you, but you can automate the process using a resolver function.

riak.resolver.default_resolver(riak_object)

The default conflict-resolution function, which does nothing. To implement a resolver, define a function that sets the siblings property on the passed RiakObject instance to a list containing a single RiakContent object.

Parameters:riak_object (RiakObject) – an object-in-conflict that will be resolved
riak.resolver.last_written_resolver(riak_object)

A conflict-resolution function that resolves by selecting the most recently-modified sibling by timestamp.

Parameters:riak_object (RiakObject) – an object-in-conflict that will be resolved

If you do not supply a resolver function, or your resolver leaves multiple siblings present, accessing the Persistence will result in a ConflictError being raised.

Data Types

Traditionally all data stored in Riak was an opaque binary type. Then in version 1.4 came the introduction of a counter, the first Convergent Data Type supported in Riak. In Riak 2.0, several additional Data Types were introduced. Riak “knows” about these data types, and conflicting writes to them will converge automatically without presenting sibling values to the user.

Here is the list of current Data Types:

  • Counter increments or decrements integer values
  • Set allows you to store multiple distinct opaque binary values against a key
  • Map is a nested, recursive struct, or associative array. Think of it as a container for composing ad hoc data structures from multiple Data Types. Inside a map you may store sets, counters, flags, registers, and even other maps
  • Register stores binaries accoring to last-write-wins logic within Map
  • Flag is similar to a boolean and also must be within Map

All Data Types must be stored in buckets bearing a BucketType that sets the datatype property to one of "counter", "set", or "map". Note that the bucket must have the allow_mult property set to true.

These Data Types are stored just like RiakObjects, so size constraints that apply to normal Riak values apply to Riak Data Types too.

An in-depth discussion of Data Types, also known as CRDTs, can be found at Data Types.

Examples of using Data Types can be found at Using Data Types.

Sending Operations

Riak Data Types provide a further departure from Riak’s usual operation, in that the API is operation-based. Rather than fetching the data structure, reconciling conflicts, mutating the result, and writing it back, you instead tell Riak what operations to perform on the Data Type. Here are some example operations:

  • increment a Counter by 10
  • add 'joe' to a Set
  • remove the Set field called 'friends' from a Map
  • enable the prepay Flag in a Map

Datatypes can be fetched and created just like RiakObject instances, using Bucket.get and Bucket.new, except that the bucket must belong to a bucket-type that has a valid datatype property. If we have a bucket-type named “social-graph” that has the datatype “set”, we would fetch a Set like so:

graph = client.bucket_type('social-graph')
graph.datatype  # => 'set'
backet = graph.bucket('followers')
myfollowers = await bucket.get('seancribbs')
# => a Set datatype

Once we have a datatype, we can stage operations against it and then send those operations to Riak:

myfollowers.add('javajolt')
myfollowers.discard('roach')
await myfollowers.update()

While this looks in code very similar to manipulating RiakObject instances, only mutations are enqueued locally, not the new value.

Context and Observed-Remove

In order for Riak Data Types to behave well, you must have an opaque context received from a read when you:

  • disable a Flag (set it to false)
  • remove a field from a Map
  • remove an element from a Set

The basic rule is “you cannot remove something you haven’t seen”, and the context tells Riak what you’ve actually seen, similar to the Vector clock on RiakObject. The Python client handles opaque contexts for you transparently as long as you fetch before performing one of these actions.

Datatype abstract class

Persistence methods

coroutine Datatype.store(**params)

This is an alias for update().

Counter

Counter.value

The current value of the counter.

Set

Set.value

An immutable copy (frozenset) of the current value of the set.

Map

Map.counters

Filters keys in the map to only those of counter types. Example:

map.counters['views'].increment()
del map.counters['points']
Map.flags

Filters keys in the map to only those of flag types. Example:

map.flags['confirmed'].enable()
del map.flags['attending']
Map.maps

Filters keys in the map to only those of map types. Example:

map.maps['emails'].registers['home'].set("user@example.com")
del map.maps['spam']
Map.registers

Filters keys in the map to only those of register types. Example:

map.registers['username'].set_value("riak-user")
del map.registers['access_key']
Map.sets

Filters keys in the map to only those of set types. Example:

map.sets['friends'].add("brett")
del map.sets['favorites']

Map-only datatypes

Two of the new Data Types may only be embedded in Map objects (in addition to Map itself):

Register

Flag

Flag.value

The current value of the flag.

# query # security # advanced

Indices and tables