Data Types

Traditionally all data stored in Riak was an opaque binary type. Then in version 1.4 came the introduction of a counter, the first Convergent Data Type supported in Riak. In Riak 2.0, several additional Data Types were introduced. Riak “knows” about these data types, and conflicting writes to them will converge automatically without presenting sibling values to the user.

Here is the list of current Data Types:

  • Counter increments or decrements integer values
  • Set allows you to store multiple distinct opaque binary values against a key
  • Map is a nested, recursive struct, or associative array. Think of it as a container for composing ad hoc data structures from multiple Data Types. Inside a map you may store sets, counters, flags, registers, and even other maps
  • Register stores binaries accoring to last-write-wins logic within Map
  • Flag is similar to a boolean and also must be within Map

All Data Types must be stored in buckets bearing a BucketType that sets the datatype property to one of "counter", "set", or "map". Note that the bucket must have the allow_mult property set to true.

These Data Types are stored just like RiakObjects, so size constraints that apply to normal Riak values apply to Riak Data Types too.

An in-depth discussion of Data Types, also known as CRDTs, can be found at Data Types.

Examples of using Data Types can be found at Using Data Types.

Sending Operations

Riak Data Types provide a further departure from Riak’s usual operation, in that the API is operation-based. Rather than fetching the data structure, reconciling conflicts, mutating the result, and writing it back, you instead tell Riak what operations to perform on the Data Type. Here are some example operations:

  • increment a Counter by 10
  • add 'joe' to a Set
  • remove the Set field called 'friends' from a Map
  • enable the prepay Flag in a Map

Datatypes can be fetched and created just like RiakObject instances, using Bucket.get and Bucket.new, except that the bucket must belong to a bucket-type that has a valid datatype property. If we have a bucket-type named “social-graph” that has the datatype “set”, we would fetch a Set like so:

graph = client.bucket_type('social-graph')
graph.datatype  # => 'set'
backet = graph.bucket('followers')
myfollowers = await bucket.get('seancribbs')
# => a Set datatype

Once we have a datatype, we can stage operations against it and then send those operations to Riak:

myfollowers.add('javajolt')
myfollowers.discard('roach')
await myfollowers.update()

While this looks in code very similar to manipulating RiakObject instances, only mutations are enqueued locally, not the new value.

Context and Observed-Remove

In order for Riak Data Types to behave well, you must have an opaque context received from a read when you:

The basic rule is “you cannot remove something you haven’t seen”, and the context tells Riak what you’ve actually seen, similar to the Vector clock on RiakObject. The Python client handles opaque contexts for you transparently as long as you fetch before performing one of these actions.

Datatype abstract class

class aioriak.datatypes.Datatype(bucket=None, key=None, value=None, context=None)[source]

Base class for all convergent datatype wrappers. You will not use this class directly, but it does define some methods are common to all datatype wrappers.

value

The pure, immutable value of this datatype, as a Python value, which is unique for each datatype. NB: Do not use this property to mutate data, as it will not have any effect. Use the methods of the individual type to effect changes. This value is guaranteed to be independent of any internal data representation.

context

The opaque context for this type, if it was previously fetched.

Return type:str
modified

Whether this datatype has staged local modifications.

Return type:bool

Persistence methods

coroutine Datatype.reload(**params)[source]

Reloads the datatype from Riak.

Return type:Datatype
coroutine Datatype.update(**params)[source]

Sends locally staged mutations to Riak.

Return type:a subclass of Datatype
coroutine Datatype.store(**params)

This is an alias for update().

coroutine Datatype.delete(**params)[source]

Deletes the datatype from Riak. See RiakClient.delete() for options.

Datatype.clear()[source]

Removes all locally staged mutations.

Counter

class aioriak.datatypes.Counter(bucket=None, key=None, value=None, context=None)[source]

A convergent datatype that represents a counter which can be incremented or decremented. This type can stand on its own or be embedded within a Map.

Counter.value

The current value of the counter.

Counter.increment(amount=1)[source]

Increments the counter by one or the given amount.

Parameters:amount (int) – the amount to increment the counter
Counter.decrement(amount=1)[source]

Decrements the counter by one or the given amount.

Parameters:amount (int) – the amount to decrement the counter

Set

class aioriak.datatypes.Set(bucket=None, key=None, value=None, context=None)[source]

A convergent datatype representing a Set with observed-remove semantics. Currently strings are the only supported value type.

Example::
myset.add(‘barista’) myset.add(‘roaster’) myset.add(‘brewer’)
Likewise they can simply be removed::
myset.discard(‘barista’)

This datatype also implements the Set ABC, meaning it supports len(), in, and iteration.

Set.value

An immutable copy (frozenset) of the current value of the set.

Set.add(element)[source]

Adds an element to the set.

Parameters:element (str) – the element to add
Set.discard(element)[source]

Removes an element from the set.

Parameters:element (str) – the element to remove

Map

class aioriak.datatypes.Map(bucket=None, key=None, value=None, context=None)[source]

A convergent datatype that acts as a key-value datastructure. Keys are pairs of (name, datatype) where name is a string and datatype is the datatype name. Values are other convergent datatypes, represented by any concrete type in this module.

You cannot set values in the map directly (it does not implement __setitem__), but you may add new empty values or access non-existing values directly via bracket syntax. If a key is not in the original value of the map when accessed, fetching the key will cause its associated value to be created.:

map[('name', 'register')]

Keys and their associated values may be deleted from the map as you would in a dict:

del map[('emails', 'set')]

Convenience accessors exist that partition the map’s keys by datatype and implement the collections.Mapping behavior as well as supporting deletion:

map.sets['emails']
map.registers['name']
del map.counters['likes']
Map.value

Returns a copy of the original map’s value. Nested values are pure Python values as returned by Datatype.value from the nested types.

Return type:dict
Map.counters

Filters keys in the map to only those of counter types. Example:

map.counters['views'].increment()
del map.counters['points']
Map.flags

Filters keys in the map to only those of flag types. Example:

map.flags['confirmed'].enable()
del map.flags['attending']
Map.maps

Filters keys in the map to only those of map types. Example:

map.maps['emails'].registers['home'].set("user@example.com")
del map.maps['spam']
Map.registers

Filters keys in the map to only those of register types. Example:

map.registers['username'].set_value("riak-user")
del map.registers['access_key']
Map.sets

Filters keys in the map to only those of set types. Example:

map.sets['friends'].add("brett")
del map.sets['favorites']

Map-only datatypes

Two of the new Data Types may only be embedded in Map objects (in addition to Map itself):

Register

class aioriak.datatypes.Register(bucket=None, key=None, value=None, context=None)[source]

A convergent datatype that represents an opaque string that is set with last-write-wins semantics, and may only be embedded in Map instances.

Register.value

Returns a copy of the original value of the register.

Return type:str
Register.assign(new_value)[source]

Assigns a new value to the register.

Parameters:new_value (str) – the new value for the register

Flag

class aioriak.datatypes.Flag(bucket=None, key=None, value=None, context=None)[source]

A convergent datatype that represents a boolean value that can be enabled or disabled, and may only be embedded in Map instances.

Flag.value

The current value of the flag.

Flag.enable()[source]

Turns the flag on, effectively setting its value to True.

Flag.disable()[source]

Turns the flag off, effectively setting its value to False.