Data Types¶
Traditionally all data stored in Riak was an opaque binary type. Then in version 1.4 came the introduction of a counter, the first Convergent Data Type supported in Riak. In Riak 2.0, several additional Data Types were introduced. Riak “knows” about these data types, and conflicting writes to them will converge automatically without presenting sibling values to the user.
Here is the list of current Data Types:
- Counter increments or decrements integer values
- Set allows you to store multiple distinct opaque binary values against a key
- Map is a nested, recursive struct, or associative array. Think of it as a container for composing ad hoc data structures from multiple Data Types. Inside a map you may store sets, counters, flags, registers, and even other maps
- Register stores binaries accoring to last-write-wins logic within Map
- Flag is similar to a boolean and also must be within Map
All Data Types must be stored in buckets bearing a BucketType that sets the datatype property to one of "counter", "set", or "map". Note that the bucket must have the allow_mult property set to true.
These Data Types are stored just like RiakObjects, so size constraints that apply to normal Riak values apply to Riak Data Types too.
An in-depth discussion of Data Types, also known as CRDTs, can be found at Data Types.
Examples of using Data Types can be found at Using Data Types.
Sending Operations¶
Riak Data Types provide a further departure from Riak’s usual operation, in that the API is operation-based. Rather than fetching the data structure, reconciling conflicts, mutating the result, and writing it back, you instead tell Riak what operations to perform on the Data Type. Here are some example operations:
Datatypes can be fetched and created just like RiakObject instances, using RiakBucket.get and RiakBucket.new, except that the bucket must belong to a bucket-type that has a valid datatype property. If we have a bucket-type named “social-graph” that has the datatype “set”, we would fetch a Set like so:
graph = client.bucket_type('social-graph')
graph.datatype # => 'set'
myfollowers = graph.bucket('followers').get('seancribbs')
# => a Set datatype
Once we have a datatype, we can stage operations against it and then send those operations to Riak:
myfollowers.add('javajolt')
myfollowers.discard('roach')
myfollowers.update()
While this looks in code very similar to manipulating RiakObject instances, only mutations are enqueued locally, not the new value.
Context and Observed-Remove¶
In order for Riak Data Types to behave well, you must have an opaque context received from a read when you:
The basic rule is “you cannot remove something you haven’t seen”, and the context tells Riak what you’ve actually seen, similar to the Vector clock on RiakObject. The Python client handles opaque contexts for you transparently as long as you fetch before performing one of these actions.
Datatype abstract class¶
- class riak.datatypes.Datatype(bucket=None, key=None, value=None, context=None)¶
Base class for all convergent datatype wrappers. You will not use this class directly, but it does define some methods are common to all datatype wrappers.
- value¶
The pure, immutable value of this datatype, as a Python value, which is unique for each datatype.
NB: Do not use this property to mutate data, as it will not have any effect. Use the methods of the individual type to effect changes. This value is guaranteed to be independent of any internal data representation.
- context¶
The opaque context for this type, if it was previously fetched.
Return type: str
- modified¶
Whether this datatype has staged local modifications.
Return type: bool
Persistence methods¶
- Datatype.reload(**params)¶
Reloads the datatype from Riak.
Parameters: - r (integer, string, None) – the read quorum
- pr (integer, string, None) – the primary read quorum
- basic_quorum (bool) – whether to use the “basic quorum” policy for not-founds
- notfound_ok (bool) – whether to treat not-found responses as successful
- timeout (int) – a timeout value in milliseconds
- include_context (bool) – whether to return the opaque context as well as the value, which is useful for removal operations on sets and maps
Return type:
- Datatype.update(**params)¶
Sends locally staged mutations to Riak.
Parameters: - w (integer) – W-value, wait for this many partitions to respond before returning to client.
- dw (integer) – DW-value, wait for this many partitions to confirm the write before returning to client.
- pw (integer) – PW-value, require this many primary partitions to be available before performing the put
- return_body (bool) – if the newly stored object should be retrieved, defaults to True
- include_context (bool) – whether to return the new opaque context when return_body is True
- timeout (int) – a timeout value in milliseconds
Return type: a subclass of Datatype
- Datatype.delete(**params)¶
Deletes the datatype from Riak. See RiakClient.delete() for options.
- Datatype.clear()¶
Removes all locally staged mutations.
Counter¶
- class riak.datatypes.Counter(bucket=None, key=None, value=None, context=None)¶
A convergent datatype that represents a counter which can be incremented or decremented. This type can stand on its own or be embedded within a Map.
- Counter.value¶
The current value of the counter.
Return type: int
- Counter.increment(amount=1)¶
Increments the counter by one or the given amount.
Parameters: amount (int) – the amount to increment the counter
- Counter.decrement(amount=1)¶
Decrements the counter by one or the given amount.
Parameters: amount (int) – the amount to decrement the counter
Set¶
- class riak.datatypes.Set(bucket=None, key=None, value=None, context=None)¶
A convergent datatype representing a Set with observed-remove semantics. Currently strings are the only supported value type. Example:
myset.add('barista') myset.add('roaster') myset.add('brewer')
Likewise they can simply be removed:
myset.discard('barista')
This datatype also implements the Set ABC, meaning it supports len(), in, and iteration.
- Set.value¶
An immutable copy of the current value of the set.
Return type: frozenset
- Set.add(element)¶
Adds an element to the set.
Parameters: element (str) – the element to add
- Set.discard(element)¶
Removes an element from the set.
Parameters: element (str) – the element to remove
Map¶
- class riak.datatypes.Map(bucket=None, key=None, value=None, context=None)¶
A convergent datatype that acts as a key-value datastructure. Keys are pairs of (name, datatype) where name is a string and datatype is the datatype name. Values are other convergent datatypes, represented by any concrete type in this module.
You cannot set values in the map directly (it does not implement __setitem__), but you may add new empty values or access non-existing values directly via bracket syntax. If a key is not in the original value of the map when accessed, fetching the key will cause its associated value to be created.:
map[('name', 'register')]
Keys and their associated values may be deleted from the map as you would in a dict:
del map[('emails', 'set')]
Convenience accessors exist that partition the map’s keys by datatype and implement the Mapping behavior as well as supporting deletion:
map.sets['emails'] map.registers['name'] del map.counters['likes']
- Map.value¶
Returns a copy of the original map’s value. Nested values are pure Python values as returned by Datatype.value from the nested types.
Return type: dict
- Map.counters¶
Filters keys in the map to only those of counter types. Example:
map.counters['views'].increment() del map.counters['points']
- Map.flags¶
Filters keys in the map to only those of flag types. Example:
map.flags['confirmed'].enable() del map.flags['attending']
- Map.maps¶
Filters keys in the map to only those of map types. Example:
map.maps['emails'].registers['home'].set("user@example.com") del map.maps['spam']
- Map.registers¶
Filters keys in the map to only those of register types. Example:
map.registers['username'].set_value("riak-user") del map.registers['access_key']
- Map.sets¶
Filters keys in the map to only those of set types. Example:
map.sets['friends'].add("brett") del map.sets['favorites']
Map-only datatypes¶
Two of the new Data Types may only be embedded in Map objects (in addition to Map itself):
Register¶
- class riak.datatypes.Register(bucket=None, key=None, value=None, context=None)¶
A convergent datatype that represents an opaque string that is set with last-write-wins semantics, and may only be embedded in Map instances.
- Register.value¶
Returns a copy of the original value of the register.
Return type: str
- Register.assign(new_value)¶
Assigns a new value to the register.
Parameters: new_value (str) – the new value for the register
Flag¶
- class riak.datatypes.Flag(bucket=None, key=None, value=None, context=None)¶
A convergent datatype that represents a boolean value that can be enabled or disabled, and may only be embedded in Map instances.
- Flag.value¶
The current value of the flag.
Return type: bool, None
- Flag.enable()¶
Turns the flag on, effectively setting its value to True.
- Flag.disable()¶
Turns the flag off, effectively setting its value to False.