Data Types¶
Traditionally all data stored in Riak was an opaque binary type. Then in version 1.4 came the introduction of a counter, the first Convergent Data Type supported in Riak. In Riak 2.0, several additional Data Types were introduced. Riak “knows” about these data types, and conflicting writes to them will converge automatically without presenting sibling values to the user.
Here is the list of current Data Types:
Counter
increments or decrements integer valuesSet
allows you to store multiple distinct opaque binary values against a keyMap
is a nested, recursive struct, or associative array. Think of it as a container for composing ad hoc data structures from multiple Data Types. Inside a map you may store sets, counters, flags, registers, and even other mapsRegister
stores binaries accoring to last-write-wins logic withinMap
Flag
is similar to a boolean and also must be withinMap
All Data Types must be stored in buckets bearing a
BucketType
that sets the
datatype
property to one of
"counter"
, "set"
, or "map"
. Note that the bucket must have
the allow_mult
property set to true
.
These Data Types are stored just like RiakObjects
, so size constraints that apply to
normal Riak values apply to Riak Data Types too.
An in-depth discussion of Data Types, also known as CRDTs, can be found at Data Types.
Examples of using Data Types can be found at Using Data Types.
Sending Operations¶
Riak Data Types provide a further departure from Riak’s usual operation, in that the API is operation-based. Rather than fetching the data structure, reconciling conflicts, mutating the result, and writing it back, you instead tell Riak what operations to perform on the Data Type. Here are some example operations:
Datatypes can be fetched and created just like
RiakObject
instances, using
RiakBucket.get
and
RiakBucket.new
, except that the
bucket must belong to a bucket-type that has a valid datatype
property. If we have a bucket-type named “social-graph” that has the
datatype “set”, we would fetch a Set
like so:
graph = client.bucket_type('social-graph')
graph.datatype # => 'set'
myfollowers = graph.bucket('followers').get('seancribbs')
# => a Set datatype
Once we have a datatype, we can stage operations against it and then send those operations to Riak:
myfollowers.add('javajolt')
myfollowers.discard('roach')
myfollowers.update()
While this looks in code very similar to manipulating
RiakObject
instances, only mutations are
enqueued locally, not the new value.
Context and Observed-Remove¶
In order for Riak Data Types to behave well, you must have an opaque context received from a read when you:
The basic rule is “you cannot remove something you haven’t seen”, and
the context tells Riak what you’ve actually seen, similar to the
Vector clock on RiakObject
. The Python
client handles opaque contexts for you transparently as long as you
fetch before performing one of these actions.
Datatype abstract class¶
-
class
riak.datatypes.
Datatype
(bucket=None, key=None, value=None, context=None)¶ Base class for all convergent datatype wrappers. You will not use this class directly, but it does define some methods are common to all datatype wrappers.
-
value
¶ The pure, immutable value of this datatype, as a Python value, which is unique for each datatype.
NB: Do not use this property to mutate data, as it will not have any effect. Use the methods of the individual type to effect changes. This value is guaranteed to be independent of any internal data representation.
-
context
¶ The opaque context for this type, if it was previously fetched.
Return type: str
-
modified
¶ Whether this datatype has staged local modifications.
Return type: bool
-
Persistence methods¶
-
Datatype.
reload
(**params)¶ Reloads the datatype from Riak.
Parameters: - r (integer, string, None) – the read quorum
- pr (integer, string, None) – the primary read quorum
- basic_quorum (bool) – whether to use the “basic quorum” policy for not-founds
- notfound_ok (bool) – whether to treat not-found responses as successful
- timeout (int) – a timeout value in milliseconds
- include_context (bool) – whether to return the opaque context as well as the value, which is useful for removal operations on sets and maps
Return type:
-
Datatype.
update
(**params)¶ Sends locally staged mutations to Riak.
Parameters: - w (integer) – W-value, wait for this many partitions to respond before returning to client.
- dw (integer) – DW-value, wait for this many partitions to confirm the write before returning to client.
- pw (integer) – PW-value, require this many primary partitions to be available before performing the put
- return_body (bool) – if the newly stored object should be retrieved, defaults to True
- include_context (bool) – whether to return the new opaque context when return_body is True
- timeout (int) – a timeout value in milliseconds
Return type: a subclass of
Datatype
-
Datatype.
delete
(**params)¶ Deletes the datatype from Riak. See
RiakClient.delete()
for options.
-
Datatype.
clear
()¶ Removes all locally staged mutations.
Counter¶
-
class
riak.datatypes.
Counter
(bucket=None, key=None, value=None, context=None)¶ A convergent datatype that represents a counter which can be incremented or decremented. This type can stand on its own or be embedded within a
Map
.
-
Counter.
value
¶ The current value of the counter.
Return type: int
-
Counter.
increment
(amount=1)¶ Increments the counter by one or the given amount.
Parameters: amount (int) – the amount to increment the counter
-
Counter.
decrement
(amount=1)¶ Decrements the counter by one or the given amount.
Parameters: amount (int) – the amount to decrement the counter
Set¶
-
class
riak.datatypes.
Set
(bucket=None, key=None, value=None, context=None)¶ A convergent datatype representing a Set with observed-remove semantics. Currently strings are the only supported value type. Example:
myset.add('barista') myset.add('roaster') myset.add('brewer')
Likewise they can simply be removed:
myset.discard('barista')
This datatype also implements the Set ABC, meaning it supports
len()
,in
, and iteration.
-
Set.
value
¶ An immutable copy of the current value of the set.
Return type: frozenset
-
Set.
add
(element)¶ Adds an element to the set.
Parameters: element (str) – the element to add
-
Set.
discard
(element)¶ Removes an element from the set.
Parameters: element (str) – the element to remove
Map¶
-
class
riak.datatypes.
Map
(bucket=None, key=None, value=None, context=None)¶ A convergent datatype that acts as a key-value datastructure. Keys are pairs of
(name, datatype)
wherename
is a string anddatatype
is the datatype name. Values are other convergent datatypes, represented by any concrete type in this module.You cannot set values in the map directly (it does not implement
__setitem__
), but you may add new empty values or access non-existing values directly via bracket syntax. If a key is not in the original value of the map when accessed, fetching the key will cause its associated value to be created.:map[('name', 'register')]
Keys and their associated values may be deleted from the map as you would in a dict:
del map[('emails', 'set')]
Convenience accessors exist that partition the map’s keys by datatype and implement the
Mapping
behavior as well as supporting deletion:map.sets['emails'] map.registers['name'] del map.counters['likes']
-
Map.
value
¶ Returns a copy of the original map’s value. Nested values are pure Python values as returned by
Datatype.value
from the nested types.Return type: dict
-
Map.
counters
¶ Filters keys in the map to only those of counter types. Example:
map.counters['views'].increment() del map.counters['points']
-
Map.
flags
¶ Filters keys in the map to only those of flag types. Example:
map.flags['confirmed'].enable() del map.flags['attending']
-
Map.
maps
¶ Filters keys in the map to only those of map types. Example:
map.maps['emails'].registers['home'].set("user@example.com") del map.maps['spam']
-
Map.
registers
¶ Filters keys in the map to only those of register types. Example:
map.registers['username'].set_value("riak-user") del map.registers['access_key']
-
Map.
sets
¶ Filters keys in the map to only those of set types. Example:
map.sets['friends'].add("brett") del map.sets['favorites']
Map-only datatypes¶
Two of the new Data Types may only be embedded in
Map
objects (in addition to
Map
itself):
Register¶
-
class
riak.datatypes.
Register
(bucket=None, key=None, value=None, context=None)¶ A convergent datatype that represents an opaque string that is set with last-write-wins semantics, and may only be embedded in
Map
instances.
-
Register.
value
¶ Returns a copy of the original value of the register.
Return type: str
-
Register.
assign
(new_value)¶ Assigns a new value to the register.
Parameters: new_value (str) – the new value for the register
Flag¶
-
class
riak.datatypes.
Flag
(bucket=None, key=None, value=None, context=None)¶ A convergent datatype that represents a boolean value that can be enabled or disabled, and may only be embedded in
Map
instances.
-
Flag.
value
¶ The current value of the flag.
Return type: bool, None
-
Flag.
enable
()¶ Turns the flag on, effectively setting its value to
True
.
-
Flag.
disable
()¶ Turns the flag off, effectively setting its value to
False
.