Reusable Patterns for Riak in Scala

We use Riak at Boundary to persist a lot of data. Kobayashi for example is a highly tuned system optimized to read and write a tremendous amout of analytics data to Riak from our streaming engine. We also write a lot of Dropwizard applications and these services tend to push JSON around, sometimes this information is low volume configuration objects but nontheless important customer data. The code for this post is available on Github.

With the beta release of our latest feature AppVis I thought that it would be cool to provide the ability for customers to create and save unique views of their application monitoring topology.

Application Topology, Throughput & Latency: Discovered and Monitored by Boundary Application Topology, Throughput & Latency: Discovered and Monitored by Boundary

I decided that I'd write this service in Scala and use Riak for persistence. I'm lazy and I can't stand doing CRUD stuff so I looked around at our code at Boundary and on the internet and I didn't find any simple reusable persistence layer in Scala for Riak so I decided to write my own.

Basho has a nice Riak Java client so I decided to use this for my client libraries.

GenericKeyValueDAO trait.

Our base trait is the GenericKeyValueDAO, Riak is a key/value store so this trait provides persistence and deletion based on keys. The DAO is a well tread pattern so nothing really new here, just notice we're specifying our parameterized key and type with K and T.

AbstractRiakEntityDAO Implementation

AbstractRiakEntityDAO is a parameterized class that requires a reference to a RiakStorageDriver. AbstractRiakEntityDAO extends the GenericKeyValueDAO trait and implements the Converter interface for our type T. The Converter provides the logic required to transform an IRiakObject into domain objects and vice versa.

Also of note you'll see we've implemented a few 2i related methods. Riak supports secondary indexing so the findFor2i and deleteFor2i methods provide us the ability to retrieve or delete entities on criteria other than primary keys.

The Riak Driver

Here's where the majority of our work gets done. The RiakDriver class is parameterized with our domain model type and accepts a string for our Bucket and a reference to an IRiakClient. By splitting out the driver layer we can quickly swap driver implementations.

For example the Riak Java client also supports a lower level client called the RawClient but for this post we're just going to focus on the IRiakClient. I've also implemented an InMemoryDriver for use in unit testing without running Riak.

Interesting bits, using Converters and 2i methods

As previously mentioned methods like getByKey and persist use the Converter to tranform data from IRiakObject to domain models. We'll get into the Converter interface later. Methods like findFor2i and deleteFor2i accept an indexed field and value and use the fetchIndex(BinIndex.named()) functions to retrieve matching Riak entities and either convert them to model objects or in the case of the latter delete them.

Implementing the Converter and an example

Let's look at the unit test AbstractRiakEntityDAOSpec for an example of implementing a Converter and extending the AbstractRiakEntityDAO. We're creating a case class called Guitar and a GuitarDAO that extends the AbstractRiakEntityDAO for persistence. The Riak Java client cookbook has an example of storing our object as a byte[]. I'm going to save the data as JSON using the Jerkson library. After that all we need to do to is implement our two Converter methods.

Note: because we're using Scala case classes and not Java objects and runtime retainable annotations that are cross platform must be written in Java we can't use the @RiakIndex and @RiakKey annotations to discover these attributes. Therefore we have to manually add indexes with the .addIndex() method. We can now install Riak and run our unit tests to verify our persistence layer works

Being really lazy or one Converter to rule them all

If you're anything like me you're probably thinking that's cool but that converter code is probably going to be pretty much the same every time so it would be nice not to have to create a DAO per class. Well I got you covered let's check out the RiakJSONEntityDAO.

Here we use a Manifest to gain access to the erasure of the type and reflection to set our 2i fields. We can now we simply create a new instance of a RiakJSONEntityDAO in one line of code and we're ready to go.

comments powered by Disqus