Riak spring-data support

[J. Brisbin]

I'd like to start working on the Riak for spring key/value data support.

Where should I start? Copy the spring-keyvalue-redis stuff and rename everything and change the internals to be Riak-specific?

What about Map/Reduce? How is that being approached in the mongo support? I see a directory for mongo in inconsequential but nothing is there yet. Has there been discussion on what that would look like?

BTW- I'm (eventually) going to be using this as job data support in a grails app that will run under tcServer on the AS/400 (IBM i5). I want a direct bridge between the AS/400 data and our NOSQL stuff in Riak.

[J. Brisbin]

Not to answer my own question (I'm still wanting some feedback here), but I'm going to start with the simple datastore first. It looks like there's a lot inside the Redis support that would be good as examples, but not necessary to start with.

I'm also looking at the Cassandra and AppEngine support as well.

This is really going to be straightforward, AFAIK. Great job on abstracting things such that we can plug things in easily!

[Mark Pollack]

Hi,

I did have a conversation with Justin Sheehy at Basho a few months ago and we saw a few areas of opportunity. I would like to catch up with him again and we can now more more towards implementation.

The first would be an abstract interface that allows you to easily switch between using the REST and the protocol buffer implementation. Defining that abstract API should be the first order of business.

Second would be supporting a variety of marshallers for the payload. At the moment in spring-redis there is support for java object serialization, but this needs to expand to support JSON, XML, protocol buffers, avro etc. There can probably be some code here moved into a commons area shared between the two projects eventually.

Third, as it relates to map reduce, I would very much like to be able to provide an out of the box repository base class that allows for the basic CRUD functionalty on an object stored in Riak. For the next level of sophistication in query support (not just findall or findbyid), the current strawman design is to follow a 'finder by example' style pattern. This is what is currently done in the Ektrop project for CouchDB and is the path that the CouchDB support in Spring will follow. (I've been in touch with the Ektorp author who will help contribute to the spring couchdb support). This is also done in the Hades project for JPA, so it is quite a portable approach to maping simple method names onto the underlying query language. See this link. (This also in part will move into the Spring Data project as its author Oliver Gierke is at SpringSource). Once one needs to get more advanced in terms of queries, then one can put the JavaScript in code somewhere (either in an annotation or link to a seperate file) or drop down to other lower level facilities.

I will start to setup repositories and other project infrastructure as it relates to Riak so that we can better collaborate. Please contact me by email so we can coordinate on those matters. mpollack at vmware dot com

Cheers,
Mark

[litius]

@Mark,

Is there a set of such common interfaces in Spring Data:

                  
                  REST
                  Protocol Buffer
                  toJson, toXml, etc.. / Jsonizable ...
                  findXyz, findByXyz, etc..

?

Last time I looked there was no real hierarchy, and Redis implementation mostly followed its own structure.

Would be really nice if we can reuse the same set of interfaces for various NoSQL / Distributed Cache implementations. Concrete implementation ( for a certain data store ) can of course provide more functionality, so the power of individual data store is available, but the set of core interfaces would be something nice to have.

/Anatoly

[J. Brisbin]

@Mark,
Would be really nice if we can reuse the same set of interfaces for various NoSQL / Distributed Cache implementations. Concrete implementation ( for a certain data store ) can of course provide more functionality, so the power of individual data store is available, but the set of core interfaces would be something nice to have.
/Anatoly

I agree and think this would likely be worth the effort.

It seems that most NoSQL stores could have a Document-oriented interface since that's such a common use-case. Redis is even almost document-like with the hashes.

Since three of those stores (Riak, Couch, and Mongo) support Map/Reduce, it's likely that we'll also need a common Map/Reduce interface.

[duergner]

One thing I wonder while I digged a little bit through the code was, if there are any plans for using Riak's Link feature to support relationships. Or if it would be welcome to implement this functionality into the existing codebase.

[Jon Brisbin]

One thing I wonder while I digged a little bit through the code was, if there are any plans for using Riak's Link feature to support relationships. Or if it would be welcome to implement this functionality into the existing codebase.

There are three methods to support links and link walking in the Riak template. I simply forgot to cover them in the reference documentation. But they're pretty easy to understand from the Javadoc:

http://bit.ly/hjFP9g