Spring Data Couchbase: A New Generation (2.0 Preview)

The Spring Data Couchbase community project has been historically built upon the 1.4 generation of the official Couchbase Java SDK, although the SDK 2.0 has been out for quite some time.

But now is definitely a great time to upgrade Spring Data Couchbase to the latest 2.2 SDK, especially since it is the one with support for N1QL, aka. “SQL for Documents”, the new Couchbase query language.

The 2.x SDK generation is a complete rewrite, built upon a fully asynchronous layer and exposing RxJava Observables for the async API. As such it has a separate Maven artifact name, and warrants a major version of Spring Data Couchbase.

The main differences between the 1.x generation of Spring Data Couchbase and its 2.x version are:

• Configuration elements are closer to the Couchbase reality: Environment, Cluster, Bucket (potentially allowing you to create CouchbaseTemplates that each connect to a different bucket, or even cluster!)
• Backing custom methods is not always done with views anymore, it is now by default rather done via N1QL, which is much more flexible and requires far less server-side maintenance.
• Custom methods using views have been modified a little to better stick to Spring Data philosophy. This reduces flexibility but the implementations are generated from the method name (“query derivation“).
• View reduction is now supported.
• Changing the name of the JSON field that store type information (“_class“) is now supported. For example, a Sync Gateway-compatible one, javaClass, is offered in MappingCouchbaseConverter.TYPEKEY_SYNCGATEWAY_COMPATIBLE.

Of course you can still access the lower level API by using the CouchbaseTemplate rather than a CouchbaseRepository interface, and you can even access the underlying Bucket from the SDK.

Let’s dig a little bit deeper into these changes!

Repository methods through N1QL

The big new feature in Couchbase 4.0 is N1QL, a superset of SQL that works on JSON document (so it added JSON-related specificities to SQL).

This is especially great for the Repository pattern and query derivation in Spring Data, because the vast majority of query derivation keywords are also easily translated to N1QL.

N1QL is now the default backing Couchbase feature for Repository methods. You can also elect to use the @Query interface if you want to be explicit.

For example, a query-derived method using N1QL could look like:

This in turn, for parameters "fruit", "cheesecake", 5, gives us a N1QL query looking like:

As you can see, the framework will even correctly choose which fields to select (including metadata) in order to be able to unmarshall the document into a PartyCake entity.

As for views, firstX syntax for LIMIT and countBy instead of findAllBy to do a SELECT COUNT(*) are also supported.

Another advantage vs views is that you can have a single general purpose index (a “primary index” from the perspective of N1QL) and use it for all your queries, so this is less server-side tuning compared to views where each different query will need a different view.

Note: of course you can also create more specialized more efficient secondary indexes in N1QL.

Repository methods through Views

One big change in this version is that now, Repository Queries (aka custom repository methods) that are based on views are more in line with the Spring Data philosophy. They also have to be annotated explicitly with @View(viewName="something").

This means that nothing Couchbase-specific should leak into your repository interface. Instead, what you can do is use query derivation mechanisms for the most simple queries.

This means that you can do something like this:

The framework will take that interface and create a query out of the method name and the given parameters. For example calling it with a List of fruit and sugar will result in a query like ?keys=["fruit","sugar"].

Please refer to the documentation for an exhaustive list of the keywords in your method name that can get translated to a query parameter. For every other use, you should instead provide the implementation yourself.

Using reduce function from Views

Another new thing that wasn’t previously supported is the execution of the reduce function if you have one. Now, in order to execute it, you simply declare a method starting with count instead of findAll, which returns a long.

Note that the reduce function in Couchbase can be something else than the preexisting _count one, as long as it returns a long.

Similarly, adding the variation “topX” or “firstX” in a method name will result in an additional limit being set on the request (eg. findFirst5ByLastName will limit the list to 5 results).

Configuring consistency, Read Your Own Writes

One thing that comes up often when using asynchronously populated secondary indexes like views and GSI (the new secondary index engine backing N1QL), is the need to Read Your Own Writes.

This implies that the view/N1QL shouldn’t answer as long as the data is still in the process of being indexed, so this sacrifies some performance in favor of consistency.

The opposite (and current default for Spring Data Couchbase) is to favor performance by accepting stale data to be returned.

We added a global mean of configuring that for all queries (view-based or N1QL-based) that are constructed by the framework through query derivation, by providing a small abstraction around the concept of Consistency.

This is done by overriding the AbstractCouchbaseConfiguration‘s getDefaultConsistency() method. Consistency is an enum that lets you choose between READ_YOUR_OWN_WRITES, STRONGLY_CONSISTENT, UPDATE_AFTER and EVENTUALLY_CONSISTENT.

You can also do that in XML by using the consistency attribute on the tag.

Changing the type information field in stored JSON

Lastly, some users have reported issues with Spring Data and the Couchbase Mobile side of things, with the Sync Gateway rejecting documents containing fields prefixed by an underscore.

This is problematic for Spring Data stores the type information in a _class field :(

The solution is to allow, through the configuration, to choose the name of that type information field. You can do so by overriding the typeKey() method in AbstractCouchbaseConfiguration. For instance, you can use the constant MappingCouchbaseConverter.TYPEKEY_SYNCGATEWAY_COMPATIBLE (which is “javaClass“).

This field is the one used by generated N1QL queries to filter only documents corresponding to the repository’s entity.

Configuration

The 2.0 SDK is now closer to the terminology of a Couchbase cluster, with objects like Cluster and Bucket as first class citizens. As a result, in the Spring Data Configuration rather than a CouchbaseClient you configure more diverse beans.

The tuning of the SDK is done in a separate class, the CouchbaseEnvironment, which allows you to tune io pools, timeouts, etc… Environments should be reused as much as possible accross Clusters, and Clusters should be reused to open singleton-scoped Buckets (everything should be reused as much as possible, basically).

To obtain a CouchbaseTemplate, one needs an Environment, a Cluster and a Bucket. These are all automatically created when extending the JavaConfig AbstractCouchbaseConfiguration, the only thing you need to provide are:

• the list of nodes to bootstrap from (just the IPs or the hostnames)
• the bucket’s name
• the bucket’s password

Of course, if you want to override, say, the default Environment configuration, you can override the corresponding methods in the AbstractCouchbaseConfiguration (as shown in the snippet below):

The equivalent in xml is:

Setting up on the server side

Spring Data Couchbase CRUD methods obtained from a CrudRepository interface are all still backed by either the Key/Value operations of Couchbase (when working with single entities or mutating entities), or by a view (when working on multiple entities we don’t know the ID of, for instance for count() or findAll() methods).

This means that you still have to have an index of all your entities somehow, in the form of a view in Couchbase.

By default, the framework will look for the name of your entity, uncapitalized, for the design document and a view named all (eg. partyCake/all for a PartyCake entity class).

Conclusion

That’s all (for now) for this first preview version of Spring Data Couchbase 2.0.x. Hope you like it!

The documentation of the project and the README have been updated with these new features and all the code and docs are visible in the 2.0.x branch in the Spring Data Couchbase Github repository.

Thanks to Oliver from the Spring Data team for his support, and to our users that stepped in and offered contributions or improvement suggestions (not an exhaustive list):

vasilievip, KlausUnger, kdheerendra, jloisel, DWvanGeest, ajbrown, wilsondy

To download and use this preview, use Maven (with both the Spring snapshot and Couchbase snapshot repositories):