Couchbase Java SDK 2.2 Developer Preview 1

On behalf of the SDK Team, it is my pleasure to introduce the next big release for the Java SDK, the 2.2 version. The team has been working hard to bring you new awesome features, and today we think it’s time to shed some light on them by releasing a first developer preview.

2.2 brings a host of new features:

• Improved N1QL support
• Simple Entity Mapping
• Various API enhancements
• Helper classes for simplified retry handling
• Version bumps of some dependencies

Jump to the conclusion if you can’t wait to get your hands on the code :)

N1QL-related changes

Recently, Couchbase Server v4.0 Developer Preview was released. Among the many features of this new version, the integration of our new query language “SQL for Documents” (codenamed N1QL) is a major one.

In previous versions of the Java SDK, we already had support for N1QL (especially for the 4th separate Developer Preview that was released by the Query Team earlier this year), and we continued in this way.

Since N1QL DP4, we’ve extended the DSL, added a DSL for index management and added bucket password support.

DSL Extension

Various parts of the DSL have been extended and improved. We’ve made it so the whole N1QL Tutorial can be implemented through the Java DSL.

The FROM clause can refer to Document IDs via the USE KEYS clause, while other clauses (like JOIN or NEST) on the other hand refer to Document IDs via ON KEYS clause.
The existing DSL for that was just producing KEYS keyword, which is not enough, so both variants were re-implemented as useKeys/useKeysValues and onKeys/onKeysValues in the DSL.

In N1QL, we have the concept of collection predicates (or comprehensions) with the ANY, EVERY, ARRAY and FIRST operators. We’ve introduced a mini-DSL, Collections, to build such predicates and produce an Expression that represents them, to be used in the rest of the DSL.

In some cases, N1QL offers syntax alternatives and defaults. Three of them come to mind as we’ve tackled both a bit differently:

• the AS aliasing syntax can sometimes omit the keyword altogether, like in FROM default AS src vs FROM default src. We elected to only produce the explicit form with the AS keyword (since the user has to call the as(String alias) method anyway).
• USE KEYS and ON KEYS have optional alternative syntax USER PRIMARY KEYS and ON PRIMARY KEYS. These are semanticaly equivalent and we only produce the first form.
• ORDER BY has a default ordering direction where one doesn’t explicitely ask for DESC nor ASC. We’ve implemented this option in the Sort operator with the def(Expression onWhatToSort) method.

A few functions, some of which were previously implemented in the Functions class, have been added/moved to the ...query.dsl.functions package, in separate helper classes that match the category of the functions inside (eg. StringFunctions).

In Expression, we added a few factory methods to construct Expressions from a Statement (like doing a sub-query) and constructing a path:

We also added arithmetic operators add, subtract, multiply and divide:

Finally, we added another mini-DSL for the CASE statement. CASE can be used to produce alternative results depending on a condition. Either the condition is made explicit in each alternative WHEN clause (the expression there is a conditional one, and this form of CASE is called a “search case”), or the condition is just equality with an identifier/expression given at the beginning of the CASE (like CASE user.gender WHEN "male" THEN 1). Here is an example of the DSL in action:

Index Management DSL

If you have already played with N1QL, you know that you can create indexes to significantly speed up your queries. Up until now, the only way to create them programatically was to issue a raw String statement. Not any more!

The index management DSL offers support for the various operations that relate to indexes, namely:

Creating an index:

Creating a primary index:

Dropping indexes:

Building indexes that were deferred upon creation:

Notice how most identifiers (index names, namespace/keyspace) are automatically escaped by backticks.

Bucket Password Support

When querying the N1QL service, we now automatically enrich the request headers with authentication information obtained from the calling Bucket, which allows to query a password-protected bucket transparently (as was already the case for ViewQuerys).

Entity Mapping

This feature has been requested a lot in the past: the ability to easily map a domain object to a Couchbase document, and vice-versa.

We have started exploring this Entity Mapping feature (or ODM, Object Document Mapping) in 2.2. The approach is to provide an API similar to Bucket dedicated to ODM. This API is described in the Repository interface, and one can obtain a repository from a bucket by calling bucket.repository().

Methods in the repository API deal with a new type of Document, the EntityDocument. It sticks to the Document semantic that separate metadata from content (your domain object in this case). The domain object is expected to be annotated (see below) for mapping to work.

EntityDocument can be used to explicitely give an id, to be used as primary key in Couchbase, but contrary to the other Document implementations you’re used to, this is optional. As an alternative, you can annotate a String attribute in your domain class with @Id to use its as document id. @Id attribute is never stored into the JSON content in Couchbase…

Only attributes that are marked with the @Field annotation are taken into account and made part of the JSON content in database/injected back into the object upon get. This annotation can also bear an alias for the field name:

Current limitations (some of which may be lifted in the future) are:

• @Field is only supported on basic type attributes (the ones that can be added to a JsonObject), and in general this ODM is for simple cases. Support for Maps, Lists and custom Converters is in the works.
• only classes with zero-arg constructors are supported for ODM.
• mapping is done via reflection to get/set values of the attributes the first time (it is internally cached after).

Enhancements on the API

The existing Bucket API has seen a couple of enhancements.

First, a new operation has been introduced to easily check for the existance of a key without paying the cost of actually getting the content of the document. This is the exist(String key) method:

Secondly, a feature related to views that was present in the 1.x generation of the SDK but not the 2.x one is the ability to request a bulk get of the documents when doing a view query (the includeDocs option).

This is not really hard to do in 2.x when using the async API (using the bulk pattern that relies on RxJava), but it is lacking when using sync API. Indeed, the only way to get each row’s corresponding document in sync mode is to call row.document(), which will block and fire one request per row, serially. This is pretty inefficient!

So we’ve reintroduced the includeDocs option in the view querying API. This will trigger efficient async loading of each row’s document in the background, from which the blocking API benefits as well (the rows will already have the Document cached when calling document() on them). You can also use it on the asynchronous API, but there it is more or less a “noop” when you call .document() on the AsyncViewRow.

Retry helper

This is actually something that was part of release 2.1.2, but it deserves mention in a blog post.

One of the benefits of using RxJava is that you can meaningfuly compose Rx operators and benefit from advanced error handling primitives. Among them is the retry and retryWhen variants, that allow to retry an asynchronous flow on various conditions.

For example, you may want to retry a get request when receiving BackpressureException (the rate at which you request is too high for the SDK/server to cope) or TemporaryFailureException (the server notified that it was too busy and that requests should be delayed a little bit). You may only want to retry up to 4 times, and wait a bit before doing so… Maybe you even want this delay to grow between each attempt (Exponential Backoff)?

Previously this required a little bit of Rx knowledge to implement, but since this is such a common use case we’ve brought you helper classes to easily do that, in the com.couchbase.client.java.util.retry package.

The helper can either produce a Function that can be directly passed to an Observable‘s retryWhen operator, or it can also wrap an existing Observable intro a retrying one.

To wrap an Observable, use Retry.wrapForRetry(...) static methods. These will allow you to cover basic requirements like retrying for a maximum number of attempts, describing the kind of Delay you want between attempts.

For even more advanced use cases, there’s the RetryBuilder (which produces a function to be passed to Rx’s retryWhen operator). This builder allows you to:

• choose on which kind of errors to retry (either any(), anyOf(...), allBut(...)).
• choose how many times to retry (either once() or max(n) times).
• customize the Delay and on which Scheduler to wait (delay(...) methods).

Here is a complete example that matches the requirement expressed above:

Dependency updates

In this version, RxJava has been bumped to version 1.0.9. We’ve also upgraded the internal dependencies (which are repackaged, so it’s transparent to your application) to the latest bugfix releases.

Conclusion

We hope that you will enjoy these new features of the 2.2 release. As always, we welcome feedback, especially on the Entity Mapping feature and direction. So go play with it, and tell us what you think on the forums for instance!

To get your hands on the 2.2-dp, use the following in a Maven pom.xml (or directly download the jars for core and client):

Of course, we also fixed bugs in this developer preview, that will also be part of 2.1.3 official release scheduled next week.

We’re not done for 2.2 yet! We’ll have plenty of features upcoming, including more N1QL support, and built-in profiling and metrics.

Happy Coding!

– The Java SDK Team