Preview using N1QL DP4 from the Java SDK

As some of you may know, the Query Team over at Couchbase have been working hard on a new and exciting feature, N1QL, which brings the power of query languages (like SQL) to Couchbase.

They just release Developer Preview 4 of N1QL (see the blog post here), with a bunch of improvements, REST APIs, etc…

We are happy to announce that a Developer Preview of the Java SDK compatible with N1QL DP4 is now available. Query management underwent a host of changes in this version, which better reflects the state it will be in 2.1 (at which time N1QL should also gain General Availability stability).

How to Get the Code and Use the Feature

To get N1QL and activate it, please see the intro documentation! There are notably steps to activate N1QL indexing on your buckets.

UPDATE: A second developer preview of the SDK is out, pom and links below have been updated accordingly. Scroll to the bottom to see changes…

To grab the code for this Developer Preview, use the following snippet in your Maven pom.xml or download the core-io and java-client jars directly.

Since this is an experimental feature, you need to explicitly activate it in the SDK. This can be done in two ways: passing -Dcom.couchbase.queryEnabled=true as a JVM parameter or using CoreEnvironment when initializing the Cluster in code:

To reproduce this blog’s examples (which build one upon another, ie copy all the snippets in one Java class to get a full working example), you can also use the code below to generate two documents that can be queried:

What Changed Since 2.0.3 / N1QL DP3?

Most notable changes are described below.

Query interface renamed to Statement

In N1QL, a complete query is composed at a minimum of a Statement (eg. SELECT * FROM default), but can also have positional/named values and additional request parameters (like server-side timeouts, scan consistency, …).

The Query interface in previous versions was just representing this statement component, and thus has been renamed to Statement.

Query class hierarchy introduced

Query has in fact been kept, but to represent the whole N1QL query. A hierarchy of class have been introduced, representing queries from the simplest (a single statement, SimpleQuery) to more advanced ones (ParametrizedQuery, PreparedQuery).

Queries can be constructed via factory methods on the Query abstract class.

One can issue queries with advanced statements not covered by the DSL using the methods that take a String as statement, for example to ensure there’s a N1QL index on our target bucket:

Parameters for statements and queries

DP4 introduces the concept of parametrized statements. These statements have placeholders that can be filled by the server by passing their values in the query. Placeholders are either named (with the $name format) or positional (with the $1 format).

Such queries are represented by a ParametrizedQuery, taking values in the form of a JsonObject of name-value pairs for named placeholders or JsonArray of values for positional placeholders. Note: There’s currently a bug in N1QL DP4 that prevents named parameters to work, use positional parameters instead.

Furthermore, QueryParams describes all remaining query parameters that are supported by the SDK: client context ID, server-side timeouts, scan consistencies. They can be added to any request using the ad-hoc Query factory method.

Prepared Statements

Another big new feature of N1QL DP4 is the introduction of prepared statements. Such statements are built in two steps: – analyze the statement and produce a query plan. – execute the query plan and return the results.

The preparation step, returning a query plan, can be skipped after first occurence provided the plan has been cached somewhere by the user. This allows the server to skip a step which is a gain in time.

A QueryPlan can be obtained from the server by calling Bucket.prepare(s), s being any Statement. This object can be cached and reuse multiple times later.

To execute a plan, use Bucket.query(Query.prepared(plan)). Note that the prepared factory method can also accept query parameters and placeholder values (as long as the original statement had the corresponding placeholders).

Changes to result format

DP4 introduced several changes in the format of the server response, which have been reflected in the SDK, in AsyncQueryResult (the synchronous version QueryResult has similar changes):

• Results can be streamed to the client before a definitive status has been determined (eg. when processing one of the results fails, or there are non-breaking warnings). As such, a new finalSuccess() boolean method has been introduced. success() becomes parseSuccess(), allowing the user to catch an early error upfront, but the real final status of the query can only be known at the very end by waiting for finalSuccess().
• Errors can be multiple, and have different levels. As such error() is replaced by errors(), returning an Observable of errors and warnings.
• info() no longer contains errors or warning in any capacity, but rather a one shot emission of metrics like timers, number of rows, size of the response values, etc…

Note that obtaining the synchronous QueryResult now blocks until the whole response is available. Until now, it would block until the first chunk of the response was processed, potentially before rows were available, so this change may introduce a slightly bigger delay in obtaining the result (previous initial delay + the one incurred by a subsequent call to allRows()).

Changes in Developer Preview 2

This release fixes several bugs in querying:

• Fix a parsing error when the response is too large and a chunk is only made of result rows.
• Fix a blocking error leading to constant timeouts when querying for some users.

Additionally, a few changes have also been made feature-wise:

• Added accessors for the request ID (generated by N1QL server) and the client context ID (provided by the user when querying, but truncated to 64 bytes of UTF8 chars by the server).
• Added a prepare override on Bucket that accepts a String statement in order to obtain a QueryPlan for it.

Thanks to our users on the forums who provided feedback and signaled issues!

The release also contains various bugfixes and new features unrelated to N1QL, that we won’t detail here (please refer to the release notes of java-client and core-io).

Conclusion

We hope you enjoy N1QL and coding N1QL requests with the latest Java SDK. As always, we welcome feedback, suggestions, bug reports (well, not that we enjoy them to the level of introducing bugs willingly) and even contributions :)

Happy Coding!

The Java SDK Team