I’m very happy to report that Couchbase now has a supported Scala SDK, allowing you to get and fetch documents, run N1QL queries, perform analytics and full text search lookups – all with native Scala.
In this blog I’m going to touch on the key features and design principles of the Scala SDK. Or if you want to get going right away, then check out the getting started guide here. The Scala SDK is available to download right now, in a pre-release alpha form.
The Scala SDK uses the same high-performance and stable internal core as the new Java SDK 3.x, so although it’s brand new, you can rest assured that it will be production-ready come GA.
Functional Error Handling – No Exceptions!
The Scala SDK presents a functional interface, with the majority of operations returning a Scala Try. This can either be a Success and contain the expected value, or a Failure containing a Throwable. Though at first this seems a little verbose, and requires some programming discipline to always handle errors, this form of error handling quickly becomes second nature and lets you develop safe in the knowledge that a random exception isn’t going to be thrown from the SDK.
Here’s a simple demo of inserting a document:
|
1 2 3 |
val json = JsonObject("status" -> "awesome!") val result: Try[MutationResult] = collection.insert("document-key", json) |
And then pattern matching on the Try result:
|
1 2 3 4 5 |
result match { case Success(mr) => println("Upsert was successful") case Failure(err: KeyAlreadyExistsException) => println("document-key already exists") case Failure(err) => println("Error: " + err) } |
(Of course, a production app would handle errors properly, rather than just using println!)
If you’d rather have something similar to the Java SDK, which throws exceptions, this is easy to simulate by simply calling .get() on any Try, which will throw if it’s a Failure:
|
1 |
val result: MutationResult = collection.insert("document-key", json).get() |
It’s easy to combine multiple Try’s through flatMap. Here we upsert a document, then get it, retrieve its content converted into the built-in JSON library JsonObjectSafe (more on that later), and finally getting a field named “status” from that. All in a completely functional style, so for instance the get operation will only be tried if the upsert is successful.
|
1 2 3 4 5 6 7 8 9 |
val result: Try[String] = collection.upsert("document-key", json) .flatMap(_ => collection.get("document-key")) .flatMap(_.contentAs[JsonObjectSafe]) .flatMap(_.str("status")) result match { case Success(status) => println(s"Couchbase is $status") case Failure(err) => println("Error: " + err) } |
(For conciseness, most of the examples below will leave out checking the result. But production apps should of course ABC – always be checking.)
Forward Looking
Out of the box, the SDK supports the new stronger durability that will be available in Couchbase Server 6.5:
|
1 |
val result = collection.upsert(“docId”, json, durability = Durability.Majority) |
And bakes in support for collections, a new method of organizing documents inside buckets which will be a cornerstone feature of a future release of Couchbase Server (with a preview coming soon). The keen will soon be able to create some toy collections to play with, but for now it will suffice to open the default collection:
|
1 2 |
val cluster = Cluster.connect("cluster-ip", "username", "password") val collection = cluster.bucket("bucket-name").defaultCollection |
This logic will work on all supported Couchbase Server versions (5.x and above).
In addition, support is included for OpenTracing (now OpenTelemetry), the platform that’s becoming the defacto standard for tracing complex distributed systems:
|
1 |
val result = collection.upsert(“docId”, json, parentSpan = Some(mySpan)) |
Simple API
The SDK leverages named and default parameters in place of overloads, so your code can be as simple:
|
1 |
val result = collection.upsert(“docId”, json) |
or as customized:
|
1 |
val result = collection.upsert(“docId”, json, durability = Durability.Majority, expiration = 60.minutes, parentSpan = Some(mySpan)) |
as you need it to be.
Asynchronous
Like the Java SDK, three APIs are presented, giving you the flexibility to structure your application the way you need to.
First is the simple blocking synchronous API you’ve seen in the examples above. This can be a good default choice, but of course requires you to explicitly manage your own threading if needed.
Second is one built around reactive programming, using standard reactive types (Mono and Flux) from Project Reactor.
An example of upserting in a reactive way:
|
1 2 3 |
val mono: Mono[MutationResult] = collection.reactive.upsert("document-key", json) .doOnError(err => println(s"Error during upsert: ${err}")) .doOnNext(mutationResult => println("Success")) |
With reactive programming, nothing will happen until we subscribe to the Mono. The simplest way to do this for testing purposes is to use block(), though this is bad practice for a real app:
|
1 |
val result: MutationResult = mono.block() |
Reactive programming is a complex and deep subject that I can’t go into too much here, but when mastered it provides a powerful tool, especially for handling the complexities of real-world distributed systems. In particular, the reactive API provides backpressure on N1QL, FTS and analytics queries – so if the application is struggling to keep up with the data, it will automatically slow down consumption of rows and avoid out-of-memory errors. I encourage the curious to check out the Project Reactor documentation to learn more.
Finally, there is the asynchronous API built around Scala Futures, which provides a little of the thread management and composability of reactive programming, but in a form that’s more familiar to many Scala developers. A simple upsert example looks like this:
|
1 2 3 4 5 6 |
val result: Future[MutationResult] = collection.async.upsert("document-key", json) result onComplete { case Success(_) => println("Successfully upserted") case Failure(exception) => println("Error: " + exception) } |
JSON Flexibility
A key aim was to not fence you into any particular JSON implementation, and out-of-the-box the Scala SDK will play nice with several popular Scala JSON libraries, including Circe, µPickle / µJson, Json4s and Jawn. You can see snippets of how to use each of them in the JSON docs, but to whet your appetite here’s some Circe in action – note how Circe types are directly sent to, and returned from, the SDK:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
import io.circe.generic.auto._ import io.circe.syntax._ val user = User(“John Smith”, 29) val json: io.circe.Json = user.asJson val result: Try[io.circe.Json] = collection.insert("id", json) .flatMap(_ => collection.get("id")) .flatMap(doc => doc.contentAs[io.circe.Json]) result match { case Success(content: io.circe.Json) => // Handle Circe Json here case Failure(err) => println("Error: " + err) } |
Built-in JSON
Taking a ‘batteries included’ approach there’s also a simple JSON library included with the SDK, JsonObject. Its main goals are:
- Convenience. Not everyone wants to evaluate multiple JSON libraries before getting started. JsonObject is a decent default choice.
- Speed. Our internal benchmarking (source in case you’d like to run them yourself) indicates JsonObject can be up to 20 times faster than the nearest Scala JSON library on some important operations. It achieves this mostly by being built around simple, but very fast, mutable JVM data structures.
- Flexibility. The default JsonObject interface throws exceptions (the only place in the SDK that does). Or you can easily convert it to a JsonObjectSafe, which provides a functional Try-based interface.
- Ease-of-use and mutability. We find ourselves in agreement with the author of µJson that though immutability is usually desirable, it’s not always the best choice in the particular case of JSON. Dealing with deeply nested JSON requires functional tools such as lenses which are rarely easy to read and use, not to mention possibly incurring a performance penalty. And JSON is most often dealt with briefly and in a limited scope (e.g. getting and modifying a document), so rarely benefits from the safety of immutability. So JsonObject presents a simple mutable API.
You can delve into the JsonObject docs here. Of course, if you’d rather have a more ‘typically Scala’ JSON library with immutable data, algebraic data types, lenses, cursors and other functional goodies, then one of the other supported libraries may be a better choice – the choice is yours, and you can easily mix and match multiple JSON libraries in the same app.
Direct Case Class Support
Though the Scala SDK has great support for JSON, sometimes it’s easier and preferable to work directly with Scala case classes.
By just adding this tiny bit of boilerplate for your case class, the SDK can automatically generate serialization logic:
|
1 2 3 4 5 |
case class User(name: String, age: Int) object User { implicit val codec: Codec[User] = Codecs.codec[User] } |
And then you can easily send that case class directly to, and receive it back from, the SDK:
|
1 2 3 |
val r: Try[User] = collection.insert(“docId”, User(“John Smith”, 29)) .flatMap(_ => collection.get(“docId”) .flatMap(doc => doc.contentAs[User]) |
Under the hood this is converted and stored on Couchbase Server as the JSON you’d expect:
|
1 |
{"name":"John Smith","age":29} |
This is plain ordinary JSON with no extra serialization metadata or similar, so it can be freely written as a case class and read out into a JSON library or by another SDK – or vice versa.
Check out the documentation on using case classes for more.
Easy Queries
We’ve made it as easy as possible to use Couchbase Analytics, N1QL and Full Text Search. Here’s an example of a N1QL query where rows are converted into JsonObject.
|
1 2 3 4 5 6 7 8 |
cluster.query("""select * from `travel-sample` limit 10;""") .flatMap(_.allRowsAs[JsonObject]) match { case Success(rows: Seq[JsonObject]) => rows.foreach(row => println(row)) case Failure(err) => println(s"Error: $err") } |
And of course it’s possible to get those results as any of the supported JSON types above, or directly as Scala case classes.
And as mentioned above, using the reactive API means you get automatic backpressure for free, ensuring that your application consumes rows at a manageable rate and cannot hit out-of-memory issues on very large queries.
If you’re excited to get started with the Couchbase Scala SDK, then check out the quickstart guide. It’s available right now, though in an early alpha form. There may be a few breaking changes ahead before going to GA, but on the flip-side this is the perfect time to try it out and give us your feedback by dropping a comment below, or raising it on our forums, Twitter or gitter. Nothing is set in stone and we welcome the chance to make some changes and make this the best SDK possible.
English