This blog post is based on an earlier blog post by Jeff Morris that covered the sub-document API while it was still in developer preview. There have been some changes to the API since that release.
With Couchbase Server 4.5 and the .NET SDK 2.3.x, you can now use the Sub-document feature in your .NET application.
In previous Couchbase releases, all document mutations were atomic and involve the entire document. If you only want to change a single field and then do an update, the entire document in Couchbase server is copied over by the new revision. The problem with is that if the document is large or the network slow (or both), then a lot of resources are wasted sending data that hasn’t been modified. A better, more performant solution would be to just send the portion of the document or the value which has been mutated. Essentially, that is what you get with sub-document API; when you update an element or delete an element of a document, only the path of the fragment to be mutated is sent over the wire and only that part of the document is modified.
There are several different operations that are supported by the API, from mutations on individual nested elements (aka sub-documents) to array and dictionary modifications. Counter operations are also supported, as are retrieval operations for embedded JSON fragments.
The API is exposed via a fluent interface which allows you to append multiple operations and then execute them against the document atomically. There are two different “builders”: a builder for mutation operations and a builder for reads or “lookups” (which can also check if an element exists at a given path).
Prerequisite: Couchbase Server 4.5
In order to follow the examples below, you’ll need to download and install Couchbase Server 4.5. If you’ve never installed Couchbase Server before, you can check out my video on how to install Couchbase Server on Windows. It’s really easy, no matter what OS you are using.
Sub-Document API Overview
The following examples will use a document with an id of “puppy” and will start out looking like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
{ "type": "dog", "breed": "Pitbull/Chihuahua", "name": "Puppy", "toys": [ "squeaker", "ball", "shoe" ], "owner": { "type": "servant", "name": "Don Knotts", "age": 63 }, "attributes": { "fleas": true, "color": "white", "eyeColor": "brown", "age": 5, "dirty": true, "sex": "female" } } |
All of the examples are available on Github so you can clone the project and play around with the API.
MutateInBuilder and LookupInBuilder
The Sub-document API offers two new Types that utilize a builder pattern via a fluent-interface for chaining together multiple operations on a document. Both objects are created by calling MutateIn
or LookupIn
on a CouchbaseBucket
object and passing in the key of the document you are working against:
1 2 3 4 5 6 7 8 9 10 11 |
//Initialize the cluster helper with the default settings - i.e. localhost ClusterHelper.Initialize(); var bucket = ClusterHelper.GetBucket("default"); //create a mutation builder for the document "thekey" var mutate = bucket.MutateIn("thekey"); //create a lookup builder for the document "thekey2" var lookup = bucket.LookupIn("thekey2"); ClusterHelper.Close(); |
Once you have the builder object, you can chain together a number of operations to execute against the document, for example:
1 2 3 4 5 |
var builder = bucket.LookupIn(id). Get("type"). Get("name"). Get("owner"). Exists("notfound"); |
Then you can send all of the operations to the server in a single batch:
1 |
var fragment = builder.Execute(); |
You can check the result of one operation, by using OpStatus and a path. In this example, I’m using the path "type"
:
1 2 3 4 5 |
if (fragment.OpStatus("type") == ResponseStatus.Success) { string format = "Path='{0}' Value='{1}'"; Console.WriteLine(format, "type", fragment.Content("type")); } |
These are some of the methods and fields you’ll find on the IDocumentFragment
interface.
Name |
Description |
Content(…) |
Gets the content for a given path or index. |
Exists(…) |
Returns true if there is a result for a given path or index. |
Count() |
The count of current operations maintained by the builder. |
OpStatus(…) |
The |
Status |
The |
Success |
True if the entire multi-operation succeeds. |
Besides these properties or methods, there are all of the other properties inherited from OperationResult
(which is the standard response from a key/value operation): Upsert, Remove, Replace, and so on.
Error Handling
When sending multiple mutations, if one of them fails, the entire multi-operation request fails. This allows transactional “all-or-nothing” semantics when performing mutations within a single document.
When sending multiple lookups, some operations may succeed and some may fail, with the server attempting to return as many items as requested.
If the operation(s) failed, then theStatus
property will contain a top-level error response such as SubDocMultiPathFailure
. This is an indication that you should dig deeper into the operation’s results to get the specific error. You can do this by iterating: calling the OpStatus
method and passing either the index or the path:
1 2 3 4 5 6 7 8 9 10 11 |
var builder = bucket.LookupIn(id). Get("type"). Get("somepaththatdoesntexist"). Get("owner"); var fragment = builder.Execute(); Console.WriteLine("Generic error: {0}{1}Specific Error: {2}", fragment.Status, Environment.NewLine, fragment.OpStatus(1)); Console.WriteLine("Generic error: {0}{1}Specific Error: {2}", fragment.Status, Environment.NewLine, fragment.OpStatus("somepaththatdoesntexist")); |
In this case, since the path “somepaththatdoesntexist” didn’t exist within the document, the specific error returned was SubDocPathNotFound
. There are many different combinations of errors depending upon the builder type and the condition for the error.
LookupInBuilder Examples
The LookUpInBuilder
type supports two operations: fetching a value by path and checking for the existence of a value at a given path.
Get:
Let’s lookup the owner
fragment. If I pass in “owner” as the path parameter to this method…
1 2 3 4 5 6 7 8 9 |
public static void GetExample(IBucket bucket, string path, string id) { var builder = bucket.LookupIn(id). Get(path). Execute(); var fragment = builder.Content(path); Console.WriteLine(fragment); } |
…the output to console would be:
1 2 3 4 5 |
{ "type": "servant", "name": "Don Knotts", "age": 63 } |
Exist:
We can also check to see if a path exists. If I pass in “owner” as the path to this method…
1 2 3 4 5 6 7 8 9 |
public static void ExistsExample(IBucket bucket, string path, string id) { var builder = bucket.LookupIn(id). Exists(path). Execute(); var found = builder.Content(path); Console.WriteLine(found); } |
…the output is true
, because the path owner
does indeed exist within the document.
MutateInBuilder
The MutateInBuilder offers a number of methods supporting mutations on scalar values, dictionaries and arrays, along with support for atomic counter operations.
Insert:
Insert adds a value to a dictionary, optionally allowing for the containing element (the dictionary itself) to be added.
1 2 3 4 5 6 7 8 9 |
public static void InsertExample(IBucket bucket, string id, string path, string value) { var fragment = bucket.MutateIn(id). Insert(path, value, true). // false is the default Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
If I called the above method like so:
1 |
InsertExample(bucket, id,"attributes.hairLength", "short"); |
Then the document’s attributes dictionary will now look like this:
1 2 3 4 5 6 7 8 9 10 11 12 |
... "attributes": { "fleas": true, "color": "white", "eyeColor": "brown", "age": 5, "dirty": true, "sex": "female", "hairLength": "short" } ... |
Note that the Insert
method has an optional boolean parameter called createParents
. It is false by default. If it’s true, then the sub-document API will create the necessary path for the field to exist. If it’s false, the sub-document API will only create the field if the parents of the field already exist. In the above example, the attributes
field already existed.
In this next example, I’ll use a path with a parent field (anewattribute
) that doesn’t already exist.
1 |
InsertExample(bucket, id, "anewattribute.withakey", "somevalue"); |
This will create the new attribute called anewattribute
in the document and add a single key called withakey
with a value of somevalue
.
1 2 3 4 5 6 |
... "anewattribute": { "withakey": "somevalue" } ... |
Now, if we passed false
for createParents
and the parent attribute did not exist, then the multi-mutation would fail with a top-level response status of SubDocMultiPathFailure
and the specific error would be SubDocPathNotFound
.
Upsert
Upsert will add or replace an existing dictionary entry. The usage is exactly the same as Insert
with the exception of the method name being Upsert
.
Remove
Remove
will remove an element at a given path.
1 2 3 4 5 6 7 8 9 |
public static void RemoveExample(IBucket bucket, string id, string path) { var fragment = bucket.MutateIn(id). Remove(path). Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
When I call this method:
1 |
RemoveExample(bucket, id, "owner.name"); |
Here’s how the document will look afterwards:
1 2 3 4 5 6 7 |
... "owner": { "type": "servant", "age": 63 }, ... |
Replace
Replace will swap the value of element at a given path, failing if the path does not exist:
1 2 3 4 5 6 7 8 9 |
public static void ReplaceExample(IBucket bucket, string id, string path, object value) { var fragment = bucket.MutateIn(id). Replace(path, value). Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
After I call this method:
1 |
ReplaceExample(bucket, id, "owner", new { CatLover=true, CatName="celia"}); |
The document will now have a different value for “owner”:
1 2 3 4 5 6 7 |
... "owner": { "catLover": true, "catName": "celia" }, ... |
ArrayAppend
ArrayAppend adds a value to the end of an array, optionally adding the parent element (the array element itself) if it doesn’t exist.
1 2 3 4 5 6 7 8 9 |
public static void ArrayAppendExample(IBucket bucket, string id, string path, object value) { var fragment = bucket.MutateIn(id). ArrayAppend(path, value, false). Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
After that method with the “toys” path…
1 |
ArrayAppendExample(bucket, id, "toys", "slipper"); |
…the toys
array in the document will then have the value “slipper” in the last ordinal:
1 2 3 4 5 6 7 8 9 |
... "toys": [ "squeaker", "ball", "shoe", "slipper" ], ... |
ArrayPrepend
ArrayPrepend works the same way as ArrayAppend, except it adds a value to the front of an array.
1 2 3 4 5 6 7 8 9 |
public static void ArrayAppendExample(IBucket bucket, string id, string path, object value) { var fragment = bucket.MutateIn(id). ArrayAppend(path, value, false). Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
Calling that method with the “toys” path…
1 |
ArrayAppendExample(bucket, id, "toys", "slipper"); |
The toys
array now has the value “slipper” in it’s first ordinal:
1 2 3 4 5 6 7 8 9 |
... "toys": [ "slipper", "squeaker", "ball", "shoe" ], ... |
ArrayInsert
ArrayPrepend puts a value at the beginning, ArrayAppend puts it at the end. Just to round things out, you can use ArrayInsert to put a value somewhere in between (at a given index).
1 2 3 4 5 6 7 8 9 |
public static void ArrayInsertExample(IBucket bucket, string id, string path, object value) { var fragment = bucket.MutateIn(id). ArrayInsert(path, value). Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
And then calling that method with “toys[2]”…
1 |
ArrayInsertExample(bucket, id, "toys[2]", "slipper"); |
The toys
array now has the value “slipper” at it’s 3rd ordinal (index 2):
1 2 3 4 5 6 7 |
"toys": [ "squeaker", "ball", "slipper", "shoe" ], |
ArrayAddUnique
ArrayAddUnique will inserts a value into an array, but it will fail if that value already exists (i.e. the value must be unique within the array).
1 2 3 4 5 6 7 8 9 |
public static void ArrayAddUniqueExample(IBucket bucket, string id, string path, object value) { var fragment = bucket.MutateIn(id). ArrayAddUnique(path, value). Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
When I call that with “shoe”…
1 |
ArrayAddUniqueExample(bucket, id, "toys", "shoe"); |
…since the value “shoe” already exists in the original document’s toys
array, this will fail with the status SubDocPathExists
.
Note that this method only allows for JSON primitives to be inserted: strings, numbers, and special values for true, false or null. There is no way to compare for uniqueness without descending into each JSON object and comparing elements item by item.
Counter
Adds the specified delta (change) to an existing value, creating the element if it doesn’t exist. It will default both the value and delta to 0. If the delta is negative, the value of the element will be decremented by the given delta.
I’ll create a method that uses Counter
:
1 2 3 4 5 6 7 8 9 |
public static void CounterExample(IBucket bucket, string id, string path, long delta) { var fragment = bucket.MutateIn(id). Counter(path, delta). Execute(); var status = fragment.OpStatus(path); Console.WriteLine(status); } |
Then, I’ll call the method twice using a positive 1 and a negative 1 as the “deltas”:
1 2 |
CounterExample(bucket, id, "likes", 1); CounterExample(bucket, id, "likes", -1); |
After the first call, since the element doesn’t exist it will be created and then set to one (1). The document will now look like this:
1 2 3 4 |
... ], "likes": 1 } |
The second call passes a negative one (-1), so then the counter for likes
will be decremented to zero (0). The JSON document will now look like this:
1 2 3 4 |
... ], "likes": 0 } |
Conclusion
Since the developer preview blog post, the Couchbase .NET SDK has been updated to version 2.3.2 (as of the time of this blog post). You can check out the work that’s been done in the Release Notes of Version 2.3.2.
How to get v2.3.x
- Download the binaries for .NET SDK 2.3.x from our repository (2.3.3 is the latest release).
- You can find the latest SDKs on NuGet.
- The source code is available on Github
Final Notes
The sub-document API gives you the ability to be more granular in your interactions with documents. You can modify and retrieve just the portions that you need.
Leave a comment below, talk to me on Twitter, or email me (matthew.groves AT couchbase DOT com) if you have any questions or comments.