Attachment with couchbaseLite Attachment v/s Syncgateway Admin API Attachment

Mobile Sync Gateway
1

We have two sets of attachment documents in couchbase server , one is created by Mobile(couchbase Lite) which looks like as below

{
2C3KA63H88H294740-7.jpg: {
@type: “blob”,
content_type: “image/jpeg”,
digest: “sha1-VtRRBo+zPVZCxOdMZVjBgYuVqwg=”,
length: 3231323
},
2C3KA63H88H294740-8.jpg: {
@type: “blob”,
content_type: “image/jpeg”,
digest: “sha1-kG4Jz6LV7OB30s1e/h9R0pRHUMM=”,
length: 2924523
},
attachments: {
blob/2C3KA63H88H294740-7.jpg: {
content_type: “image/jpeg”,
digest: “sha1-VtRRBo+zPVZCxOdMZVjBgYuVqwg=”,
length: 6417888,
revpos: 9,
stub: true
},Preformatted text
blob_/2C3KA63H88H294740-8.jpg: {
content_type: “image/jpeg”,
digest: “sha1-kG4Jz6LV7OB30s1e/h9R0pRHUMM=”,
length: 2924523,
revpos: 9,
stub: true
}
}

However, if document is created from AdmiAPI

{
attachments: {
blob/2C3KA63H88H294740-7.jpg: {
content_type: “image/jpeg”,
digest: “sha1-VtRRBo+zPVZCxOdMZVjBgYuVqwg=”,
length: 6417888,
revpos: 9,
stub: true
},Preformatted text
blob_/2C3KA63H88H294740-8.jpg: {
content_type: “image/jpeg”,
digest: “sha1-kG4Jz6LV7OB30s1e/h9R0pRHUMM=”,
length: 2924523,
revpos: 9,
stub: true
}
}

It does not seem to be adding any top section while adding through admin API. which was not really problem in terms of storage; however, document added through AdminAPI when it is being sync’ed to Mobile device. Couchbaselite SDK uses other section to download the image.

Blockquote

InputStream is = getAsset(“avatar.jpg”);
if (is == null) { return; }
try {
Blob blob = new Blob(“image/jpeg”, is);
newTask.setBlob(“avatar”, blob);
database.save(newTask);

Blob taskBlob = newTask.getBlob("avatar");
byte[] bytes = taskBlob.getContent();

} catch (CouchbaseLiteException e) {
Log.e(TAG, e.toString());
} finally {
try { is.close(); }
catch (IOException ignore) { }
}

I am looking for some kind of implementation for CBL so attachments could be downloaded (It is one way sync from server to mobile)

Any help is greatly appreciated.

2

@abhi.chouksey, When you say admin API, Is it Sync gateway Admin API you try to add and you are not able to get first section?

Here is what you can try to add attachments through CBL

mutable_document =
InputStream stream = args.get(“stream”);
if (stream != null) {
blob_value = new Blob(“image/jpeg”, stream);
}
mutable_document .setBlob( “_attachments”, blob_value)
database.save(document)

3

@ sridevi.saragadam
@househippo
@priya.rajagopal
Thanks for the reply.

I was wondering though is this best way to handle that?

based on documentation

Attachments persisted in a 1.x database are copied to the new database. NOTE: The relevant Couchbase Lite API is now called the Blob API not the Attachments API.

https://docs.couchbase.com/couchbase-lite/2.5/java.html

The functionally is identical but the internal schema for attachments has changed. Blobs are stored anywhere in the document, just like other value types, whereas in 1.x they were stored under the _attachments field. The automatic upgrade functionality does not update the internal schema for attachments, so they remain accessible under the _attachments field. The following example shows how to retrieve an attachment that was created in a 1.x database with a 2.x API

Dictionary attachments = document.getDictionary(“_attachments”);
Blob blob = attachments != null ? attachments.getBlob(“avatar”) : null;
byte content = blob != null ? blob.getContent() : null;

based on document it also indicates that attachment is now changed to blob API.

copied text from documentation

‘We’ve renamed “attachments” to “blobs”. The new behavior should be clearer too: a Blob is now a normal object that can appear in a document as a property value. In other words, you just instantiate a Blob and set it as the value of a property, and then later you can get the property value, which will be a Blob object. The following code example adds a blob to the document under the avatar property.’

Before we go to suggested route, just wanted to make sure, no other changes in couch base release pipe line which could potentially break this fix?

4

I am unclear on the issue and what “suggested route” you are referring to. If you are 2.x, that text comparing and contrasting with 1.x attachments should not be relevant to your original question ? The formatting in your posts are messed up making it quite difficult to interpret what is quoted and what is your original post.

5

@priya.rajagopal
@sridevi.saragadam
@househippo
I will try to explain the problem right from the start

Attachments can be created either via AdminAPI or through couchbaseLite API(from mobile); however, document with attachments is created little differently

{
    2C3KA63H88H294740-7.jpg: {
       @type: “blob”,
       content_type: “image/jpeg”,
      digest: “sha1-VtRRBo+zPVZCxOdMZVjBgYuVqwg=”,
      length: 3231323
},
    2C3KA63H88H294740-8.jpg: {
       @type: “blob”,
       content_type: “image/jpeg”,
      digest: “sha1-kG4Jz6LV7OB30s1e/h9R0pRHUMM=”,
     length: 2924523
},
_attachments: {
    blob_1: {
                  content_type: “image/jpeg”,
                  digest: “sha1-VtRRBo+zPVZCxOdMZVjBgYuVqwg=”,
                  length: 6417888,
                  revpos: 9,
                 stub: true
   }, 
  blob_2: {
             content_type: “image/jpeg”,
             digest: “sha1-kG4Jz6LV7OB30s1e/h9R0pRHUMM=”,
            length: 2924523,
           revpos: 9,
         stub: true
 }

}

Document created by Admin API

{
_attachments: {
      2C3KA63H88H294740-7.jpg: {
                  content_type: “image/jpeg”,
                  digest: “sha1-VtRRBo+zPVZCxOdMZVjBgYuVqwg=”,
                  length: 6417888,
                  revpos: 9,
                  stub: true
}, 
     2C3KA63H88H294740-8.jpg: {
                 content_type: “image/jpeg”,
                 digest: “sha1-kG4Jz6LV7OB30s1e/h9R0pRHUMM=”,
                 length: 2924523,
                 revpos: 9,
                 stub: true
}
}

If you notice, i both cases _attachments section has been added however, in first JSON you see extra key with attachment detail have been added.

From Couchbase Lite following a snippet of code which describe how to read/write attachment from/to a document. if keenly observed it uses the file name (from the first section file name and not the blob from attachment section) .

InputStream is = getAsset(“2C3KA63H88H294740-8.jpg”);
if (is == null) { return; }
try {
Blob blob = new Blob(“image/jpeg”, is);
newTask.setBlob(“2C3KA63H88H294740-8.jpg”, blob);
database.save(newTask);


Blob taskBlob = newTask.getBlob("2C3KA63H88H294740-8.jpg");
byte[] bytes = taskBlob.getContent();


} catch (CouchbaseLiteException e) {
Log.e(TAG, e.toString());
} finally {
try { is.close(); }
catch (IOException ignore) { }
}

When Document with attachments is created through Admin API(because couchbase SDK does not support attachment, we have to create using admin API). Since document does not contain first section (which is expected to CBL to read the attachment).

At the time when i first wrote this on (may-14) I was trying to find a solution which i have found

Use CBL Attachment API and read the attachment as attachment dictionary. That would work if mobile just reads document from Server; however, in order to read document created by mobile, then implementation would look different (because of new blob concept)

Dictionary attachments = document.getDictionary("_attachments");
Blob blob = attachments != null ? attachments.getBlob("avatar") : null;
byte[] content = blob != null ? blob.getContent() : null;

Based on available documentation, my understanding is now attachments are renamed to blobs, it changed the behavior that how documents will appear as property value.

Please refer to https://docs.couchbase.com/couchbase-lite/2.6/java.html#blobs.

Why is there two different sets of behavior for creating a document, when a document created from CBL it uses blob and property value for a file name; however, when document is created, why does it use just file name under attachment and not a blob?

It seems a bug from SGW admin side, should not it create the document as it creates from CBL.(afterall CBL uses public API behind the scene) ?

I hope i am clear this time, Please let me know if it is not.

6

That assumption is incorrect. Couchbase Lite does not connect via the public API. When you make calls via CBL API, it is interfacing with the local database. Changes gets synced over to the Sync Gateway opportunistically . That’s how the system is designed for get offline-first support (if CBL was merely interacting with Sync Gateways public API, that presumes a connected model and no offline storage)

As far as the reason for disparity , the Sync Gateway handles it slightly differently to remain backwards compatible with 1.x couchbase lite clients as Sync Gateway supports both 1.x and 2.x clients. The Sync Gateway automatically handles blobs created by couchbase lite 2.x as well as attachments created via REST API (which is equivalent to how attachments were handled in 1.x) . It unifies it in the background so you don’t have to worry about it.

As you have identified, the support is not built-into Couchbase Lite (we probably have room to improve that in a future version)

So for now, Couchbase Lite apps must be implemented to handle both versions

So your code would look something like this (this is swift, map to Java)

// First attempt to extract  2.x style blobs
if let imageVal = userVal.blob(forKey::"image")?.content {
               userRecord.imageData = imageVal 
 }
  else {
            // Handle campatability with attachments created via sync gateway REST API
           // Example REST command would have been : curl -X PUT  'http://localhost:4985/userprofile/docId/blob_%2Fimage?rev=1-736642bf90b0badeb9833711ff240045'  
            // Those attachments are created with 1.x style.
            let attachments = userVal.dictionary(forKey: "_attachments")
            let imageVal = attachments?.blob(forKey:"blob_/\( "image)")?.content
           userRecord.imageData = imageVal
 }

When Document with attachments is created through Admin API(because couchbase SDK does not support attachment, we have to create using admin API).

“Admin API” ? You should probably use the public API unless you are guaranteeing that the requester is a trusted source.

7

@priya.rajagopal
That clarifies lot of things. Thank you! Is there a possibility at SGW side to configure compatibility mode to only care for CBL2.X?

8

No because as mentioned the REST API compatible with 1.x protocol. So if you are using the REST API, you are using 1.x style attachments.

9

I truly appreciate your help! Thank you! In future, is there any changes related to attachment + SGW + CBL in couchbase release pipeline that we should be aware of?