How do i work with attachments via couchbase server

Mobile
1

I’m using sync gateway to sync large document sets with attachments down to mobile which works find for test data sets i’ve created via the syncgateways bulk api.

For our application we need the documents to be created and edited via the web direct into couchbase server using the java sdk.
I’m able to create/edit documents direct in couchbase which get allocated to chanels in sync gateway and sync up to mobile, but cannot see how to create an attachment.
I cannot find anywhere in the documentation how i can create a binary asset in couchbase and associate it with a document so that its picked up as an attachment by the sync-gateway/couchbase-lite.
I know that i could write the document + attachments via the sync gateway but this does not seem write since everything else on the webapp is going via the sdk + n1ql etc.

Looking at the document + attachement created via sync-gateway via couchbase server looks like this.
Its not clear whether the binary is embedded in the document or stored separately.

e.g.
"_attachments": {
“photo.jpg”: {
“content_type”: “image/jpg”,
“digest”: “sha1-ee2gBR7WI0WNb+EVeM2nGeMUuzo=”,
“length”: 13161,
“revpos”: 1,
“stub”: true
}
}…

2

Up until now our advice has been that you should never write into the Sync Gateway bucket using Couchbase Server SDKs, because it will mess up Sync Gateway; only write using the Sync Gateway REST API. However, that might be allowable now if you’re using Server 5 and Sync Gateway 1.5.

But there’s no supported way to create attachments without using Sync Gateway. The attachment happens to be a separate document in the bucket with a specific ID, but don’t try to reverse-engineer the naming format or you’re liable to break in the future if/when we make changes to this. Instead, use the REST API to PUT a document with an attachment using MIME multipart or base64.

3

Thanks for your prompt reply Jens.

So save the document direct to CB using the SDK then post the binary via the REST api?
Can this be done separately from the doc or does it all need to be posted together.

Given what you say it may be best for us to always write to CB via the sync gateway so as not to break the _sync meta, though I understand that to be one of the key features of CB5/SG1.5

We want 2 way sync in our app but cannot allow the mobile to directly write to the db and allow that sync back to CB for a number of reasons.

  • We’re dealing with sensitive personal data in a multi tenanted setup and cannot trust the mobile app to write the correct data and sync directly to the db. i.e. the tenantID in the document controls the channel. the mobile app could change the tenantID thus re-assigning the channel.

  • Audit information needs to be written e.g. modified by and modified date that we cannot trust the mobile app to write and sync directly back to the DB.

  • Additional audit data, events and process are triggered on receipt of data created or updated on the mobile. Normally these would be triggered prior to the data being committed, whereas via the sync gateway we’d need to listen to the change feed and invoke process from that.

Because of these we’re only using the live replication from CB->SG->Mobile and Posting the data back.
The channels control the view that’s replicated, and a background queue posts creates and updates back to a service api which handles, audit, security, etc before writing direct to CB, or POST to SG as described above using user with Write permissions.

This appears to work in practice though i suspect we will hit some conflict issues when the mutated document stored locally on the mobile merges with the updated version posted via the api and replicated back up to the mobile.

4

The Sync Gateway REST API won’t let you add a doc with an attachment, without the attachment data itself. So you have to upload the whole thing through SG.

5

There are some nice examples in this test code of adding documents and attachments: https://github.com/couchbase/sync_gateway/blob/241dc3f26e04815db13d3c601d48ef042db5e557/rest/api_test.go

It’s also all laid out here:
https://docs.couchbase.com/sync-gateway/2.1/rest-api.html#/attachment

The steps are as follows:

  • Create a document in sync gateway using the REST API:
PUT /{bucket name}/{doc primary key}

(set the accept header to application/json and your basic auth header and make the body of the request the document json and store the revision ID that comes back in the response – the rev field in the json response)
or get the most up to date revision ID of a currently existing document from sync gateway using the REST API:

GET /{bucket name}/{doc primary key}?revs=1&revs_limit=1

(set your basic auth header and store the most recent revision ID that comes back in the _rev property of the json response).

  • Upload the attachment to that document, specifying the latest revision ID (the attachment name should be something like: blob_/filename, but url encoded, so replace the / by %2F) using the API:
PUT /{bucket name}/{doc primary key}/{attachment name}?rev={revision ID}

(set the accept header to application/json, set the Content-Type header to the type of attachment you’re sending, e.g. image/png, set your basic auth header and store the revision ID you get back).

Be aware that there are private fields that you don’t want to overwrite when you are editing documents via the REST API, e.g.

{
  "_attachments": {
    "blob_/photos/0/annotated": {
      "content_type": "bytes",
      "digest": "sha1-EZgfsMqWBgeSzukxyyvVlWmw/iY=",
      "length": 17,
      "revpos": 39,
      "stub": true
    },
    "blob_/photos/1/raw": {
      "content_type": "bytes",
      "digest": "sha1-vbx2pOWQTW07QA+EyDkih8sHBPg=",
      "length": 11,
      "revpos": 41,
      "stub": true
    }
  },
  "_id": "blah-blah-2",
  "_rev": "41-c94b95f764226f9beac57d7c58e923e1"
}

Apparently there are also _exp and _revisions fields but I haven’t seem them yet…

6

I think attachments when migrated to 2.X takes time every time we do one shot. If no please let me know.