How to do binary attachments from server side through gateway to couch-base lite?

[feihtthief]

I’m new to Couchbase & NoSQL in general.

I’m trying to get to the position where we have our documents about our facilities stored in Couchbase Server, so that our existing CMS can push the relevant data into the couchbase server. They can then be made available to a xamarin forms mobile app via couchbase sync gateway. This will allow us to add additional information as needed. With a little bit of work I managed to get the documents from the Couchbase Server replicated into the gateway with the config settings of

		,"enable_shared_bucket_access": true
		,"import_docs": "continuous"

That worked great, we can make our changes to the documents in CB server and the documents arrive on the devices like magic and are available offline. So far so good.

Now I need to attach images to go with those documents, but I’m getting stuck. There is no way (that I could find) in the Server SDK to create attachments on documents.

My next approach was to try and store the documents as plain blobs by just passing the bytes of the image into a _bucket.Upsert call. That created the documents, but none of those documents seem to appear on the sync gateway. I’m not sure if this is because there are no channels in the document, or if it is something else.

My next approach was to try to create the documents via the Sync Gateway Rest API and then create attachments for them. Creating the documents worked OK, but the attachments refused to create with a response of {“error”:“Forbidden”,“reason”:“forbidden”}.

On a side note: If I try to PUT a document that already exists I keep getting a response of {“error”:“conflict”,“reason”:“Document exists”}, which strikes me as odd given that I understand a PUT to mean “Create or update document” from the documentation

Hope I have not over-loaded the question

Sync gateway 1.5
Couchbase server 5.0.1 Community edition

[priya.rajagopal]

@feihtthief

Now I need to attach images to go with those documents, but I’m getting stuck. There is no way (that I could find) in the Server SDK to create attachments on documents.

That’s right. There isn’t an SDK API that allows you to directly add attachments to documents. You have the option of embedding it as base64 encoded field associated with document or just uploading the binary blob as a document . FYI - related blog. The recommended approach is to store the binary data is an external store and reference it from the database instead of storing it in line - this is especially true for large attachments.

@feihtthief

That created the documents, but none of those documents seem to appear on the sync gateway. I’m not sure if this is because there are no channels in the document, or if it is something else.

OK. So looks like you used out one of the alternate approaches mentioned above. If the document is added to the CB bucket but isn’t getting processed by SGW, then check your import function in SGW config file.

In fact, please share your SGW config file.

@feihtthief

My next approach was to try to create the documents via the Sync Gateway Rest API and then create attachments for them. Creating the documents worked OK, but the attachments refused to create with a response of {“error”:“Forbidden”,“reason”:“forbidden”}.

Are you providing the right credentials and is there anything in your SGW config file sync function that may be preventing this upload ? Because you should be able to create a document and PUT an attachment .
Please share the exact request that you are using for creating the attachment via the SGW API.

@feihtthief

On a side note: If I try to PUT a document that already exists I keep getting a response of {“error”:“conflict”,“reason”:“Document exists”}, which strikes me as odd given that I understand a PUT to mean “Create or update document” from the documentation

You have to provide the right revision Id of the document that you trying to update via the “rev” parameter.

[feihtthief]

Hi Priya.

Thank you for the assistance.

My SGW config file:

{
	 "log": ["*"]
	,"adminInterface": "127.0.0.1:4985"
	,"interface": "0.0.0.0:4984"
	,"databases": {
		"ourapp": {
			 "server": "http://localhost:8091"
			,"bucket": "ourapp"
			,"username": "ourapp-user"
			,"password": "ourapp-password"
			,"enable_shared_bucket_access": true
			,"import_docs": "continuous"
			,"users": {
				"user1": {
					"password": "password",
					"disabled": false,
					"admin_channels": [
						"*"
					]
				}
			}
		}
	}
}

I have not specified the sync function in that config, so I think it uses a default one, as in the GW admin portal (http://localhost:4985/_admin/db/ourapp/sync) it shows this:

function(doc){
  channel(doc.channels)
}

I suspect this would have prevented the raw binary documents from getting through the sync gateway, as they would have no channel data? Is that understanding correct?

Regarding the third approach, below are the requests going to the sync gateway Rest API. I am making them with RestSharp.

Raw request to create an attachment on a document (from fiddler)

PUT http://tryst:4984/ourapp/residence_12/CarouselImageResources_1 HTTP/1.1
Authorization: Basic dXNlcjE6cGFzc3dvcmQ=
Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
User-Agent: RestSharp/106.2.1.0
Content-Type: application/octet-stream
Host: tryst:4984
Content-Length: 106
Accept-Encoding: gzip, deflate

*binary data here* 

Raw response (from fiddler)

HTTP/1.1 409 Conflict
Content-Type: application/json
Server: Couchbase Sync Gateway/1.5.1
Date: Tue, 13 Mar 2018 07:47:50 GMT
Content-Length: 47

{"error":"conflict","reason":"Document exists"}

[priya.rajagopal]

@feihtthief

I suspect this would have prevented the raw binary documents from getting through the sync gateway, as they would have no channel data? Is that understanding correct?

Yes- that seems like a reasonable conclusion.

@feihtthief

Regarding the third approach, below are the requests going to the sync gateway Rest API. I am making them with RestSharp.

Raw request to create an attachment on a document (from fiddler)

Please provide “rev” query parameter. This is the latest revision of the document that you are updating with the attachment. ( Its probably not very clear in the docs )