Couchbase lite replication over https

Hi guys,

We are embedding Couchbase lite in our .net MVC web projects. I have setup replications between two dbs (using CouchbaseLiteTcpListener) over http without Sync Gateway. Just wondering, how can I use https instead?

Thanks,
HS

Maybe @jens can help here?

More a job for @borrrden, our mobile .NET engineer…

This is not possible with 1.1.1 as it was too complicated to get done in time. All the instructions for Windows involve a non-automated process (and all the instructions for Mono involve a completely different process). However, if you can follow those steps then the master branch more than likely will already work for you. You just need to pass in the UseTLS flag on the listener options in the constructor for the class you mentioned. I’m just looking into ways to automate it.

You can follow this ticket that I will use to post updates about the progress of this feature. I’ve gotten HTTPS replication working with Mono, but I haven’t tried Windows yet since I’m still looking into a way to automate the above process.

Thank you very much for the prompt replies guys!

@borrrden The instructions from the link said I need to add https links to the prefix collection on the HttpListener. However I couldn’t access the collection from the CouchbaseLiteTcpListener…

I downloaded the latest master branch from GitHub - couchbase/couchbase-lite-net: A lightweight, document-oriented (NoSQL), syncable database engine for .NET. The HttpListener is declared on CouchbaseLiteTcpListener as private readonly. The prefix is hard coded to http from within the constructor. Also it looks like CouchbaseLiteTcpOptions.UseTLS flag is not used anywhere… Did I download the right version of the code?

public CouchbaseLiteTcpListener(Manager manager, ushort port, CouchbaseLiteTcpOptions options, string realm = "Couchbase") { _manager = manager; _realm = realm; _listener = new HttpListener(); string prefix = String.Format("http://*:{0}/", port); _listener.Prefixes.Add(prefix); _allowBasicAuth = options.HasFlag(CouchbaseLiteTcpOptions.AllowBasicAuth); }

Sorry I forgot that this was not on the master branch, but on the issue branch. Check out the issue/433 branch for the work in progress.

Thanks for the tips @borrrden. I have been trying to build Issue/433 but without luck… Getting error while trying to recursively checkout submodule sqlite3-unicodesn… Any hint?

$ git submodule update --init --recursive Cloning into 'vendor/sqlite3-unicodesn'... fatal: unable to connect to github.com: github.com[0: 192.30.252.128]: errno=No error

Clone of ‘git://github.com/snej/sqlite3-unicodesn.git’ into submodule path ‘vendor/sqlite3-unicodesn’ failed
Failed to recurse into submodule path ‘src/Couchbase.Lite.Shared/vendor/cbforest’

It looks like your machine is unable to handle git:// style addresses. There is a quick git config setting you can add to help that out though. See more details on this stack overflow post.

So close… Got Issue/433 up and running in my project and bound ssl certs to the ports. The app is doing Post to the replication URL instead of Get…

Request

POST https://localhost:9992/exampledb2/_changes HTTP/1.1
Content-Type: application/json
Host: localhost:9992
Content-Length: 79
Expect: 100-continue
Accept-Encoding: gzip, deflate
{“feed”:“longpoll”,“heartbeat”:300000,“style”:“all_docs”,“since”:0,“limit”:500}

Response

HTTP/1.1 405 method_not_allowed
Content-Length: 43
Content-Type: application/json
Server: Couchbase Lite Unofficial (master: 7c86c34) Microsoft-HTTPAPI/2.0
Date: Fri, 15 Jan 2016 04:43:56 GMT

{“status”:405,“error”:“method_not_allowed”}

Also noticed the db files created by running from the nuget package and from the dll I built from the branch are different…

From nuget

From DLL built from branch

That actually looks like a listener bug that may have been introduced between 1.1.1 and the current master. _changes is supposed to be POST now, and not GET but the listener may be mistakenly expected GET. I’ll check on that.

As for the database difference, the file layout is different for 1.2 than it is for previous versions. You don’t need to worry about that.

I had a look and indeed the listener is still expecting GET. I’ll just change it to expect POST and then rebase the issue branch. It should be ready in under 30 minutes.

You, sir, are a legend :smiley: We must be in the same time zone…

Changing it to POST required a lot more changes than I thought so I restored the fallback to GET instead. The fallback heuristic is too broad and will cause a GET to anything other than Sync Gateway for now, but I will look into either supporting POST on the listener, or or finding some other way to check whether or not the remote host supports POST (For example, Cloudant doesn’t). There is a chance it will be after the 1.2 release, since we’ve already reached feature freeze. GET was restored in the new commit 428d4d5, and the issue branch has been rebased off of that so you will need to delete and re-pull it.

@borrrden Just tried and it is all working over https now. Thank you!

I will keep tracking Issue/433. Couchbase lite is now part of our application so it would be great if the https stuff get included in the next release so we can update :relaxed: Thanks again.

The question on stack overflow was from me. Would it be ok if I link this post as the solution?

@HenryS

I’ve run into severe problems making HTTPS work on Mono, so I can’t officially say it will be supported in 1.2. However, the changes I made on the issue/433 branch will be included (in fact that branch was merged into master and it no longer exists) so if it works for you now it will continue to work that way after 1.2 (assuming you don’t switch to Mono for some reason). I’ll update here if the changes get backed out for some reason, however the only change needed for your use is the change that uses the UseTLS flag. Everything else revolves around trying to make the manual process you followed more automated.

@borrrden,

Sorry about reusing this post… Let me know if you prefer me to start a new post.

I am reading about deleting a document actually just creates a new revision called a “tombstone”… I understand it is for replication to work properly…

However, one of our security requirements are, we actually need to delete the record off the hard drive after a user deletes a record from our website… Does Couchbase Lite only keep those “deleted” records for a certain amount of time? Is there a way to permanently delete the record off the hard drive periodically and still have the replication working properly?

Thanks,
Henry

Couchbase Lite does not delete these entries automatically, however you can use the Purge() mechanism to achieve the result of deleting them out of the local database. Just be careful because then you enter a slippery slope of the local database thinking you don’t have the document, and purging it doesn’t prevent it from being pulled back down again in the future if it is modified again. Furthermore, the tombstone entries have no body in them and after a compaction (you can achieve this by calling Compact()) there will be no data left except for the document ID and revision ID so unless the document ID is sensitive then you might not have to worry.