I’m using sync gateway to replicate documents to a mobile app running couchbase lite. When the app first starts up it initiates continuous replication. I’m seeing the initial replication of all the documents take noticeably longer and longer as the revision counts increase. I’m wondering how I can best mitigate this. The sync gateway 1.5 documentation says the minimum value for revs_limit is 20, but even with just that many revs per document the delay before they are all replicated is far longer than with single revision per doc.
The documents don’t change very frequently. Is there a way to set revs_limit lower than 20? Is it possible to expire old revisions to get rid of them after a few days to keep the count well under 20? Is there some other way to improve the performance?
What version/platform of Couchbase Lite are you using
Interesting. What is the delay ? How many documents and what is the typical size of the documents ?
Setting the revs_limit to a very small value (< 20) can have undesirable consequences as it can result in parent nodes of conflicting branches getting pruned away prematurely before the conflicts are resolved.
Compaction is the process of purging the JSON bodies of non-leaf revisions. Leaf revisions are not purged during compaction. The SGW runs the compaction process periodically to get rid of old revision bodies but
If you want to, you can invoke the compact API via the Admin interface
I’m using couchbase lite 1.4.1. I’ve read that blog post, and found the information interesting, but I didn’t see a solution to my problem. For example, right now my dev couchbase setup has 78K documents, but 980K revisions, as the documents have been changing over time. The time needed to sync all of the revision history in this state is about 10x the time it took when there was one revision per document.
The documents are small, so I suspect the overhead of downloading just the revision metadata isn’t significantly different than downloading the document itself, which explains why it makes such a huge difference. If the documents were large, the overhead would likely not be as noticeable. This also means that compaction likely doesn’t help much, since that just removes the JSON body but leaves the revision metadata.
In my case I really don’t care about conflicts, the documents are read-only on the client devices. Therefore it seems like it would be safe in this case to use a very low revs_limit. Is that true, and is there a way to set it below 20?
@davidj yeah there is no way to force Sync Gateway to reduce the revs_limit below 20. I guess in retrospect maybe there should have been a “danger zone” feature that let users override that behavior. Currently you’d have to rebuild it yourself to do so – let me know if you want more details. I believe the Couchbase Lite side has a setting to control the rev_tree side, which might have a lower limit (@jens / @pasin any idea?)
The next version of Sync Gateway (2.0) will support a “no-conflicts” mode, which AFAIK will completely eliminate the revision history. If this is a green-field app you might look into that.
Thanks @traun. I guess if it becomes enough of an issue we’ll look into doing our own sync gateway build. I’m excited about what y’all are doing with 2.0, but unfortunately we use p2p replication heavily, so I guess I’ll have to wait until that’s available in 2.1.