FLASHBACK :
We download almost 60000 documents when the app starts for the very first time and 30000 documents every week in the enterprise app we have developed using CBL 1.4.1 and Android.
No matter what we did we never been able to download more then 100 Docs per second even after having multiple no network lag, more then 2 sync gateway , latest couchbase server with 3 nodes for mere 100k documents. more then enough RAM and everything.
It was heart breaking but we found some work arounds like creating a service which will have a couchbase lite inside it and will provide the pre populated latest database to the app whenever it will install and also every week. But then this got a lot of issue.
We became sure that this is not an issue of Couchbase server or Sync gateway. It was JVM and polling in CBL 1.4.1 which never let us be in peace.
BIRTH OF COUCHBASE LITE 2.X:
When couchbase 2.0 came the good news started coming like linq like queries , websocket support for android etc. but we were not ready to adopt this as there were a lot of breaking changes and deadline was near. Also peer to peer and encryption was not available.
Come 2.1.1 : I tries creating a simple app which downloads documents and displays documents and time it took to download. This app is created in android and CBL 1.4.1 for 1 version and 2.1.1 for another.
Results CBL 2.1.1 without network lag is 6 to 10 times faster than 1.4.1.
You heard it right 6X to 10X faster. The web sockets along with CBL core in C++ is amazingly fast. And these results are with a very basic machine where couchbase and sync gateway both are on same machine. The only catch is look for the network delay between sync gateway and your app’s network.
60K documents were downloaded under a minute without any fancy work around. The best part is speed keeps on increasing opposite to 1.4.1.
I’m so happy to hear real-world confirmation of the speed up — a lot of work went into that!
By coincidence, yesterday I found and fixed an inefficient design in the pull code; the fix speeds up local-to-local replication a lot and should improve networked replication too. So stay tuned for even more speed
I am working on an app with 70-75.000 documents and I just tested it on Android.
It took for ages to sync - and it even broke down after 16 minutes having synced 47-48.000 documents. I print the number of docs completed (arg.Status.Progress.Completed) in the OnReplicationEvent when the activity is “Busy”. For Android I can see that this is called for each and every document - whereas for iOS I only see “bulks” of data.
Is that something I can somehow adjust/tune?
I tested on a device (the emulator was way slower) running Android 8.1. The Sync.Gateway is 2.1.1 and the server is Community Edition 6.0.0.
Can you profile the app to get an idea of what’s taking so much time? Replicator performance should be pretty consistent across platforms since most of the work is done in C++ code. (Aside from differences in hardware capabilities, of course.)
