Realm’s benchmark results were for Couchbase Lite 1.0 or 1.1, which was admittedly much slower, and the implementation of their benchmark for Couchbase Lite was extremely inefficient since they didn’t make use of indexing (map/reduce views.)
Glad to hear we should be seeing faster performance, currently our benchmarks are at 1.2k inserts per second on an iPad Air. So Realm’s benchmarks of 1.7k inserts per second seems to be faster than what we are seeing on 2.6. We are looking at the insert performance under the inserts tab on that webpage.
There’s a lot of code in the Shared directory of your repo. Could you describe what the benchmark does at a high level?
Restructured the Shared directory to make it clearer what the benchmark does.
The entire benchmark code can be found here, lines 14 - 160.
It has 6 steps called out with the following comments.
// STEP 1: DELETE DB
// STEP 2: NEW DB
// STEP 3: ENSURE INDEXES
// STEP 4: CREATE MODELS
// STEP 5: CREATE MUTABLE DOCUMENTS
// STEP 6: INSERT INTO COUCHBASE
I can see it has custom data model classes which it then encodes to CBL Documents; what does it do after saving them?
We are focusing on just the saving/inserting performance at the moment.
So the benchmark ends after saving to Couchbase, nothing is done after saving the documents.
I’m happy to run a benchmark and share the results. Could you please make a release apk available. I lack the knowledge of setting up a C# developer IDE.
I adjusted every other setting on the “Android Build” page.
Nothing else had any further effect on the benchmark timings.
Just to point out - “Enable AOT” is experimental (non-production) and Visual Studio Enterprise only option. It also did not change the performance of the benchmark timings.
Interesting, thanks for running that! I analyzed each APK after changing each setting and none of the options stripped the MainActivity method names. The methods look like mono generated code so it might be optimized but still modified by Mono? Not sure I know enough to try anything more on it. If ya have any ideas please let me know.
Hi @benjamin_glatzeder and @jens; just wanted to check in and see if the benchmarking app has helped to identify any root causes with insert performance?
I’m unable to give any advice on how to build a release apk with Visual Studio. I grabbed the apk version 1.1 and have these results:
Delete DB: 0 ms
Create New DB: 151 ms
Create 11 Indexes: 332 ms
Create 11000 Models: 128 ms
Create 11000 Documents: 781 ms
Batch Save 11000 Documents: 8583 ms
So that’s a big improvement over my previously posted benchmarks. I’m also unable to help you profile method calls. I might be able to help you port the benchmark app from .Net to Java. If there are differences in the benchmark results then at least you would know that it makes sense to profile the .Net code first and afterwards the Couchbase Lite Core module.
