Lcb_store is faster than lcb_store3

C SDK
1

Hello,

I’ve created a test program that bulk imports data in batches. I’m using the C SDK (version 2.60) with Couchbase Server (4.1.1-5914 CE).

I’ve noticed that, unless I’m doing something wrong, using lcb_store is faster than using lcb_store3! To import 1 million documents, using lcb_store takes 50 seconds whilst lcb_store3 takes 80 seconds.

Using lcb_store:
I’m putting commands into an array (size=1000), then call lcb_store and lcb_wait whenever the array becomes full.

Using lcb_store3:
I’m creating a new command and immediately call lcb_store3, then call lcb_wait3 after every 1000 commands.

lcb_store3 has replaced lcb_store, but has there been a performance regression? Or am I doing something wrong? lcb_store will presumably be removed from the API in a future release, because it’s in api-legacy.h?

Thanks in advance,

Graham

2

That’s impossible, because lcb_store in all versions since 2.4 just calls lcb_store3: https://github.com/couchbase/libcouchbase/blob/master/src/operations/store.c#L327

You shouldn’t be calling lcb_wait after each call of lcb_store3. You should only be calling it once after you’ve scheduled all the commands needed with lcb_store3, pretty much exactly how you see in the implementation.

3

That’s good to know, thanks. I’ve got to track down what I’m doing wrong then!

You shouldn’t be calling lcb_wait after each call of lcb_store3

I’m not:

Using lcb_store3:
I’m creating a new command and immediately call lcb_store3, then call lcb_wait3 after every 1000 commands.

So I’m only calling lcb_wait3 once for every 1000 times I call lcb_store3. I could increase the batch size, but then the memory usage of my program increases. I’ll have a play around with different settings.

4

Thanks mnunberg, the link you posted pointed me in the right direction.

I’ve increased the batch size (from 1,000 to 100,000) and enclosed each batch within lcb_sched_enter and lcb_sched_leave, and now lcb_store3 is just as fast as using lcb_store.