Does eventing happen one document at a time?

Eventing
1

We have an eventing script that adds the current epoch time in milliseconds to each document as it’s moved from one bucket to another. I’m wondering if this happens one document at a time, or will couchbase perform this on multiple documents at the same time?

The reason I ask is because I need to know if it’s possible for 2 documents to have the same epoch time appended from our eventing script.

2

Hi @Jayfry

If you are running 6.6.0 or earlier you most likely are using N1QL in your Eventing function to do this.

but if you are using 6.6.1+ things are much cleaner and 4X to 5X faster

I’m wondering if this happens one document at a time, or will couchbase perform this on multiple documents at the same time?

Eventing is triggered one (1) document at a time on the given mutation. Eventing is fast but once you try to do a ‘bulk’ update you if you don’t have access to the KEYS via the mutated document (or another document via a direct KV fetch from the Eventing function) you might need to have an index and utilize N1QL and in this later case things may be slower.

Stated another way if you can lookup your KEYS without N1QL you can update or delete documents directly via a bucket binding aliases, i.e. an exposed JavaScript MAPs to the Data Service in the Eventing function this is very fast and in this case you can avoid N1QL and also do not need and index.

3

To add to what Jon indicated - if you need a unique key, you should consider using an atomic counter, please see Function: Advanced INCREMENT operation | Couchbase Docs

Best Regards,
Siri

4

Johnathon [@Jayfry]

I reread this post and I have to apologize I didn’t answer it fully. I assumed you were talking about expiry and in early version adjusting expiry was a N1QL operation, however you might just have been adding a timestamp (as in Date.now()) as a new field as follows:

function OnUpdate(doc, meta) {
    // 1. get integer timestamp in millis
    var ts = Date.now();
    // 2. add or overwrite the property
    doc.cur_ts = ts;
    // 3. update peer document in a different bucket
    log(ts,doc,meta.id);
    dst_bkt[meta.id] = doc;
}

If you were asking about the above this this warrants a different answer…

Yes the same millisecond is very possible, a single Eventing thread does process one document at a time, however by default in 6.5 and 6.6 you have three workers and each work has two threads. This means you have six (6) execution units to process the DCP stream (DCP is database change protocol).

So Eventing execution units run in parallel

Couchbase is very fast if we are just talking about KV (look at the bottom of this public page http://showfast.sc.couchbase.com/#/timeline/Linux/eventing/scaling/Function)

Here we see Couchbase could perform your function (add a timestamp on a bucket move to a new bucket) just over 1M ops/sec on a high end cluster with more workers in the Function’s settings. Thats 1,000 docs per millisecond and you only have millisecond granularity on JavaScript Dates.

So yes it is not only completely possible but probable especially if you deploy from Everything.

Sriri pointed out you could use atomic counters to guarantee uniqueness, alternatively you could add Math.random() to the timestamp. Perhaps something like

var ts_dot_random = Date.now() + "." + Math.random();