Couchbase Eventing Questions

Couchbase Server
1

Couchbase Eventing Questions

For example, source collection A has a data TTL of 10 days

I’m going to designate a data TTL for target collection A as 30 days

Use XDCR to move the values from source collection A to target collection A

I would like to designate the TTL of target collection A as 30 days.

I understand that XDCR can remove TTL but not change.

ex:
source A, Metadata

{
  "meta": {
    "id": "k03",
    "rev": "1-177151cce693000064af7af300000000",
    "expiration": 1689826632,
    "flags": 0,
    "type": "json"
  },
  "xattrs": {}
}

source B, Metadata

{
  "meta": {
    "id": "k03",
    "rev": "1-177151cce693000064af7af300000000",
    "expiration": 1691900232,
    "flags": 0,
    "type": "json"
  },
  "xattrs": {}
}
2

Hi @bellpumpkin - the easiest way here might be just to specify the TTL at the collection level: Manage Scopes and Collections | Couchbase Docs (step 5)

If you need more complex handling of it, then here’s an example of using Eventing to set/change the expiration: Function: Document Controlled Expiry | Couchbase Docs

The key part of this code is using a N1QL function to update the document’s TTL:

var stmt = "UPDATE `source` USE KEYS \""+key+
                       "\" SET meta().expiration = " + Math.floor(ttlMs/1000);
            N1QL(stmt);

Here’s a broader example of using Eventing to archive data from one collection to another: Document Archival | Couchbase Docs

[EDIT] - A couple different options to choose from:
1 - XDCR from collection A to collection B, with a 30-day TTL set on collection B: this is the simplest and has the least moving parts, but is also the least flexible if you need more complex logic or need to change in the future
2 - An eventing function listening to collection A and writing to collection B with a 30-day TTL (no XDCR): this puts the onus on the eventing function to move the data but gives you the most flexibility over logic and updating the behaviour.
3 - XDCR from collection A to collection B, with an eventing function listening to collection B to set the TTL: this is the most complex in terms of moving parts, but gives you the most flexibility in terms of combining different functionality.

cc @jon.strabala

3

@bellpumpkin Parry gave some solid information, I do want to point out a few additional items.

In Eventing using an advanced bucket accessor to set the TTL can be 5X faster.

Two variants of this function are available - a 6.6 version that relies on SQL++ and a 6.6.1+/7.0.0+ version (this Function) that directly sets the expiration. You can completely avoid N1QL(…​) and use couchbase.replace(bucket_binding, meta, doc) as the advancedDocControlledSelfExpiry variant is much faster.

For the later the key part of the code is:

        var	ttlMs = docTimeStampMs + keepDocForMs;
        var expiryDate = new Date(ttlMs);
        var res = couchbase.replace(src_col,{"id":meta.id,"expiry_date":expiryDate},doc);

In future Couchbase releases starting with 7.6 we will be implemented a ‘touch’ function which can operate than using the advanced bucket accessor method (2X faster on 5k byte documents and 50X faster on very large documents 1M byte documents).

        var	ttlMs = docTimeStampMs + keepDocForMs;
        var expiryDate = new Date(ttlMs);
        var res = couchbase.touch(src_col, {"id": meta.id, "expiry_date": expiryDate});

Best

Jon Strabala
Principal Product Manager - Server‌

4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.