Performance of Conditional Delete at high volume vs Flush

Couchbase Server
1

I have a Couchbase bucket that receives approximately 31mm documents / day. It is flushed 3 times / day.

The problem is that I have a cron that runs every 5 minutes and queries couchbase - I am at risk for losing up to 5 minutes of data during the flush.

Is there a way to put a condition on FLUSH, ie “only flush items with c_time < 30 minutes”?

Alternatively, if I replaced “flush” with a conditional bulk delete, how will performance compare over such a large number of documents?

2

Hi @jbackes,

This seems like a good place to use Couchbase’s built-in TTL (time-to-live) functionality: https://developer.couchbase.com/documentation/server/5.5/architecture/core-data-access-bucket-expiration.html

You can set a TTL on documents or on an entire bucket.