Is it better to use touch() or remove() ?
Remove() gets the Json documents
touch() renews the time to live.Even if we delay it to a longer time,Couchbase will have to check it at intervals.
touch also cause fragmentation
The application is such that it has continuous writes to CouchBase.For such an application would you suggest touch() or remove().
The two are quite different.
touch (and getAndTouch) prevent a document from being automatically deleted because of its expiry, by renewing the TTL of the document. Also, the documents that expire are not necessarily removed immediately when they expire, but rather by a regular expiry pager scheduled process. See the docs for more info on expiry.
remove directly removes a document from the database, but that is an active operation (you need to call it and know which key to delete).
touch also cause fragmentation
Both cause “fragmentation” in the sense that Couchbase will keep a “tombstone” metadata to mark the deletion of a key, so that it can replicate the deletion to all the nodes. The storage format is append only, so it also keeps the deleted document on disk. But once compaction kicks in, these deleted documents will be removed and after a while these tombstones will be removed as well.
For such an application would you suggest touch() or remove().
Overall, I’d say if you have a good way of identify which documents need to be deleted, go for remove.
If there is a large number of deletions to be performed, and a short number of documents whose life needs to be “extended”, use TTL and touch on these extended life documents.
But it highly depends on your application needs and workload, so don’t hesitate to measure and plan for a switch in strategy later on.
The applications needs to write/add an equal number of records and delete equal number of records.That is if we insert 500 record, we delete the old 500 records . Then we have another set that comes after 30 to 60 seconds, 500 records to write in and 500 records to delete . Do you consider this as “large” number of deletions and reccomend using TTL?
We dont want to extend to the life of the document. Instead we use touch to sets it time to live for only 1 sec as in:
bucket.async().touch(keyToBeDeleted, 1) . In essence we are expiring it.
good way of identify which documents need to be deleted:
We know the key(id of the document) that needs to be deleted. I guess we need to know that key in both cases, remove as well as touch.
I’d definitely use remove in this configuration. If I understand correctly, what you do is use touch as a sort of delayed remove using expiry, which I don’t think brings much benefit with this number of documents…
Unless you’d want to delete the documents of the previous batch after a fixed amount of time, no matter what? (in which case you could set an expiry when you insert the documents)
It doesn’t seem to be the case though, due to this 30 to 60 second variation.
Also, if the records could share the same key then a replace or upsert would be even more straightforward!