Expiry, Purge, Shared bucket access and _delete

Mobile Sync Gateway
1

Hi All,

Before couchbase server 5.1 and sync gateway 1.5 it was not possible to update the couchbase server and get the changes on sync gateway.

In sync gateway 2.0 Expiry function was introduced when the expiry function was used with cbs 4.6.X it used to purge the data and the mobiles never used to get the notification about the purging which was perfect for our use case. As the data which we were purging was of no use when it was purge. And not change will happen in that data.

But after using sync gateway 2.0 and CBS 5.X the expiry function soft deletes the data i.e. _deleted is marked true. This happens when shared_bucket_access is marked true. Which I understand is required for XATTR and need to be done when one of the sync gateway is importing documents using import_docs=“continuous”.

This data travels to local devices which will now need compaction. The Mobile is using CBL 1.4.1 in android which doesn’t support auto compaction.

If we do not compact on local what will happen? Because when the deleted is marked true it will travel to the device before device can purge the data. What is the best strategy for compaction in local db.

What will happen if we do not use shared_bucket_access property so that it doesn’t soft deletes the data and only purge it.

We want to have a system where we want to purge data using expiry function and also want that the changes done on couchbase server directly using N1QL reaches sync gateway and to mobile device.

Please note that we have around 97000 documents to work with. and in every one week 50% will be purged and new documents will be inserted.

CC:- @househippo

Regards
Pankaj Sharma

2

Hi @pankaj.sharma,

You’re correct - with shared bucket access enabled, expiry of the document triggers creation of a mobile tombstone - it’s equivalent to any other type of delete performed directly against the bucket.

Purging the documents through Sync Gateway’s REST API instead of using expiry would preserve the previous behaviour. However, purging of documents from only one part of the stack (i.e. SG/CBS but not Couchbase Lite) is more commonly used for removal of obsolete documents, and not as part of a high-frequency workflow, as it can lead to unexpected behaviour (such as documents being resurrected) when they aren’t removed from the entire system.

When shared bucket access was not enabled and you were purging ~48K documents per week, what was your strategy for cleaning up versions of those documents that had already been replicated to Couchbase Lite?

1 Like
3

The app was also purging the documents from couch base lite as soon as the app starts. So the documents were getting purged from entire system. And even if the documents come back previous expiry function implementation will purge them with having older expiry time on the server.