I am using sync gateway v1.1.0. I want to set an expiry time for my document, but I don’t see any way to specify expiry time through the sync gateway rest api in the documentation. How do I set a document expiry time when storing a document using the sync gateway rest api? E.g. POST http://localhost:4985/db/
Sync Gateway doesn’t aloow setting expiry time for regular documents. It’s not just a matter of exposing the underlying Couchbase Server support for expiry - Sync Gateway needs to treat deletes as tombstoned revisions, to ensure that deletes can be replicated to clients.
So how does sync gateway handle documents that expire? I have documents that only have a limited lifetime and since a ton of them are going to be created they have to be regularly deleted/expire.
Does this mean sync gateway has no expiry mechanism? Is this a feature planned for any future release? For now I could bypass sync gateway when creating documents that need to expire but it would be a really useful feature in a future release.
That’s correct - Sync Gateway doesn’t have an expiry mechanism. This is by design - to ensure that notification of deleted documents is replicated Sync Gateway and clients - even clients connecting after an extended period of time - deletes need to be handled as tombstones, not purge/expiry.
We’ve had discussions about adding purge functionality on the Sync Gateway side, but this wouldn’t be recommended for short-lived document management (because of the replication implications).
However, I’d like to better understand your use case, to understand why you’d like to permanently delete documents, as opposed to just tombstoning. If you can share more info on the issues you think expiry would address, it would be very helpful.
I don’t know what you mean by tombstones, sorry. But I wanted expiry because we are going to be doing notifications for users in our online portal based on events happening. A user will only see notifications for the past week, and if they don’t explicitly delete them those notifications will just sit on the server forever. I would like to be able to set an expiry date on them of a week so I can make sure we don’t end up with a ton of these documents. For now we are not syncing them to the app so they will just fill up server space, but in the future if we want to display them on the app then we will have serious issues because the app will try to sync a ton of documents.
Tombstoning is the canonical way to delete a document with Couchbase. It is a special type of revision that simply has a _deleted flag present (and most likely nothing else). The reason for this is because normally there is no way to distinguish between a document being deleted and a document sitting there with nothing happening to it. If it were to simply vanish from the database, it would not be picked up in the changes feed because the changes feed is forward looking only. Any new information on the document needs to be in a revision or it will potentially be missed by a client. This includes deletions.
That being said, you may be able to write a backend service that sits next to sync gateway that can perform this kind of expiration via the proper deletion mechanism (The most robust, if not performant way to do it would be a garbage collection of sorts which periodically scans the documents for their expiration dates and creates a delete revision for them if they are past due)