Expiring documents in sync gateway with cb 1.3.0-274

Mobile
1

I’m trying to use the new _exp flag in 1.3. I’ve followed the docs and can see my document being created, and then after the expiry it appears to be deleted. However, I can’t see the delete in my CBL 1.3.1 clients. I also can see any evidence of the delete in the sync-gateway logs. Do / should the deleted documents be sent to clients via the _changes feed because its not there as a delete, it just appears as if it still exists:

seq: 215056,
id: "521f4647-f41e-4a22-91fb-3b98d7c1f61a",
changes: [
  {
    rev: "1-b34299f338806ae5d48ecd9b5e440857"
  }
]

Using sync-gateway 1.3.0-274

2

Expiration doesn’t sync; it’s a local attribute of a document. This should be covered in the docs on document expiration (attn @jamiltz)

3

Ok… What’s a local attribute? Where can I read about them?

What’s expiration in the sync gateway for if it doesn’t sync to clients? What sort of use case did you have in mind?

4

I just mean that its effect is local. Setting an expiration time on either SG or CBL only affects the document in that database, not anywhere else.

Expiration in Sync Gateway is to control storage usage by Sync Gateway. It’s pretty common to have data piling up over time that’s no longer of interest.

If you want the documents to expire on the clients, then add some code to the client to set an expiration time for the documents as they arrive.

It’s not necessarily the case that the client and server want the same lifetime of a document. Sometimes it should be expired on the server but kept around by clients, sometimes vice versa. For example a chat server may want to expire documents after a day or so, but a user would expect that once on their client the messages would stay around indefinitely.

5

If you want the documents to expire on the clients, then add some code to the client to set an expiration time for the documents as they arrive.

Is the _exp flag supported in CBL as well then? If a client receives a document via replication, sets the _exp flag and saves it, what happens? Do they create a new revision of the document, and does it get replicated to the sync gateway? How does the client know that the delete has happened?

Are there any plans to support expiring documents that are replicated from the sync-gateway?

6

Is the _exp flag supported in CBL as well then?

Yes _exp is supported on the CBL native SDK Couchbase Capella for Mobile Developers.

Not sure if it’s available on the REST API.

The native SDK docs mention that the _exp is not replicated, I’ve added the same paragraph in the Sync Gateway _exp request body property description to make it more explicit.

If a client receives a document via replication, sets the _exp flag and saves it, what happens?

It will be purged from the database where the TTL was set.

Do they create a new revision of the document, and does it get replicated to the sync gateway?

No. The functionality is local only. I believe there are plans to look into replicated TTLs in the future.

How does the client know that the delete has happened?

The doc is purged so it’s not in the database anymore but you know when it gets removed by setting the desired TTL.

I don’t know a if a database or document change event is fired for purge operations.

James

7

Expiration isn’t implemented in the CBL REST API yet, sorry. When it’s implemented it’ll use the same API as SG.

We will probably implement propagation of expiration in a future release. If/when we do this, it’ll probably work by making the expiration time a property that’s visible to the replicator, so when a peer receives a revision it gets the expiration time along with it and can set it locally.

How does the client know that the delete has happened?

It won’t. By definition a purge removes all trace of the document, so afterwards there won’t be any evidence that it happened. The doc has to be replicated (with its expiration time set) before it expires.

8

Thanks for the info, very helpful.