Active_only=true on non-empty client database

Mobile
1

Hi there,

We’ve developed an Android Cordova based application that is already in Production. We are using CouchbaseLite/SyncGateway architecture for our client/server synchronization.
We are using channel removal as our retention policy in order to keep client data usage as minimum as possible. That means that after some time we move documents out of the user’s channel transforming them into tombstones.

After several days, tombstones volume may increase the client data usage, so what we do is to provide users a mechanism to manually clear the database.

As per Couchbase lite java client source, the queries to SyncGateway include a parameter active_only which will determine if documents that were deleted (or removed from user’s channel) need to be included in the results. Also, we could see in the code that this flag (active_only) will be always false unless the client database is empty (lastSequence==null). This means that, after clearing the client database, tombstones won’t be replicated. That’s why we clear the client’s data eventually.

Now, we have detected that from time to time, document channel removals are not being replicated to the client. We weren’t sure where to look, but after digging into nginx logs we found out that our clients are doing these kinds of queries to SyncGateway

/syncgw/_changes?feed=longpoll&heartbeat=30000&style=all_docs&since=464984&active_only=true

As far as we know, this means that we are asking for active documents only, even when the client database is already initialized and it has data. If this is true, clients will not be notified upon a document channel removal.

I’ll now describe an example:

  • Users have access to one channel: ‘alive’. Documents are created and associated with this specific channel.

  • We’ve created a channel ‘avoid’ to which users have no access. Documents that are moved to this channel will be either marked as removed in the clients (generating a tombstone) or won’t be replicated at all to the client. This will depend on if the client database has already data or not.

  • So, User1 creates doc1 which is replicated to SyncGateway (_id: doc1; _rev: 1).

  • After some time, we move doc1 from ‘alive’ into ‘avoid’ channel creating a new revision of the document (_id: doc1; _rev: 2).

  • On a normal use case, when User1 pulls new changes from the server, all the changes from the last sequence number should be returned, including nonactive documents (active_only=false). In the case of doc1 we would receive:
    {
    “seq”: 7936,
    “id”: “doc1”,
    “removed”: [
    “alive”
    ],
    “changes”: [
    {
    “rev”: “2”
    }
    ]
    }

  • But if what we’ve seen in nginx logs is true, and the active_only=true s set. Clients won’t be notified of the document channel removal, and then the client will assume that doc1 hasn’t been updated at all, leaving the rev1 version of the document intact.

Is there any kind of edge case condition that may be causing this to happen? Or is this expected to happen? If so, how clients should be notified upon a document
t channel removal?

Thanks in advance.

CouchbaseLite version: 1.4.1
SyncGateway version: 1.4.1