Question about "empty" documents (tombstones?)

Mobile Couchbase Lite
1

I have an app where I disable sync (by using a push filter that checks a field value on the document) for some types of documents recorded live - until the user finishes the recording. If the user cancels the whole recording then I delete the docs and that’s it.

Now, I have discovered that I have some “empty” documents in the database - and I just want to make sure that I’m not creating issues for myself (the solution is not in live production yet - with the app - the core web app is).

The meta data for a document could look like this: { "meta": { "id": "FishingTrip:4fe6b8e7faae456ab71bd4c0b73e8b93", "rev": "45-15ea0e7b241700000000000000000000", "expiration": 0, "flags": 0, "type": "json" }, "xattrs": { "_sync": { "rev": "2-99923b648e6b4b3d758da4f90dfbc6cffa5e9ad5", "flags": 1, "sequence": 144058, "recent_sequences": [ 144058 ], "history": { "revs": [ "2-99923b648e6b4b3d758da4f90dfbc6cffa5e9ad5", "1-29892e66d6e2bbcae6739d63e4ec251013225ce5" ], "parents": [ 1, -1 ], "deleted": [ 0 ], "channels": [ null, null ] }, "cas": "0x000017247b0eea15", "value_crc32c": "0x297bd0aa", "tombstoned_at": 1579090541, "time_saved": "2020-01-15T13:15:41.393098142+01:00" } } }
Is this just a tombstone waiting to be purged?

I have seen different values for the deleted field (0,1,2).

I guess I could avoid it if I do a db.purge(...) instead of a delete? The user could have created the data on the server as well - and if he deletes from the app it should obviously be a db.delete(...) instead. Not sure if I can query the document directly to know if it is an “existing” document or I’ll have to control that using my own “flag”?

2

That JSON you showed is from Sync Gateway. I’m not familiar with current SG metadata, but that does look like a tombstone / deleted doc.

If the doc you want to get rid of isn’t on the server, then it’s better to purge it in CBL. But there’s not really a built-in way to tell. Maybe you could set a special property in the document when you create it locally, and make sure to remove that property before you allow the doc to be pushed to the server?

3

I’ve tried to change my code to ad a specific field to disable sync. I’ve then added a PushFilter that tests for this field and returns false if it is set.

        config.PushFilter = (document, flags) =>
            {
                if (document.GetBoolean(nameof(BaseDoc._syncDisabled).ToLower()))
                {
                    Logger.Debug($"Pushfilter: Replication disabled for {document.Id}. Skip...");
                    return false;
                }
                return true;
            };
        replicator = new Replicator(config);

That works fine.

The “funny” thing is that having the filter (with the log message) I sometimes see documents being handled by the filter with doc.ids that I have not created:

[DbDataStore] Pushfilter: Replication disabled for -rsHpcIF6gU24C4vVzgpq-Q. Skip...
[DbDataStore] Pushfilter: Replication disabled for -KlX_bkGWzky5WuE9aFX2yQ. Skip...

What could these be?? Have I created a “clash” to a reserved name with the field name? - I’m using the field name: _syncdisabled…

When later I delete a document it seems that the replicator tries to find it - but can’t, so I get an error in the Application Output console. The complete scenario for creating a doc and adding a photo (adds a photo and an image doc with sync. disabled) and then cancel creation of the document with the photo (i.e. purge photo and image docs) look like this:

[DataService] Saving doc: 8cf29c438c824a08a1b5fd3439f45913 (type: Photo)
Thread started:  #68
[DbDataStore] Pushfilter: Replication disabled for Photo:Image:8cf29c438c824a08a1b5fd3439f45913. Skip...
[DbDataStore] Pushfilter: Replication disabled for Photo:8cf29c438c824a08a1b5fd3439f45913. Skip...
Thread finished: <Thread Pool> #4
  :
  :
[DataService] removeFromDb: Purge Photo:Image:8cf29c438c824a08a1b5fd3439f45913...
[DataService] removeFromDb: Purge Photo:8cf29c438c824a08a1b5fd3439f45913...
Thread started:  #29
Thread started:  #34
Thread finished:  #34
Thread started:  #35
[DbDataStore] Push replication finished sending 2 documents
[DbDataStore]   Doc.id: Photo:Image:8cf29c438c824a08a1b5fd3439f45913, Error: Couchbase.Lite.CouchbaseLiteException: CouchbaseLiteException (LiteCoreDomain / 7): not found.
[DbDataStore]   Doc.id: Photo:8cf29c438c824a08a1b5fd3439f45913, Error: Couchbase.Lite.CouchbaseLiteException: CouchbaseLiteException (LiteCoreDomain / 7): not found.

Not sure if that matters? No document are sent to the server. So I guess that it is Ok? On the other hand I don’t really like error messages and would prefer a “clean” console :innocent:

4

The key _syncDisabled isn’t reserved, nor used internally for anything. Maybe set a breakpoint on that Logger.Debug call and look at the mysterious documents? That might give you a clue what they are.

The not-found error could be a race condition, where the replicator sees a change to the document, but by the time it tries to send the revision the doc has been purged. Is the app making a change to the document right before it purges it?

5

Good that it is not a reserved name :slight_smile:

I don’t make any changes to the document prior to purging it. A short example is:

var imgId = BaseDoc.GetID(typeof(Photo).Name, typeof(Image).Name, BaseDoc.GetKeyFromID(id));
var imgDoc = dataStore.Db.GetDocument(imgId);
removeFromDb(imgDoc);

and the removeFromDb() function looks like this:

private void removeFromDb(Document doc)
{
    if (!(doc is null))
    {
        // If never synced - then just purge to avoid creating tombstones....
        if (doc.GetBoolean(nameof(BaseDoc._syncDisabled).ToLower()))
        {
            Logger.Debug($"removeFromDb: Purge {doc.Id}");
            dataStore.Db.Purge(doc);
        }
        else
        {
            Logger.Debug($"removeFromDb: Delete {doc.Id}");
            dataStore.Db.Delete(doc);
        }
    }
}

So the only thing I do is check for the “sync disabled” field - and then purge it :slight_smile: