Confusion about different fields in sync gateway metadata

Mobile
1

A document that I have is not deleting properly, because it doesn’t seem to have the oldDoc parameter of the sync function populated. There are a few confusing fields in the document’s metadata:
Why does it have _rev and _new_rev? Which one should be used when trying to create a new revision?
Why does it have a deleted object:
“deleted”: [
11
]
, does this mean revision 11 was deleted but then the document kept getting updated somehow?
Also could you please let me know if any of these new fields would cause the sync function to not be able to handle the document properly?

"_sync": {
       "rev": "52-40de381f0728d7da816fd47a8efa3b29",
       "new_rev": "54-cf51da69a070fc7692e917d7e431833f",
       "flags": 20,
       "sequence": 12100496,
       "recent_sequences": [
           12033013,
           12033072,
           12033109,
           12033132,
           12033131,
           12033135,
           12033159,
           12033163,
           12033166,
           12033169,
           12033172,
           12100496
       ],
       "history": {
           "revs": [
               "33-b4567da3cb9512fafff9f78dbbee0919",
               "41-0c8096c25f47c5bd17eded05263b418c",
               "37-cb358fc6fe4b52b0772aa670e6836ad1",
               "8-8c7b879cab423b13de5c878a1dabcaa4",
               "30-099a3b21d137cbfe6bff599be1007ac0",
               "53-37ea4bce6451131110e4f7a8a652d6ce",
               "12-9e05570b6e652e55947a120eb9deb368",
               "25-18a09c069b7135355497d4039f6cdd44",
               "15-ac60345b15294b6fab6d043e5727c00b",
               "24-422f54be54ba7bfc02c6c3744eb32198",
               "16-7b43c36e4c5554bd3d25d6454a369c8b",
               "54-cf51da69a070fc7692e917d7e431833f",
               "26-10e8cea4e66aff28d0753521928a16fb",
               "11-ae913ed9108821aa7957d0e7f3590b4c",
               "4-2225dd0c1004ee5e667ad5c902dd0067",
               "31-d5bc9cd63854fa692e1a0ee25cf4b65a",
               "23-c44ba21fffc6c6cb83fd6e64761e5ab0",
               "21-ffff8f4930847b4f9c3876671340026a",
               "3-ce053212560fc4a75d813f09585f60f6",
               "50-37979ab21b9c18a1e6fcce33df21f800",
               "44-b1ac94102ac0db8fa85830217607a34e",
               "20-5ebc6a866dbd0a8197d43bc16d294f31",
               "28-b256f030e7346fd9dd58e189b09f8936",
               "52-ff270aa13580ad419585e230e137d87d",
               "1-ed2eee057ca6ff7ad03773cfc4509cfe",
               "6-b956a6ef41ff1cc05b44f2b869b57291",
               "42-6705026795f9c0dd23523151a48f9ac6",
               "39-f982ef7913cb0f21d2a54200351259e9",
               "38-39d0975710ba2d80524512c1fb26b20e",
               "18-e659739678416226412497ebf912b4da",
               "10-94d8d994ae05841968ee571a62a19590",
               "49-92c1430679b9f612e4b17ed7e8c4e004",
               "47-60609c5e5a63ef1210b72ca78ac20615",
               "14-a474889d9925934f3b9259b4e6c66422",
               "7-87372ad034ae1bce4f6fd4f9676ccede",
               "52-40de381f0728d7da816fd47a8efa3b29",
               "35-3b96001dbb72d5ae648c752fa4cc150d",
               "29-68928bfaaa9116b8372fdfffd5fca2a6",
               "27-d20bf2c417d28f337ca19336d9ba77f8",
               "22-b49653c88341fc7581f7d50f070f29a4",
               "9-36c55aa6c2d81087a0186b4cf1809b74",
               "43-b5e4fced2b89ca71826a53a9a29f736d",
               "36-8f7ac33eb334a99532958f5e56ee3d9a",
               "48-5c864c2f69334694e3183f77c1682086",
               "2-ed0ff340b803e6e12d31f1cca963f38d",
               "17-7c80b360087a997a46e4cbfc1c247ea4",
               "46-3af18634c2eeb1746cc3436eeaf2908f",
               "45-97a95d81f2638e6cb74d4386ae37c288",
               "13-9432c9b98cb1e534509e57190420cf54",
               "5-61701d3a4b1b3b7fb2f7ea4044b1a000",
               "40-f701d5cbcaa3839b9f3604c43c15527f",
               "19-36f46ffaeeec1750df6e7bc5bd016265",
               "32-0a08641b61c084b707e976157cda9139",
               "51-d3b9549fefc1ff08f0daf6814e8138c6",
               "34-9ab382be76ad519ae49fe7982f1a722b"
           ],
           "parents": [
               52,
               50,
               42,
               34,
               37,
               23,
               13,
               9,
               33,
               16,
               8,
               5,
               7,
               30,
               18,
               4,
               39,
               21,
               44,
               31,
               41,
               51,
               38,
               53,
               -1,
               49,
               1,
               28,
               2,
               45,
               40,
               43,
               46,
               48,
               25,
               53,
               54,
               22,
               12,
               17,
               3,
               26,
               36,
               32,
               24,
               10,
               47,
               20,
               6,
               14,
               27,
               29,
               15,
               19,
               0
           ],
           "deleted": [
               11
           ],
           "bodymap": {
               "11": "{\"_deleted\":true}"
           },
           "channels": [
               [
                   "channel1",
                   "channel2"
               ],
			   ... <removed a ton of entries that were all the same or just null>
			   
           ]
       },
       "channels": {
           "channel1": null,
           "channel2": null
       },
       "time_saved": "2017-03-17T10:33:52.761270806Z"
   }
2

For the first question,

_rev is the current winning revision

_new_rev is the newest revision, but not necessarily the winning revision

It looks like the document has conflicting revisions.

A _deleted doc still exits as a tombstone revision and can be updated with properties again by a client.

I’m still looking at why this would result in the oldDoc not getting populated in the sync function

3

Does that mean that if I want to update the document I would use _rev, not _new_rev? So that I don’t get a document conflict error.

So a document can be deleted, but if there was a conflict during the deletion and the non-deleted version won you would get a tombstone revision? So the document is not technically deleted, just one of the conflict leaves was deleted?

4

It’s really best not to look into the internal metadata in the bucket, not unless you want to also become familiar with the Sync Gateway source code to understand what the metadata means. You can use the REST API to get information about the document, like its conflicts and history, in a documented format.

So a document can be deleted, but if there was a conflict during the deletion and the non-deleted version won you would get a tombstone revision?

Deletion always creates a tombstone revision, as a child of the revision you deleted. But if the document was in conflict, then there are other current revisions, so one of those will become the current revision, and the document as a whole isn’t deleted. To delete a document that’s in conflict you’d have to delete each of the conflicting revisions.

5

I think it’s cool to have an understanding of what’s in that document, albeit with the caveat that there will be nuances and it’s not really designed to be end-user consumable. Basically don’t blame me if you get led astray here ;).

As Andy said, _rev is the current winning revision and _new_rev is the newest revision, but not necessarily the winning revision. This raises the question of why 54-cf51... isn’t winning against 52-40de....

If we look at history, we’ve got the list of revisions, revs, which is reasonably self explanatory. parents is a bit more awkward to read, but essemtially maps each revision to it’s parent based on the position in each array, so the parent of the Nth revision in revs (rev[N]) is given by revs[parent[N]]. E.g.:

  • 33-b456... is the first entry in revs
  • The first entry in parents is 52
  • The 53rd entry in revs (revs[52] because it’s 0 indexed) is 32-0a08..., which is therefore the parent of 33-b456....

deleted follows the same pattern, so we can see that revs[11] has been deleted (or more accurately is a tombstone). Lo and behold, if we look that up it’s 54-cf51.... Because it’s deleted, it can’t be the winning revision and so the only other revision that doesn’t have any children (i.e. a leaf node) wins - 52-40de....

So what actually happened here? You had two conflicting 52-... revisions: 52-40de... and 52-ff27.... When this happens, one of them is deemed the “winner”. All this really means is that if you ask for the document and don’t specify a revision, you get the winner. It’s chosen deterministically, but it’s arbitrary (just based on the lexicographical sort). So the app asks for the document, gets 52-ff27..., updates it to create 53-37ea... then adds that back to Sync Gateway. A bit later, the app goes to delete the document and deletes 53-37ea..., creating the tombstone 54-cf51.... Now, because 52-40de... still exists, it becomes the winner by default.

As for the oldDoc issue, I can offer some wild speculation - but I think that’s probably tangential to the main issue anyway: you’ve got a conflicting revision that it sounds like you weren’t expecting. This bit of the Conflicts FAQ sounds particularly relevant, but the whole thing is worth a read, as is Resolving Conflicts if you’re interested in resolving it on the Sync Gateway side. Probably worth noting that there’s not always a silver bullet here - it’s worth considering the data model and how you want conflicts to play out before implementing anything.

6

You can get the same info more easily by issuing a GET of the document with ?open_revs=all, which will include the revIDs of all current revisions, indicating whether there’s a conflict; or ?revs=true which will include the history of the revision being displayed.

7

Thanks for the in-depth explanation! Our application uses sync gateway and couchbase lite iOS. I think the issues we are experiencing are because of some code we added to our application to resolve conflicts in one of our document types (the document metadata I posted above was of that type). We’ve been having issues with this conflict resolution code though and even though it is basically copy-pasted from the couchbase lite guide page on conflict resolution (https://developer.couchbase.com/documentation/mobile/1.4/guides/couchbase-lite/native-api/document/index.html) it seems to have a lot of unintended side effects, such as deleting a revision creating a tombstone revision rather than just resolving the conflict, or breaking the _attachments object so there is a revpos field but no digest. Is there any other more in-depth documentation on how to resolve conflicts in couchbase lite?

2 Likes