I’m running a few load tests on my IOS app. I’ve noticed that when I generate a bunch of documents in quick succession that occasionally a few documents aren’t synced to the client.
I can see the missing docs on the server (via the sync-gateway rest API), but when I query for the document id via the rest CBL API its not there. The missing docs appear if I wipe the local database and re-sync. And if I run multiple clients I can usually see the docs appears on the other clients, though they sometimes have docs missing also.
Do you see any logs either on the server side or on the client side regarding failed transfers? It’s possible due to network conditions or other factors that certain documents failed to transfer. If that happens, the replication will continue without them and swing around to try them again after a certain interval (60 seconds I believe) so this may be the behavior you are seeing.
After waiting for a few minutes I do not see the missing documents getting synced to my client. I check by querying via the CBL rest api. It’s pretty easy to reproduce, just add a few hundred documents in quick succession make it happen fairly reliably.
I don’t see any obvious errors in my client. I didn’t see any errors on the sync-gateway logs either. The output after grepping for the document id (bdb60f3b1f25b6cbc65dd510bc19a1dd) is below.
Its worth mentioning that there’s a webhook in play here as well. The document is created and then immediately a web hook will modify it, so there’s a modification very shorty after the initial document is saved. There are no other updates to the document.
I’ve tried re-starting the sync process but that doesn’t help. The only way I’ve found to get the missing docs is to wipe the client database and restart the sync process from scratch.
11:53:22.384974 2016-03-17T11:53:22.384Z CRUD+: Invoking sync on doc "bdb60f3b1f25b6cbc65dd510bc19a1dd" rev 1-1f344d80da1b4b4d143bf2cc5ed906fa
11:53:22.385590 2016-03-17T11:53:22.385Z CRUD: Doc "bdb60f3b1f25b6cbc65dd510bc19a1dd" in channels "{ch_277dca61-f32a-4eed-8693-43170bac8cc8}"
11:53:22.386041 2016-03-17T11:53:22.386Z CRUD: Stored doc "bdb60f3b1f25b6cbc65dd510bc19a1dd" / "1-1f344d80da1b4b4d143bf2cc5ed906fa"
11:53:22.386244 2016-03-17T11:53:22.386Z Events+: Event queue worker sending event Document change event for doc id: bdb60f3b1f25b6cbc65dd510bc19a1dd to: Webhook handler [http://macbookpro.local:8080/datr/webhook]
11:53:22.391074 2016-03-17T11:53:22.391Z HTTP: #10326: GET /sync_gateway/bdb60f3b1f25b6cbc65dd510bc19a1dd (as user5)
11:53:22.395936 2016-03-17T11:53:22.395Z HTTP: #10328: GET /sync_gateway/bdb60f3b1f25b6cbc65dd510bc19a1dd (as admin)
11:53:22.400649 2016-03-17T11:53:22.400Z HTTP: #10329: PUT /sync_gateway/bdb60f3b1f25b6cbc65dd510bc19a1dd (as admin)
11:53:22.401139 2016-03-17T11:53:22.401Z CRUD+: Invoking sync on doc "bdb60f3b1f25b6cbc65dd510bc19a1dd" rev 2-2bad40b3e2f7ab58ef7271d60a789e0e
11:53:22.401711 2016-03-17T11:53:22.401Z CRUD+: Saving old revision "bdb60f3b1f25b6cbc65dd510bc19a1dd" / "1-1f344d80da1b4b4d143bf2cc5ed906fa" (402 bytes)
11:53:22.402032 2016-03-17T11:53:22.402Z CRUD+: Backed up obsolete rev "bdb60f3b1f25b6cbc65dd510bc19a1dd"/"1-1f344d80da1b4b4d143bf2cc5ed906fa"
11:53:22.402917 2016-03-17T11:53:22.402Z CRUD: Stored doc "bdb60f3b1f25b6cbc65dd510bc19a1dd" / "2-2bad40b3e2f7ab58ef7271d60a789e0e"
11:53:22.403183 2016-03-17T11:53:22.403Z Events+: Event queue worker sending event Document change event for doc id: bdb60f3b1f25b6cbc65dd510bc19a1dd to: Webhook handler [http://macbookpro.local:8080/datr/webhook]
11:53:23.154894 2016-03-17T11:53:23.154Z Cache: Received deduplicated #49973 for ("bdb60f3b1f25b6cbc65dd510bc19a1dd" / "2-2bad40b3e2f7ab58ef7271d60a789e0e")
11:53:23.155396 2016-03-17T11:53:23.155Z Cache: Received #49975 after 751ms ("bdb60f3b1f25b6cbc65dd510bc19a1dd" / "2-2bad40b3e2f7ab58ef7271d60a789e0e")