Replication - 409 Checkpoint Document update conflict

Felix_123 · April 26, 2021, 3:39pm

Hi,

we have an Android app which uses the couchbaseLite for storing data and the Sync-Gateway for replication.
Sometimes after around 5 minutes of inactivity, we are getting a setCheckpoint error which leads to that documents that were created in the app are not synced into the Couchbase.

2021-04-26T14:57:49.110002011Z 2021-04-26T14:57:49.108Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#15! (flags=  110000, length=26)
2021-04-26T14:57:49.110059845Z 2021-04-26T14:57:49.108Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#19! (flags=  110000, length=26)
2021-04-26T14:57:49.110067810Z 2021-04-26T14:57:49.108Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#15!
2021-04-26T14:57:49.110072046Z 2021-04-26T14:57:49.108Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#19!
2021-04-26T14:57:49.110075888Z 2021-04-26T14:57:49.108Z [INF] SyncMsg: c:[54a9c5d7] #15: Type:proposeChanges #Changes: 0
2021-04-26T14:57:49.110079697Z 2021-04-26T14:57:49.108Z [INF] SyncMsg: c:[4d7f685d] #19: Type:proposeChanges #Changes: 0
2021-04-26T14:57:49.110083396Z 2021-04-26T14:57:49.108Z [DBG] SyncMsg+: c:[54a9c5d7] #15: Type:proposeChanges   --> OK Time:20.926µs
2021-04-26T14:57:49.110087619Z 2021-04-26T14:57:49.108Z [DBG] SyncMsg+: c:[4d7f685d] #19: Type:proposeChanges   --> OK Time:21.427µs
2021-04-26T14:57:53.673130868Z 2021-04-26T14:57:53.671Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#20! (flags=  110000, length=26)
2021-04-26T14:57:53.673181143Z 2021-04-26T14:57:53.671Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#20!
2021-04-26T14:57:53.673188803Z 2021-04-26T14:57:53.671Z [INF] SyncMsg: c:[4d7f685d] #20: Type:proposeChanges #Changes: 0
2021-04-26T14:57:53.673193211Z 2021-04-26T14:57:53.671Z [DBG] SyncMsg+: c:[4d7f685d] #20: Type:proposeChanges   --> OK Time:47.457µs
2021-04-26T14:57:53.680269101Z 2021-04-26T14:57:53.678Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#16! (flags=  110000, length=26)
2021-04-26T14:57:53.680317499Z 2021-04-26T14:57:53.678Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#16!
2021-04-26T14:57:53.680323952Z 2021-04-26T14:57:53.678Z [INF] SyncMsg: c:[54a9c5d7] #16: Type:proposeChanges #Changes: 0
2021-04-26T14:57:53.680327947Z 2021-04-26T14:57:53.678Z [DBG] SyncMsg+: c:[54a9c5d7] #16: Type:proposeChanges   --> OK Time:24.446µs
2021-04-26T14:57:54.114884799Z 2021-04-26T14:57:54.111Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#21 (flags=       0, length=117)
2021-04-26T14:57:54.114929723Z 2021-04-26T14:57:54.112Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#21
2021-04-26T14:57:54.114937150Z 2021-04-26T14:57:54.112Z [INF] SyncMsg: c:[4d7f685d] #21: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-11
2021-04-26T14:57:54.114941622Z 2021-04-26T14:57:54.113Z [INF] SyncMsg: c:[4d7f685d] #21: Type:setCheckpoint   --> 409 Document update conflict Time:1.019436ms
2021-04-26T14:57:54.114945685Z 2021-04-26T14:57:54.113Z [DBG] WS+: c:[4d7f685d] Push ERR#21
2021-04-26T14:57:54.114949444Z 2021-04-26T14:57:54.113Z [DBG] WS+: c:[4d7f685d] Sending frame: ERR#21 (flags=      10, size=   58)
2021-04-26T14:57:54.121514448Z 2021-04-26T14:57:54.118Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#17 (flags=       0, length=117)
2021-04-26T14:57:54.121581121Z 2021-04-26T14:57:54.118Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#17
2021-04-26T14:57:54.121595145Z 2021-04-26T14:57:54.118Z [INF] SyncMsg: c:[54a9c5d7] #17: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-12
2021-04-26T14:57:54.121603304Z 2021-04-26T14:57:54.119Z [DBG] SyncMsg+: c:[54a9c5d7] #17: Type:setCheckpoint   --> OK Time:963.142µs
2021-04-26T14:57:54.121610755Z 2021-04-26T14:57:54.119Z [DBG] WS+: c:[54a9c5d7] Push RPY#17
2021-04-26T14:57:54.121624033Z 2021-04-26T14:57:54.119Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#17 (flags=       1, size=   10)
2021-04-26T14:57:54.134236988Z 2021-04-26T14:57:54.131Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#22 (flags=       0, length=62)
2021-04-26T14:57:54.134291568Z 2021-04-26T14:57:54.131Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#22
2021-04-26T14:57:54.134302888Z 2021-04-26T14:57:54.131Z [INF] SyncMsg: c:[4d7f685d] #22: Type:getCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c=
2021-04-26T14:57:54.134309863Z 2021-04-26T14:57:54.132Z [DBG] SyncMsg+: c:[4d7f685d] #22: Type:getCheckpoint   --> OK Time:638.509µs
2021-04-26T14:57:54.134317324Z 2021-04-26T14:57:54.132Z [DBG] WS+: c:[4d7f685d] Push RPY#22~
2021-04-26T14:57:54.134323749Z 2021-04-26T14:57:54.132Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#22~ (flags=    1001, size=   86)
2021-04-26T14:57:54.163543609Z 2021-04-26T14:57:54.161Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#23 (flags=       0, length=117)
2021-04-26T14:57:54.163604120Z 2021-04-26T14:57:54.161Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#23
2021-04-26T14:57:54.163613049Z 2021-04-26T14:57:54.161Z [INF] SyncMsg: c:[4d7f685d] #23: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-13
2021-04-26T14:57:54.167109983Z 2021-04-26T14:57:54.164Z [DBG] SyncMsg+: c:[4d7f685d] #23: Type:setCheckpoint   --> OK Time:2.171265ms
2021-04-26T14:57:54.167143860Z 2021-04-26T14:57:54.164Z [DBG] WS+: c:[4d7f685d] Push RPY#23
2021-04-26T14:57:54.167151945Z 2021-04-26T14:57:54.164Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#23 (flags=       1, size=   10)

Does this have anything to do with the sgreplicate_websocket_heartbeat_secs config which is obsolete I think or with replicator in the app?

Sync-Gateway version: sync-gateway:2.7.3-community
Flutter libary: couchbase_lite 2.7.1+7 | Flutter Package

Thanks for your attention. I’m looking forward to your reply.

eunice.huang · April 26, 2021, 6:28pm

I don’t think it relates to sgreplicate_websocket_heartbeat_secs config. the log shows there is a conflict while trying to push a doc from android to sync gateway, updating the document on sync gateway failed.

in your sync gateway config, is conflict allowed or not allowed?
beside couchbase lite <=> sync gateway replication, do you also have inter-sync gateway replication ON? (sg-replicate) if so, while there is a conflict on document update, who will win the conflict? or is there any custom conflict function defined?
if your android couchbase lite defined remote win in its replicator, it’s expected behavior while pushing a conflict document. sync gateway (the remote) rejects the document, and wins in the sync.

priya.rajagopal · April 27, 2021, 2:48am

From the logs, I don’t see the 5 min of inactivity before the error. How did you determine that from logs?
It seems like there is a conflict when the client tries to update the checkpoint (which at high level is a record of the last sequenceID replicated by the replicator ). It would be useful to get the preceding logs that shows the setCheckpoint message processing prior to the one that failed

Felix_123 · April 27, 2021, 7:11am

“allow_conflicts”: true

Inter-Sync Gateway Replication is not enabled because we are using the community edition and this is an enterprise feature.

Ok. I don´t really know because I am working in Backend but I will try to figuring that out. But I think you can´t set this option in the flutter libary.
Currently this is the only two options that are configured in the app replicator:

config.replicatorType = ReplicatorType.pushAndPull;
config.continuous = true;

Sync-Gateway Config:

    "databases": {
        "db": {
            "bucket": "xyz",
            "server": "http://couchbase:8091",
            "import_docs": true,
            "enable_shared_bucket_access": true,
            "num_index_replicas": 0,
            "allow_conflicts": true,
            "revs_limit" : 50,
            "username": "{{ SG_USER }}",
            "password": "{{ SG_PASSWORD }}",
            "users": {
                "admin": {
                    "password": "xyz",
                    "admin_channels": ["*"]
                }
            },
            "cache": {
                "channel_cache": {
                    "max_wait_pending": 3000,
                    "max_num_pending": 500,
                    "max_wait_skipped": 300000,
                    "enable_star_channel": false,
                    "max_length": 250,
                    "min_length": 10,
                    "expiry_seconds": 300
                },
                "rev_cache": {
                    "size": 1000
                }
            },
            "sync": `function (doc, oldDoc) { if(doc._deleted) { return; } channel(doc.channels); }`
        }
    }

Felix_123 · April 27, 2021, 7:15am

I hope that this helps

  2021-04-26T14:56:56.384948202Z 2021-04-26T14:56:56.383Z [INF] WS: c:[54a9c5d7] Start BLIP/Websocket handler
2021-04-26T14:56:56.384953339Z 2021-04-26T14:56:56.383Z [DBG] WS+: c:[54a9c5d7] Sender starting
2021-04-26T14:56:56.422545017Z 2021-04-26T14:56:56.420Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#1 (flags=       0, length=62)
2021-04-26T14:56:56.422573402Z 2021-04-26T14:56:56.420Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#1
2021-04-26T14:56:56.422578884Z 2021-04-26T14:56:56.420Z [INF] SyncMsg: c:[54a9c5d7] #1: Type:getCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c=
2021-04-26T14:56:56.422582902Z 2021-04-26T14:56:56.421Z [DBG] SyncMsg+: c:[54a9c5d7] #1: Type:getCheckpoint   --> OK Time:714.841µs
2021-04-26T14:56:56.422659469Z 2021-04-26T14:56:56.421Z [DBG] WS+: c:[54a9c5d7] Push RPY#1~
2021-04-26T14:56:56.422666309Z 2021-04-26T14:56:56.421Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#1~ (flags=    1001, size=   85)
2021-04-26T14:56:56.457581602Z 2021-04-26T14:56:56.448Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#2 (flags=       0, length=58)
2021-04-26T14:56:56.457638967Z 2021-04-26T14:56:56.448Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#2
2021-04-26T14:56:56.457647291Z 2021-04-26T14:56:56.449Z [INF] SyncMsg: c:[54a9c5d7] #2: Type:subChanges Since:15081 Continuous:true
2021-04-26T14:56:56.457653741Z 2021-04-26T14:56:56.449Z [DBG] WS+: c:[54a9c5d7] Push RPY#2
2021-04-26T14:56:56.457659937Z 2021-04-26T14:56:56.449Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#2 (flags=       1, size=    1)
2021-04-26T14:56:56.457690493Z 2021-04-26T14:56:56.450Z [INF] Sync: c:[54a9c5d7] Sending changes since 15081
2021-04-26T14:56:56.457698658Z 2021-04-26T14:56:56.450Z [DBG] Changes+: c:[54a9c5d7] Int sequence multi changes feed...
2021-04-26T14:56:56.457706086Z 2021-04-26T14:56:56.450Z [INF] Changes: c:[54a9c5d7] MultiChangesFeed(channels: {*}, options: {Since: 15081, Limit: 0, Conflicts: false, IncludeDocs: false, Wait: true, Continuous: true, HeartbeatMs: 0, TimeoutMs: 0, ActiveOnly: false}) ...   (to some_name)
2021-04-26T14:56:56.457713672Z 2021-04-26T14:56:56.451Z [DBG] Changes+: c:[54a9c5d7] MultiChangesFeed: channels expand to "!:1,some_name:3688" ...   (to some_name)
2021-04-26T14:56:56.457720062Z 2021-04-26T14:56:56.451Z [DBG] Cache+: c:[54a9c5d7] GetCachedChanges("!", 15081) --> nothing cached
2021-04-26T14:56:56.458127161Z 2021-04-26T14:56:56.451Z [DBG] Changes+: c:[54a9c5d7] [changesFeed] Found 0 changes for channel "!"
2021-04-26T14:56:56.458151608Z 2021-04-26T14:56:56.451Z [DBG] Cache+: c:[54a9c5d7] GetCachedChanges("some_name", 15081) --> nothing cached
2021-04-26T14:56:56.458160786Z 2021-04-26T14:56:56.451Z [DBG] Changes+: c:[54a9c5d7] [changesFeed] Found 0 changes for channel "some_name"
2021-04-26T14:56:56.458166492Z 2021-04-26T14:56:56.451Z [DBG] Changes+: c:[54a9c5d7] MultiChangesFeed waiting...   (to some_name)
2021-04-26T14:56:56.458213068Z 2021-04-26T14:56:56.451Z [DBG] Changes+: No new changes to send to change listener.  Waiting for "a_bucket"'s count to pass 0
2021-04-26T14:56:56.458225388Z 2021-04-26T14:56:56.451Z [DBG] Sync+: c:[54a9c5d7]     Sending 0 changes
2021-04-26T14:56:56.458231781Z 2021-04-26T14:56:56.451Z [DBG] WSFrame+: c:[54a9c5d7] Queued MSG#1~
2021-04-26T14:56:56.458236444Z 2021-04-26T14:56:56.451Z [DBG] WS+: c:[54a9c5d7] Push MSG#1~
2021-04-26T14:56:56.458240145Z 2021-04-26T14:56:56.451Z [INF] Sync: c:[54a9c5d7] Sent all changes to client
2021-04-26T14:56:56.458243730Z 2021-04-26T14:56:56.457Z [DBG] WS+: c:[54a9c5d7] Sending frame: MSG#1~ (flags=  101000, size=   51)
2021-04-26T14:56:56.488169605Z 2021-04-26T14:56:56.486Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#3! (flags=  110000, length=26)
2021-04-26T14:56:56.488272729Z 2021-04-26T14:56:56.486Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#3!
2021-04-26T14:56:56.488286983Z 2021-04-26T14:56:56.486Z [INF] SyncMsg: c:[54a9c5d7] #3: Type:proposeChanges #Changes: 0
2021-04-26T14:56:56.488293708Z 2021-04-26T14:56:56.486Z [DBG] SyncMsg+: c:[54a9c5d7] #3: Type:proposeChanges   --> OK Time:21.487µs
2021-04-26T14:56:58.273979217Z 2021-04-26T14:56:58.272Z [INF] HTTP:  #064: GET /a_bucket/_all_docs (as ADMIN)
2021-04-26T14:56:59.141302746Z 2021-04-26T14:56:59.135Z [INF] Query: N1QL Query("allDocs") took 861.783444ms
2021-04-26T14:56:59.141355172Z 2021-04-26T14:56:59.139Z [INF] HTTP+: #064:     --> 200 OK  (866.7 ms)
2021-04-26T14:57:00.647812566Z 2021-04-26T14:57:00.646Z [INF] HTTP:  #065: GET /a_bucket/_design/compact/_view/view_name?stale=false (as ADMIN)
2021-04-26T14:57:00.647872862Z 2021-04-26T14:57:00.646Z [INF] HTTP: JSON view "compact"/"view_name" - opts map[stale:false]
2021-04-26T14:57:00.777962622Z 2021-04-26T14:57:00.776Z [INF] HTTP+: #065:     --> 200   (130.4 ms)
2021-04-26T14:57:01.631407851Z 2021-04-26T14:57:01.629Z [INF] HTTP:  #066: GET /a_bucket/_design/compact/_view/get_stock_transfers?stale=false (as ADMIN)
2021-04-26T14:57:01.631445518Z 2021-04-26T14:57:01.629Z [INF] HTTP: JSON view "compact"/"get_stock_transfers" - opts map[stale:false]
2021-04-26T14:57:01.643126530Z 2021-04-26T14:57:01.641Z [INF] HTTP+: #066:     --> 200   (12.2 ms)
2021-04-26T14:57:06.055353052Z 2021-04-26T14:57:06.054Z [DBG] DCP+: c:a_bucket-SGI cbdatasource: server: 172.27.0.3:11210, uprOpenName: SGI-v-2.7-commit--uuid-68d3003b-a69e-11eb-9012-0242ac1b0004, worker, looping beg, vbucketState: "running" (has 1024 vbuckets), 0-1023
2021-04-26T14:57:06.100660573Z 2021-04-26T14:57:06.099Z [DBG] DCP+: c:a_bucket-SG cbdatasource: server: 172.27.0.3:11210, uprOpenName: SG-v-2.7-commit--uuid-68dbc7ed-a69e-11eb-9012-0242ac1b0004, worker, looping beg, vbucketState: "running" (has 1024 vbuckets), 0-1023
2021-04-26T14:57:14.200470450Z 2021-04-26T14:57:14.198Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#10! (flags=  110000, length=26)
2021-04-26T14:57:14.200528667Z 2021-04-26T14:57:14.198Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#4! (flags=  110000, length=26)
2021-04-26T14:57:14.200538168Z 2021-04-26T14:57:14.199Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#4!
2021-04-26T14:57:14.200542547Z 2021-04-26T14:57:14.199Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#10!
2021-04-26T14:57:14.200546622Z 2021-04-26T14:57:14.199Z [INF] SyncMsg: c:[54a9c5d7] #4: Type:proposeChanges #Changes: 0
2021-04-26T14:57:14.200551033Z 2021-04-26T14:57:14.199Z [INF] SyncMsg: c:[4d7f685d] #10: Type:proposeChanges #Changes: 0
2021-04-26T14:57:14.200555304Z 2021-04-26T14:57:14.199Z [DBG] SyncMsg+: c:[4d7f685d] #10: Type:proposeChanges   --> OK Time:82.32µs
2021-04-26T14:57:14.200568157Z 2021-04-26T14:57:14.199Z [DBG] SyncMsg+: c:[54a9c5d7] #4: Type:proposeChanges   --> OK Time:84.771µs
2021-04-26T14:57:16.144589396Z 2021-04-26T14:57:16.143Z [INF] HTTP:  #067: GET /a_bucket/_design/compact/_view/view_name?stale=false (as ADMIN)
2021-04-26T14:57:16.144633548Z 2021-04-26T14:57:16.143Z [INF] HTTP: JSON view "compact"/"view_name" - opts map[stale:false]
2021-04-26T14:57:16.160722785Z 2021-04-26T14:57:16.159Z [INF] HTTP+: #067:     --> 200   (16.4 ms)
2021-04-26T14:57:19.191101942Z 2021-04-26T14:57:19.188Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#11 (flags=       0, length=116)
2021-04-26T14:57:19.191141244Z 2021-04-26T14:57:19.188Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#5 (flags=       0, length=116)
2021-04-26T14:57:19.191149675Z 2021-04-26T14:57:19.188Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#11
2021-04-26T14:57:19.191154051Z 2021-04-26T14:57:19.188Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#5
2021-04-26T14:57:19.191158270Z 2021-04-26T14:57:19.188Z [INF] SyncMsg: c:[4d7f685d] #11: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-6
2021-04-26T14:57:19.191162641Z 2021-04-26T14:57:19.188Z [INF] SyncMsg: c:[54a9c5d7] #5: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-6
2021-04-26T14:57:19.191166633Z 2021-04-26T14:57:19.189Z [DBG] SyncMsg+: c:[4d7f685d] #11: Type:setCheckpoint   --> OK Time:816.612µs
2021-04-26T14:57:19.191171024Z 2021-04-26T14:57:19.189Z [DBG] WS+: c:[4d7f685d] Push RPY#11
2021-04-26T14:57:19.191174610Z 2021-04-26T14:57:19.189Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#11 (flags=       1, size=    9)
2021-04-26T14:57:19.191178358Z 2021-04-26T14:57:19.190Z [INF] SyncMsg: c:[54a9c5d7] #5: Type:setCheckpoint   --> 409 Document update conflict Time:1.947194ms
2021-04-26T14:57:19.192225835Z 2021-04-26T14:57:19.190Z [DBG] WS+: c:[54a9c5d7] Push ERR#5
2021-04-26T14:57:19.192258566Z 2021-04-26T14:57:19.190Z [DBG] WS+: c:[54a9c5d7] Sending frame: ERR#5 (flags=      10, size=   58)
2021-04-26T14:57:19.222111852Z 2021-04-26T14:57:19.219Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#6 (flags=       0, length=62)
2021-04-26T14:57:19.222156521Z 2021-04-26T14:57:19.219Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#6
2021-04-26T14:57:19.222167512Z 2021-04-26T14:57:19.219Z [INF] SyncMsg: c:[54a9c5d7] #6: Type:getCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c=
2021-04-26T14:57:19.222174497Z 2021-04-26T14:57:19.220Z [DBG] SyncMsg+: c:[54a9c5d7] #6: Type:getCheckpoint   --> OK Time:696.478µs
2021-04-26T14:57:19.222181970Z 2021-04-26T14:57:19.220Z [DBG] WS+: c:[54a9c5d7] Push RPY#6~
2021-04-26T14:57:19.222188479Z 2021-04-26T14:57:19.220Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#6~ (flags=    1001, size=   85)
2021-04-26T14:57:19.281643945Z 2021-04-26T14:57:19.278Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#7 (flags=       0, length=116)
2021-04-26T14:57:19.281693064Z 2021-04-26T14:57:19.279Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#7
2021-04-26T14:57:19.281706145Z 2021-04-26T14:57:19.279Z [INF] SyncMsg: c:[54a9c5d7] #7: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-7
2021-04-26T14:57:19.281715519Z 2021-04-26T14:57:19.280Z [DBG] SyncMsg+: c:[54a9c5d7] #7: Type:setCheckpoint   --> OK Time:1.030938ms
2021-04-26T14:57:19.281736572Z 2021-04-26T14:57:19.280Z [DBG] WS+: c:[54a9c5d7] Push RPY#7
2021-04-26T14:57:19.281745075Z 2021-04-26T14:57:19.280Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#7 (flags=       1, size=    9)
2021-04-26T14:57:27.493538370Z 2021-04-26T14:57:27.491Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#12! (flags=  110000, length=26)
2021-04-26T14:57:27.494816900Z 2021-04-26T14:57:27.491Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#12!
2021-04-26T14:57:27.494832190Z 2021-04-26T14:57:27.491Z [INF] SyncMsg: c:[4d7f685d] #12: Type:proposeChanges #Changes: 0
2021-04-26T14:57:27.494838767Z 2021-04-26T14:57:27.491Z [DBG] SyncMsg+: c:[4d7f685d] #12: Type:proposeChanges   --> OK Time:74.851µs
2021-04-26T14:57:27.500329252Z 2021-04-26T14:57:27.498Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#8! (flags=  110000, length=26)
2021-04-26T14:57:27.500379949Z 2021-04-26T14:57:27.499Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#8!
2021-04-26T14:57:27.500387782Z 2021-04-26T14:57:27.499Z [INF] SyncMsg: c:[54a9c5d7] #8: Type:proposeChanges #Changes: 0
2021-04-26T14:57:27.500410941Z 2021-04-26T14:57:27.499Z [DBG] SyncMsg+: c:[54a9c5d7] #8: Type:proposeChanges   --> OK Time:26.425µs
2021-04-26T14:57:29.145309422Z 2021-04-26T14:57:29.143Z [INF] HTTP:  #068: GET /a_bucket/_all_docs (as ADMIN)
2021-04-26T14:57:29.884989829Z 2021-04-26T14:57:29.883Z [INF] Query: N1QL Query("allDocs") took 739.09455ms
2021-04-26T14:57:29.889182006Z 2021-04-26T14:57:29.887Z [INF] HTTP+: #068:     --> 200 OK  (744.0 ms)
2021-04-26T14:57:31.462079031Z 2021-04-26T14:57:31.460Z [INF] HTTP:  #069: GET /a_bucket/_design/compact/_view/view_name?stale=false (as ADMIN)
2021-04-26T14:57:31.462129200Z 2021-04-26T14:57:31.460Z [INF] HTTP: JSON view "compact"/"view_name" - opts map[stale:false]
2021-04-26T14:57:31.610999031Z 2021-04-26T14:57:31.609Z [INF] HTTP+: #069:     --> 200   (149.7 ms)
2021-04-26T14:57:32.503850417Z 2021-04-26T14:57:32.501Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#13 (flags=       0, length=116)
2021-04-26T14:57:32.503910036Z 2021-04-26T14:57:32.502Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#13
2021-04-26T14:57:32.503922206Z 2021-04-26T14:57:32.502Z [INF] SyncMsg: c:[4d7f685d] #13: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-7
2021-04-26T14:57:32.503929977Z 2021-04-26T14:57:32.502Z [INF] SyncMsg: c:[4d7f685d] #13: Type:setCheckpoint   --> 409 Document update conflict Time:482.109µs
2021-04-26T14:57:32.503939341Z 2021-04-26T14:57:32.502Z [DBG] WS+: c:[4d7f685d] Push ERR#13
2021-04-26T14:57:32.503946478Z 2021-04-26T14:57:32.502Z [DBG] WS+: c:[4d7f685d] Sending frame: ERR#13 (flags=      10, size=   58)
2021-04-26T14:57:32.511741489Z 2021-04-26T14:57:32.508Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#9 (flags=       0, length=116)
2021-04-26T14:57:32.511781175Z 2021-04-26T14:57:32.508Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#9
2021-04-26T14:57:32.511787920Z 2021-04-26T14:57:32.508Z [INF] SyncMsg: c:[54a9c5d7] #9: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-8
2021-04-26T14:57:32.511792462Z 2021-04-26T14:57:32.509Z [DBG] SyncMsg+: c:[54a9c5d7] #9: Type:setCheckpoint   --> OK Time:803.631µs
2021-04-26T14:57:32.511797118Z 2021-04-26T14:57:32.509Z [DBG] WS+: c:[54a9c5d7] Push RPY#9
2021-04-26T14:57:32.511800993Z 2021-04-26T14:57:32.509Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#9 (flags=       1, size=    9)
2021-04-26T14:57:32.524475382Z 2021-04-26T14:57:32.521Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#14 (flags=       0, length=62)
2021-04-26T14:57:32.524509793Z 2021-04-26T14:57:32.521Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#14
2021-04-26T14:57:32.524517583Z 2021-04-26T14:57:32.521Z [INF] SyncMsg: c:[4d7f685d] #14: Type:getCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c=
2021-04-26T14:57:32.525745550Z 2021-04-26T14:57:32.524Z [DBG] SyncMsg+: c:[4d7f685d] #14: Type:getCheckpoint   --> OK Time:2.719059ms
2021-04-26T14:57:32.525769767Z 2021-04-26T14:57:32.524Z [DBG] WS+: c:[4d7f685d] Push RPY#14~
2021-04-26T14:57:32.525778708Z 2021-04-26T14:57:32.524Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#14~ (flags=    1001, size=   85)
2021-04-26T14:57:32.563821454Z 2021-04-26T14:57:32.552Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#15 (flags=       0, length=116)
2021-04-26T14:57:32.563870926Z 2021-04-26T14:57:32.560Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#15
2021-04-26T14:57:32.563884582Z 2021-04-26T14:57:32.560Z [INF] SyncMsg: c:[4d7f685d] #15: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-9
2021-04-26T14:57:32.563893934Z 2021-04-26T14:57:32.561Z [DBG] SyncMsg+: c:[4d7f685d] #15: Type:setCheckpoint   --> OK Time:1.009677ms
2021-04-26T14:57:32.563901554Z 2021-04-26T14:57:32.561Z [DBG] WS+: c:[4d7f685d] Push RPY#15
2021-04-26T14:57:32.563909027Z 2021-04-26T14:57:32.561Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#15 (flags=       1, size=   10)
2021-04-26T14:57:35.313215309Z 2021-04-26T14:57:35.311Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#16! (flags=  110000, length=26)
2021-04-26T14:57:35.313284341Z 2021-04-26T14:57:35.311Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#16!
2021-04-26T14:57:35.313297936Z 2021-04-26T14:57:35.311Z [INF] SyncMsg: c:[4d7f685d] #16: Type:proposeChanges #Changes: 0
2021-04-26T14:57:35.313305714Z 2021-04-26T14:57:35.311Z [DBG] SyncMsg+: c:[4d7f685d] #16: Type:proposeChanges   --> OK Time:32.521µs
2021-04-26T14:57:35.319987035Z 2021-04-26T14:57:35.318Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#10! (flags=  110000, length=26)
2021-04-26T14:57:35.320017583Z 2021-04-26T14:57:35.318Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#10!
2021-04-26T14:57:35.320024198Z 2021-04-26T14:57:35.318Z [INF] SyncMsg: c:[54a9c5d7] #10: Type:proposeChanges #Changes: 0
2021-04-26T14:57:35.320028164Z 2021-04-26T14:57:35.318Z [DBG] SyncMsg+: c:[54a9c5d7] #10: Type:proposeChanges   --> OK Time:15.186µs
2021-04-26T14:57:36.056081390Z 2021-04-26T14:57:36.054Z [DBG] DCP+: c:a_bucket-SGI cbdatasource: server: 172.27.0.3:11210, uprOpenName: SGI-v-2.7-commit--uuid-68d3003b-a69e-11eb-9012-0242ac1b0004, worker, looping beg, vbucketState: "running" (has 1024 vbuckets), 0-1023
2021-04-26T14:57:36.101263968Z 2021-04-26T14:57:36.099Z [DBG] DCP+: c:a_bucket-SG cbdatasource: server: 172.27.0.3:11210, uprOpenName: SG-v-2.7-commit--uuid-68dbc7ed-a69e-11eb-9012-0242ac1b0004, worker, looping beg, vbucketState: "running" (has 1024 vbuckets), 0-1023
2021-04-26T14:57:39.943235728Z 2021-04-26T14:57:39.941Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#17! (flags=  110000, length=26)
2021-04-26T14:57:39.943309153Z 2021-04-26T14:57:39.941Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#17!
2021-04-26T14:57:39.943320188Z 2021-04-26T14:57:39.941Z [INF] SyncMsg: c:[4d7f685d] #17: Type:proposeChanges #Changes: 0
2021-04-26T14:57:39.943327924Z 2021-04-26T14:57:39.941Z [DBG] SyncMsg+: c:[4d7f685d] #17: Type:proposeChanges   --> OK Time:25.921µs
2021-04-26T14:57:39.950076736Z 2021-04-26T14:57:39.948Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#11! (flags=  110000, length=26)
2021-04-26T14:57:39.950144906Z 2021-04-26T14:57:39.948Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#11!
2021-04-26T14:57:39.950155138Z 2021-04-26T14:57:39.948Z [INF] SyncMsg: c:[54a9c5d7] #11: Type:proposeChanges #Changes: 0
2021-04-26T14:57:39.950162214Z 2021-04-26T14:57:39.948Z [DBG] SyncMsg+: c:[54a9c5d7] #11: Type:proposeChanges   --> OK Time:28.989µs
2021-04-26T14:57:40.311129621Z 2021-04-26T14:57:40.308Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#18 (flags=       0, length=117)
2021-04-26T14:57:40.311168843Z 2021-04-26T14:57:40.309Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#18
2021-04-26T14:57:40.311176060Z 2021-04-26T14:57:40.309Z [INF] SyncMsg: c:[4d7f685d] #18: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-10
2021-04-26T14:57:40.311180700Z 2021-04-26T14:57:40.309Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#12 (flags=       0, length=116)
2021-04-26T14:57:40.311184904Z 2021-04-26T14:57:40.309Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#12
2021-04-26T14:57:40.311188868Z 2021-04-26T14:57:40.309Z [INF] SyncMsg: c:[54a9c5d7] #12: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-9
2021-04-26T14:57:40.311200015Z 2021-04-26T14:57:40.310Z [INF] SyncMsg: c:[54a9c5d7] #12: Type:setCheckpoint   --> 409 Document update conflict Time:1.281138ms
2021-04-26T14:57:40.311204876Z 2021-04-26T14:57:40.310Z [DBG] WS+: c:[54a9c5d7] Push ERR#12
2021-04-26T14:57:40.311208537Z 2021-04-26T14:57:40.310Z [DBG] WS+: c:[54a9c5d7] Sending frame: ERR#12 (flags=      10, size=   58)
2021-04-26T14:57:40.311212334Z 2021-04-26T14:57:40.310Z [DBG] SyncMsg+: c:[4d7f685d] #18: Type:setCheckpoint   --> OK Time:1.70587ms
2021-04-26T14:57:40.312117395Z 2021-04-26T14:57:40.310Z [DBG] WS+: c:[4d7f685d] Push RPY#18
2021-04-26T14:57:40.312134976Z 2021-04-26T14:57:40.310Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#18 (flags=       1, size=   10)
2021-04-26T14:57:40.341087808Z 2021-04-26T14:57:40.338Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#13 (flags=       0, length=62)
2021-04-26T14:57:40.341135855Z 2021-04-26T14:57:40.338Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#13
2021-04-26T14:57:40.341145483Z 2021-04-26T14:57:40.338Z [INF] SyncMsg: c:[54a9c5d7] #13: Type:getCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c=
2021-04-26T14:57:40.341151923Z 2021-04-26T14:57:40.339Z [DBG] SyncMsg+: c:[54a9c5d7] #13: Type:getCheckpoint   --> OK Time:594.909µs
2021-04-26T14:57:40.341159041Z 2021-04-26T14:57:40.339Z [DBG] WS+: c:[54a9c5d7] Push RPY#13~
2021-04-26T14:57:40.341165084Z 2021-04-26T14:57:40.339Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#13~ (flags=    1001, size=   86)
2021-04-26T14:57:40.394300864Z 2021-04-26T14:57:40.391Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#14 (flags=       0, length=117)
2021-04-26T14:57:40.394343381Z 2021-04-26T14:57:40.391Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#14
2021-04-26T14:57:40.394353610Z 2021-04-26T14:57:40.391Z [INF] SyncMsg: c:[54a9c5d7] #14: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-11
2021-04-26T14:57:40.394361134Z 2021-04-26T14:57:40.392Z [DBG] SyncMsg+: c:[54a9c5d7] #14: Type:setCheckpoint   --> OK Time:921.22µs
2021-04-26T14:57:40.394368848Z 2021-04-26T14:57:40.392Z [DBG] WS+: c:[54a9c5d7] Push RPY#14
2021-04-26T14:57:40.394374472Z 2021-04-26T14:57:40.392Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#14 (flags=       1, size=   10)
2021-04-26T14:57:46.933565427Z 2021-04-26T14:57:46.931Z [INF] HTTP:  #070: GET /a_bucket/_design/compact/_view/view_name?stale=false (as ADMIN)
2021-04-26T14:57:46.933629593Z 2021-04-26T14:57:46.931Z [INF] HTTP: JSON view "compact"/"view_name" - opts map[stale:false]
2021-04-26T14:57:47.079105965Z 2021-04-26T14:57:47.077Z [INF] HTTP+: #070:     --> 200   (146.0 ms)
2021-04-26T14:57:49.110002011Z 2021-04-26T14:57:49.108Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#15! (flags=  110000, length=26)
2021-04-26T14:57:49.110059845Z 2021-04-26T14:57:49.108Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#19! (flags=  110000, length=26)
2021-04-26T14:57:49.110067810Z 2021-04-26T14:57:49.108Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#15!
2021-04-26T14:57:49.110072046Z 2021-04-26T14:57:49.108Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#19!
2021-04-26T14:57:49.110075888Z 2021-04-26T14:57:49.108Z [INF] SyncMsg: c:[54a9c5d7] #15: Type:proposeChanges #Changes: 0
2021-04-26T14:57:49.110079697Z 2021-04-26T14:57:49.108Z [INF] SyncMsg: c:[4d7f685d] #19: Type:proposeChanges #Changes: 0
2021-04-26T14:57:49.110083396Z 2021-04-26T14:57:49.108Z [DBG] SyncMsg+: c:[54a9c5d7] #15: Type:proposeChanges   --> OK Time:20.926µs
2021-04-26T14:57:49.110087619Z 2021-04-26T14:57:49.108Z [DBG] SyncMsg+: c:[4d7f685d] #19: Type:proposeChanges   --> OK Time:21.427µs
2021-04-26T14:57:53.673130868Z 2021-04-26T14:57:53.671Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#20! (flags=  110000, length=26)
2021-04-26T14:57:53.673181143Z 2021-04-26T14:57:53.671Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#20!
2021-04-26T14:57:53.673188803Z 2021-04-26T14:57:53.671Z [INF] SyncMsg: c:[4d7f685d] #20: Type:proposeChanges #Changes: 0
2021-04-26T14:57:53.673193211Z 2021-04-26T14:57:53.671Z [DBG] SyncMsg+: c:[4d7f685d] #20: Type:proposeChanges   --> OK Time:47.457µs
2021-04-26T14:57:53.680269101Z 2021-04-26T14:57:53.678Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#16! (flags=  110000, length=26)
2021-04-26T14:57:53.680317499Z 2021-04-26T14:57:53.678Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#16!
2021-04-26T14:57:53.680323952Z 2021-04-26T14:57:53.678Z [INF] SyncMsg: c:[54a9c5d7] #16: Type:proposeChanges #Changes: 0
2021-04-26T14:57:53.680327947Z 2021-04-26T14:57:53.678Z [DBG] SyncMsg+: c:[54a9c5d7] #16: Type:proposeChanges   --> OK Time:24.446µs
2021-04-26T14:57:54.114884799Z 2021-04-26T14:57:54.111Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#21 (flags=       0, length=117)
2021-04-26T14:57:54.114929723Z 2021-04-26T14:57:54.112Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#21
2021-04-26T14:57:54.114937150Z 2021-04-26T14:57:54.112Z [INF] SyncMsg: c:[4d7f685d] #21: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-11
2021-04-26T14:57:54.114941622Z 2021-04-26T14:57:54.113Z [INF] SyncMsg: c:[4d7f685d] #21: Type:setCheckpoint   --> 409 Document update conflict Time:1.019436ms
2021-04-26T14:57:54.114945685Z 2021-04-26T14:57:54.113Z [DBG] WS+: c:[4d7f685d] Push ERR#21
2021-04-26T14:57:54.114949444Z 2021-04-26T14:57:54.113Z [DBG] WS+: c:[4d7f685d] Sending frame: ERR#21 (flags=      10, size=   58)
2021-04-26T14:57:54.121514448Z 2021-04-26T14:57:54.118Z [DBG] WS+: c:[54a9c5d7] Received frame: MSG#17 (flags=       0, length=117)
2021-04-26T14:57:54.121581121Z 2021-04-26T14:57:54.118Z [DBG] WSFrame+: c:[54a9c5d7] Incoming BLIP Request: MSG#17
2021-04-26T14:57:54.121595145Z 2021-04-26T14:57:54.118Z [INF] SyncMsg: c:[54a9c5d7] #17: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-12
2021-04-26T14:57:54.121603304Z 2021-04-26T14:57:54.119Z [DBG] SyncMsg+: c:[54a9c5d7] #17: Type:setCheckpoint   --> OK Time:963.142µs
2021-04-26T14:57:54.121610755Z 2021-04-26T14:57:54.119Z [DBG] WS+: c:[54a9c5d7] Push RPY#17
2021-04-26T14:57:54.121624033Z 2021-04-26T14:57:54.119Z [DBG] WS+: c:[54a9c5d7] Sending frame: RPY#17 (flags=       1, size=   10)
2021-04-26T14:57:54.134236988Z 2021-04-26T14:57:54.131Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#22 (flags=       0, length=62)
2021-04-26T14:57:54.134291568Z 2021-04-26T14:57:54.131Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#22
2021-04-26T14:57:54.134302888Z 2021-04-26T14:57:54.131Z [INF] SyncMsg: c:[4d7f685d] #22: Type:getCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c=
2021-04-26T14:57:54.134309863Z 2021-04-26T14:57:54.132Z [DBG] SyncMsg+: c:[4d7f685d] #22: Type:getCheckpoint   --> OK Time:638.509µs
2021-04-26T14:57:54.134317324Z 2021-04-26T14:57:54.132Z [DBG] WS+: c:[4d7f685d] Push RPY#22~
2021-04-26T14:57:54.134323749Z 2021-04-26T14:57:54.132Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#22~ (flags=    1001, size=   86)
2021-04-26T14:57:54.163543609Z 2021-04-26T14:57:54.161Z [DBG] WS+: c:[4d7f685d] Received frame: MSG#23 (flags=       0, length=117)
2021-04-26T14:57:54.163604120Z 2021-04-26T14:57:54.161Z [DBG] WSFrame+: c:[4d7f685d] Incoming BLIP Request: MSG#23
2021-04-26T14:57:54.163613049Z 2021-04-26T14:57:54.161Z [INF] SyncMsg: c:[4d7f685d] #23: Type:setCheckpoint Client:cp-7f7EuLmIiavjRGv69foLf4xl03c= Rev:0-13
2021-04-26T14:57:54.167109983Z 2021-04-26T14:57:54.164Z [DBG] SyncMsg+: c:[4d7f685d] #23: Type:setCheckpoint   --> OK Time:2.171265ms
2021-04-26T14:57:54.167143860Z 2021-04-26T14:57:54.164Z [DBG] WS+: c:[4d7f685d] Push RPY#23
2021-04-26T14:57:54.167151945Z 2021-04-26T14:57:54.164Z [DBG] WS+: c:[4d7f685d] Sending frame: RPY#23 (flags=       1, size=   10)
2021-04-26T14:57:59.893418611Z 2021-04-26T14:57:59.892Z [INF] HTTP:  #071: GET /a_bucket/_all_docs (as ADMIN)

priya.rajagopal · April 29, 2021, 2:55am

From logs, I don’t see anything that indicates that there was 5 min of inactivity. Can you point me to how you concluded that.
In any case, the 409 conflict indicates that the client is attempting to set a checkpoint document that has the same revID as the checkpoint that is recorded on SGW for that client. I am not sure why you are seeing so many conflicts but I don’t see anything which indicates that the doc sync itself has failed. How are you concluding that sync didn’t get through ?.

Maybe @bbrks has some thoughts

You should also look at the stats

Also (unrelated to issue but an observation) I see calls to the all_docs endpoint and design views. Which client is making those REST calls or is it some user ? all_docs call is a very expensive operation and must be avoided.

bbrks · April 29, 2021, 12:52pm

I think the key part to focus on here are the checkpoint conflicts, and see if there are any problems after that has been resolved.

From the logs I see the checkpoint ID cp-7f7EuLmIiavjRGv69foLf4xl03c being shared between two clients ( c:[54a9c5d7] and c:[4d7f685d]). It’s expected that each client has totally unique checkpoint IDs, which are usually generated on database creation.

The only situations I can think of that this can happen is either:

A pre-built database was used without running it through the copy API first. This can result in two or more devices sharing the same checkpoint ID from the pre-built database.
See the docs here for the pre-built database/copy API example
A single client app is running two identical replicators, that are stomping on each other’s checkpoints.
Two devices generated the same checkpoint ID. The odds of this happening are pretty small, I don’t expect it’s this.

jda · May 3, 2021, 9:11am

“…running a prebuilt database through the copy API” sounds like something I should have done??? The link you mention doesn’t mention this - could you point at current info about the proper way to built the pre-built database?

Thanks!

priya.rajagopal · May 4, 2021, 3:23am

Doc on the best practices for creating and using prebuilt databases

jda · May 4, 2021, 6:17am

Thanks for that.

That is the way I do it (new db in app and sync all data down)… I was just confused by the “copy API” mentioning - but I guess that must be a synonym for one of the other methods.

priya.rajagopal · May 4, 2021, 1:42pm

the copy API is referring to the CBL side API to copy the database within your app. It is important that you use the copy() API that is provided as it handles some additional internal preprocessing like resetting UUIDs before the database is copied over to the final location

jda · May 4, 2021, 4:06pm

Ohhh… Then my implementation may not be good…?

Further discussion about details has been moved to a new separate thread

priya.rajagopal · May 4, 2021, 4:36pm

Can you please move this to a new post with relevant subject so we don’t hijack the original post. Then delete this one.

Topic		Replies	Views
When are checkpoints set? Stuck and won't update Mobile	19	3076	August 17, 2019
Revision conflicts caused by replication Mobile	3	1510	June 19, 2017
Document ChangeListener Sync Gateway	6	2795	August 23, 2016
Couchbase Lite Replicator Not Syncing after write Couchbase Lite android	14	1279	August 5, 2022
Continuous Replication, New Documents are not created Sync Gateway	2	1173	May 2, 2018

Replication - 409 Checkpoint Document update conflict

Related topics