Sync Gateway Data Synchronization Issue with Custom Collection

Mobile Couchbase Lite
,
1

With the recent enhancement in Sync Gateway supporting data synchronization from custom collections, we transitioned our data from the default collection to a custom one named “content”. We set up our sync gateway based on the guidelines provided in the official documentation, and everything seems to be correctly configured.

While the server-side adjustments went smoothly, we’re facing challenges on the client side with Couchbase Lite. We’ve updated the code to reference the “content” collection, maintaining the rest of the logic as before (which was functional). But, now, the replication consistently fails.

Unfortunately, I can’t attach the debug log directly due to forum limitations, so I’ve copied the debug log below and here’s a relevant observation: the REPLICATOR state change is going from STOPPED => CONNECTING => BUSY => IDLE => BUSY => STOPPED. I’m under the impression that a successful replication might include states like CONNECTED. Does this offer any insights?

I’d be grateful for any advice or direction from experienced Couchbase Lite users on this. If any more information is needed, please let me know and I will be happy to provide. Thanks in advance!

10-24 10:58:12.005     45 I/CouchbaseLite/REPLICATOR: {CollectionChangeNotifier#7}==> litecore::CollectionChangeNotifier 0x40700fae48 @0x40700fae48
10-24 10:58:12.006     45 I/CouchbaseLite/REPLICATOR: {CollectionChangeNotifier#7} Created, starting after #18446744073709551615
10-24 10:58:12.011     45 I/CouchbaseLite/REPLICATOR: [JAVA] Replicator is starting
10-24 10:58:12.014     45 D/CouchbaseLite/REPLICATOR: [JAVA] Replication options: {headers={User-Agent=CouchbaseLite/3.1.1-5 (Java 11.0.19; Linux) CE/release Build/5 Commit/8fb2eeca24@f547bc48fa4a Core/3.1.1 (10)}, maxRetryInterval=300, maxRetries=9, acceptParentDomainCookies=false, auth={password=<password>, type=Basic, username=<username>}, heartbeat=300, autoPurge=true}
10-24 10:58:12.037     45 D/CouchbaseLite/REPLICATOR: [JAVA] Setting c4 replicator @0x1b3e1785 for replicator @0x6856096
10-24 10:58:12.083     45 I/CouchbaseLite/NETWORK: {Connection#10}==> litecore::blip::Connection ->ws://sync-gateway:4984/main-content/_blipsync @0x4070133910
10-24 10:58:12.083     45 I/CouchbaseLite/NETWORK: {Connection#10} Opening connection...
10-24 10:58:12.089     45 I/CouchbaseLite/REPLICATOR: {Repl#11}==> litecore::repl::Replicator /tmp/assetbuilder_1126052868787576212/com.tgsys.platform.compliance/com-tgsys-platform-compliance.cblite2/ ->ws://sync-gateway:4984/main-content/_blipsync @0x407008f840
10-24 10:58:12.089     45 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} {{Coll#0} "content": {"Push": disabled, "Pull": continuous}}
Options={acceptParentDomainCookies:false, auth:{password:"********", type:"Basic", username:"<username>"}, autoPurge:true, headers:{User-Agent:"CouchbaseLite/3.1.1-5 (Java 11.0.19; Linux) CE/release Build/5 Commit/8fb2eeca24@f547bc48fa4a Core/3.1.1 (10)"}, heartbeat:300, maxRetries:9, maxRetryInterval:300}
10-24 10:58:12.097     45 I/CouchbaseLite/REPLICATOR: {C4Replicator#13}==> litecore::C4RemoteReplicator 0x4070088ca0 @0x4070088ca0
10-24 10:58:12.099     45 V/CouchbaseLite/REPLICATOR: {C4Replicator#13} C4RemoteRepl 0x4070088ca0 created Repl 0x407008f840
10-24 10:58:12.099     45 I/CouchbaseLite/REPLICATOR: {C4Replicator#13} Starting Replicator {Repl#11} with config: {{{Coll#0} "content": {"Push": disabled, "Pull": continuous}}
Options={acceptParentDomainCookies:false, auth:{password:"********", type:"Basic", username:"username"}, autoPurge:true, headers:{User-Agent:"CouchbaseLite/3.1.1-5 (Java 11.0.19; Linux) CE/release Build/5 Commit/8fb2eeca24@f547bc48fa4a Core/3.1.1 (10)"}, heartbeat:300, maxRetries:9, maxRetryInterval:300}}
10-24 10:58:12.110     54 V/CouchbaseLite/QUERY: RecordEnumerator 0x40ec00a748: (.content, 110 0)
10-24 10:58:12.116     45 I/CouchbaseLite/REPLICATOR: [JAVA] State changed STOPPED => CONNECTING(0/0): null for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:12.118     55 I/CouchbaseLite/NETWORK: {C4SocketImpl#14}==> litecore::repl::C4SocketImpl ws://sync-gateway:4984/main-content/_blipsync @0x40700d7b18
10-24 10:58:12.119     56 I/CouchbaseLite/NETWORK: {C4SocketImpl#14} Connecting...
10-24 10:58:12.124     58 D/CouchbaseLite/NETWORK: [JAVA] ^C4Socket.open@40700d7c58: null@108c22cd
10-24 10:58:12.143     65 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#0} Scanning for pre-existing conflicts...
10-24 10:58:12.146     59 I/CouchbaseLite/REPLICATOR: [JAVA] status changed: (0, 0) @C4ReplicatorStatus{level=2,completed=0,total=0,#docs=0,domain=0,code=0,info=0} for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:12.148     59 I/CouchbaseLite/REPLICATOR: [JAVA] State changed CONNECTING => CONNECTING(0/0): null for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:12.151     66 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#0} Found 0 conflicted docs in 0.049 sec
10-24 10:58:12.164     72 I/CouchbaseLite/REPLICATOR: {SequenceTracker#15}==> litecore::SequenceTracker 0x40701392a0 @0x40701392a0
10-24 10:58:12.165     73 I/CouchbaseLite/REPLICATOR: {SequenceTracker#15} begin transaction at #0
10-24 10:58:12.170     75 I/CouchbaseLite/REPLICATOR: {SequenceTracker#16}==> litecore::SequenceTracker 0x407012ec20 @0x407012ec20
10-24 10:58:12.172     76 I/CouchbaseLite/REPLICATOR: {SequenceTracker#16} begin transaction at #0
10-24 10:58:12.202     86 I/CouchbaseLite/REPLICATOR: {SequenceTracker#15} commit: sequences #1 -- #0
10-24 10:58:12.203     87 V/CouchbaseLite/REPLICATOR: {SequenceTracker#15} Removed 0 old entries (0 left; idle has 0, byDocID has 0)
10-24 10:58:12.204     88 I/CouchbaseLite/REPLICATOR: {SequenceTracker#16} commit: sequences #1 -- #0
10-24 10:58:12.205     89 V/CouchbaseLite/REPLICATOR: {SequenceTracker#16} Removed 0 old entries (0 left; idle has 0, byDocID has 0)
10-24 10:58:12.206     90 V/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Remote-DB ID 1 found for target <ws://sync-gateway:4984/main-content/_blipsync>
10-24 10:58:12.224     99 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#0} No local checkpoint 'cp-g1haE+xz5JUy9tG+D+OA+gseY1E9dbB/F/VAuLcewkc='
10-24 10:58:12.227    102 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=connecting: connectionState=1, savingChkpt=0
10-24 10:58:12.391     58 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.init: CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}
10-24 10:58:12.394     58 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.init: CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}
10-24 10:58:12.398     58 I/CouchbaseLite/DATABASE: [JAVA] New thread: CBL worker #2
10-24 10:58:12.410    103 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.coreRequestedOpen
10-24 10:58:12.413    103 D/CouchbaseLite/NETWORK: [JAVA] StateMachine@0x7beb0083: transition UNOPENED => OPENING
10-24 10:58:12.414    103 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.open: ws://sync-gateway:4984/main-content/_blipsync, {headers={User-Agent=CouchbaseLite/3.1.1-5 (Java 11.0.19; Linux) CE/release Build/5 Commit/8fb2eeca24@f547bc48fa4a Core/3.1.1 (10)}, acceptParentDomainCookies=false, maxRetries=9, maxRetryInterval=300, auth={password=<password>, type=Basic, username=<username>}, heartbeat=300, WS-Protocols=BLIP_3+CBMobile_3,BLIP_3+CBMobile_2, autoPurge=true}
10-24 10:58:12.534    104 I/CouchbaseLite/DATABASE: ... SELECT sql FROM sqlite_master WHERE name = ? AND type = ? AND tbl_name = ?
10-24 10:58:12.538    104 I/CouchbaseLite/DATABASE: ... SELECT sequence, flags, null, version, body, extra FROM "kv_info" WHERE key=?
10-24 10:58:12.635    104 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.onOpen: Response{protocol=http/1.1, code=101, message=Switching Protocols, url=http://sync-gateway:4984/main-content/_blipsync}
10-24 10:58:12.642    104 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.remoteOpened: {Server=Couchbase Sync Gateway/3.1.1 CE, Connection=Upgrade, Sec-Websocket-Protocol=BLIP_3+CBMobile_3, Upgrade=websocket, Sec-Websocket-Accept=bGnyUtu0nYoS1ct4Xcvu0m5kjhg=}
10-24 10:58:12.648    104 D/CouchbaseLite/NETWORK: [JAVA] StateMachine@0x7beb0083: transition OPENING => OPEN
10-24 10:58:12.652    104 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.ackOpenToCore
10-24 10:58:12.658    104 I/CouchbaseLite/NETWORK: {C4SocketImpl#14} Got HTTP response (status 101)
10-24 10:58:12.659    104 I/CouchbaseLite/NETWORK: {C4SocketImpl#14} Connected!
10-24 10:58:12.666    107 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=connecting: connectionState=1, savingChkpt=0
10-24 10:58:12.667    104 I/CouchbaseLite/NETWORK: {Connection#10} Connected!
10-24 10:58:12.668    108 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Connected!
10-24 10:58:12.669    109 V/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Requesting get collections
10-24 10:58:12.671    110 I/CouchbaseLite/NETWORK: {BLIPIO#17}==> litecore::blip::BLIPIO ->ws://sync-gateway:4984/main-content/_blipsync @0x407013da30
10-24 10:58:12.673    111 V/CouchbaseLite/NETWORK: {BLIPIO#17} WebSocket is hungry!
10-24 10:58:12.675    112 V/CouchbaseLite/NETWORK: {BLIPIO#17} Starting writeToWebSocket loop...
10-24 10:58:12.677    113 V/CouchbaseLite/NETWORK: {BLIPIO#17} ...Wrote 0 bytes to WebSocket (writeable=1)
10-24 10:58:12.678    114 I/CouchbaseLite/NETWORK: SENDING: REQ #0  {
	Profile: getCollections
	BODY: {"checkpoint_ids":["cp-g1haE+xz5JUy9tG+D+OA+gseY1E9dbB/F/VAuLcewkc="],"collections":["content"]} }
10-24 10:58:12.682    115 V/CouchbaseLite/NETWORK: {BLIPIO#17} Sending 'getCollections' REQ #0
10-24 10:58:12.685    116 V/CouchbaseLite/NETWORK: {BLIPIO#17} Requeuing REQ #0...
10-24 10:58:12.686    117 V/CouchbaseLite/NETWORK: {BLIPIO#17} Starting writeToWebSocket loop...
10-24 10:58:12.690    120 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=busy: connectionState=2, savingChkpt=0
10-24 10:58:12.691    121 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} now busy
10-24 10:58:12.693    122 I/CouchbaseLite/REPLICATOR: {C4Replicator#13} State: busy, progress=0.00%
10-24 10:58:12.694    124 V/CouchbaseLite/NETWORK: {BLIPIO#17}     Sending frame: REQ #1 ----, bytes 0--123
10-24 10:58:12.697    125 V/CouchbaseLite/NETWORK: {C4SocketImpl#14} Sending 126-byte message
10-24 10:58:12.700    123 D/CouchbaseLite/REPLICATOR: [JAVA] C4Replicator.statusChangedCallback(0x6f7a624c) C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}: C4ReplicatorStatus{level=4,completed=0,total=0,#docs=0,domain=0,code=0,info=0}
10-24 10:58:12.700    126 D/CouchbaseLite/NETWORK: [JAVA] ^C4Socket.write@40700d7c58(126)
10-24 10:58:12.715    128 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=busy: connectionState=2, savingChkpt=0
10-24 10:58:12.715    127 I/CouchbaseLite/REPLICATOR: [JAVA] status changed: (0, 0) @C4ReplicatorStatus{level=4,completed=0,total=0,#docs=0,domain=0,code=0,info=0} for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:12.717    127 I/CouchbaseLite/REPLICATOR: [JAVA] State changed CONNECTING => BUSY(0/0): null for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:12.719    129 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.coreWrites(126)
10-24 10:58:12.720    130 V/CouchbaseLite/NETWORK: {BLIPIO#17} Finished sending REQ #1
10-24 10:58:12.720    129 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.write(126)
10-24 10:58:12.722    131 V/CouchbaseLite/NETWORK: {BLIPIO#17} ...Wrote 126 bytes to WebSocket (writeable=1)
10-24 10:58:12.733    129 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.ackWriteToCore(126)
10-24 10:58:12.736    104 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.onBytes(47)
10-24 10:58:12.741    104 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.remoteWrites(47)
10-24 10:58:12.741    104 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.sendToCore(47)
10-24 10:58:12.744    104 V/CouchbaseLite/NETWORK: {C4SocketImpl#14} Received 47-byte message
10-24 10:58:12.745    132 V/CouchbaseLite/NETWORK: {BLIPIO#17} Received frame: RES #1 ---C, length    45
10-24 10:58:12.746    133 V/CouchbaseLite/NETWORK: {BLIPIO#17} RES #1 has reached the end of its frames
10-24 10:58:12.762    139 V/CouchbaseLite/NETWORK: {Connection#10} Receiving RES #1 Z
10-24 10:58:12.771    142 V/CouchbaseLite/NETWORK: {Connection#10} Finished receiving RES #1 Z
10-24 10:58:12.773    143 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#0} No remote checkpoint 'cp-g1haE+xz5JUy9tG+D+OA+gseY1E9dbB/F/VAuLcewkc='
10-24 10:58:12.774    144 I/CouchbaseLite/NETWORK: RECEIVED: RES #1 Z {
	Content-Type: application/json
	BODY: [{}] }
10-24 10:58:12.775    146 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=busy: connectionState=2, savingChkpt=0
10-24 10:58:12.777    145 D/CouchbaseLite/NETWORK: [JAVA] ^C4Socket.completedReceive@40700d7c58(47)
10-24 10:58:12.777    147 I/CouchbaseLite/REPLICATOR: {Puller#20}==> litecore::repl::Puller {Repl#11} @0x40700d5110
10-24 10:58:12.778    148 I/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} Starting pull from remote seq ''
10-24 10:58:12.779    149 I/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} msg["revocations"]="true" due to enableAutoPurge()=1 or progressNotificationLevel()=0 > 0
10-24 10:58:12.782    150 I/CouchbaseLite/NETWORK: SENDING: REQ #0  {
	Profile: subChanges
	collection: 0
	continuous: true
	batch: 200
	versioning: rev-trees
	activeOnly: true
	revocations: true
	filter: sync_gateway/bychannel
	channels: com.tgsys.platform.compliance }
10-24 10:58:12.786    151 I/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} activityLevel=busy: pendingResponseCount=1, _caughtUp=0, _pendingRevMessages=0, _activeIncomingRevs=0, _waitingRevMessages=0, _unfinishedIncomingRevs=0
10-24 10:58:12.788    152 V/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} now busy
10-24 10:58:12.790    153 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} pushStatus=stopped, pullStatus=busy, progress=0/0/0
10-24 10:58:12.792    154 V/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Replicator status collection-wise: pushStatus=stopped, pullStatus=busy, progress=0/0, docCount=0
10-24 10:58:12.793    155 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=busy: connectionState=2, savingChkpt=0
10-24 10:58:12.798     59 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.coreAckReceive: 47
10-24 10:58:12.799    156 V/CouchbaseLite/NETWORK: {BLIPIO#17} Sending 'subChanges' REQ #0
10-24 10:58:12.800    157 V/CouchbaseLite/NETWORK: {BLIPIO#17} Requeuing REQ #0...
10-24 10:58:12.801    158 V/CouchbaseLite/NETWORK: {BLIPIO#17} Starting writeToWebSocket loop...
10-24 10:58:12.803    160 V/CouchbaseLite/NETWORK: {BLIPIO#17}     Sending frame: REQ #2 ----, bytes 0--187
10-24 10:58:12.804    161 I/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} activityLevel=busy: pendingResponseCount=1, _caughtUp=0, _pendingRevMessages=0, _activeIncomingRevs=0, _waitingRevMessages=0, _unfinishedIncomingRevs=0
10-24 10:58:12.805    162 V/CouchbaseLite/NETWORK: {C4SocketImpl#14} Sending 190-byte message
10-24 10:58:12.806    163 D/CouchbaseLite/NETWORK: [JAVA] ^C4Socket.write@40700d7c58(190)
10-24 10:58:12.807    103 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.coreWrites(190)
10-24 10:58:12.808    103 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.write(190)
10-24 10:58:12.809    103 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.ackWriteToCore(190)
10-24 10:58:12.811    164 V/CouchbaseLite/NETWORK: {BLIPIO#17} Finished sending REQ #2
10-24 10:58:12.812    104 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.onBytes(7)
10-24 10:58:12.812    165 V/CouchbaseLite/NETWORK: {BLIPIO#17} ...Wrote 190 bytes to WebSocket (writeable=1)
10-24 10:58:12.813    104 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.remoteWrites(7)
10-24 10:58:12.813    104 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.sendToCore(7)
10-24 10:58:12.814    104 V/CouchbaseLite/NETWORK: {C4SocketImpl#14} Received 7-byte message
10-24 10:58:12.815    166 V/CouchbaseLite/NETWORK: {BLIPIO#17} Received frame: RES #2 ----, length     5
10-24 10:58:12.815    104 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.onBytes(76)
10-24 10:58:12.816    104 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.remoteWrites(76)
10-24 10:58:12.817    104 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.sendToCore(76)
10-24 10:58:12.817    167 V/CouchbaseLite/NETWORK: {BLIPIO#17} RES #2 has reached the end of its frames
10-24 10:58:12.818    104 V/CouchbaseLite/NETWORK: {C4SocketImpl#14} Received 76-byte message
10-24 10:58:12.820    169 V/CouchbaseLite/NETWORK: {Connection#10} Receiving RES #2
10-24 10:58:12.821    170 V/CouchbaseLite/NETWORK: {Connection#10} Finished receiving RES #2
10-24 10:58:12.822    171 I/CouchbaseLite/NETWORK: RECEIVED: RES #2  { }
10-24 10:58:12.822    172 D/CouchbaseLite/NETWORK: [JAVA] ^C4Socket.completedReceive@40700d7c58(7)
10-24 10:58:12.823    173 I/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} activityLevel=busy: pendingResponseCount=0, _caughtUp=0, _pendingRevMessages=0, _activeIncomingRevs=0, _waitingRevMessages=0, _unfinishedIncomingRevs=0
10-24 10:58:12.823    174 V/CouchbaseLite/NETWORK: {BLIPIO#17} Received frame: REQ #1 --NC, length    74
10-24 10:58:12.824    127 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.coreAckReceive: 7
10-24 10:58:12.827    179 V/CouchbaseLite/NETWORK: {Connection#10} Receiving 'changes' REQ #1 NZ
10-24 10:58:12.829    182 V/CouchbaseLite/NETWORK: {Connection#10} Finished receiving 'changes' REQ #1 NZ
10-24 10:58:12.830    183 I/CouchbaseLite/NETWORK: RECEIVED: REQ #1 NZ {
	Profile: changes
	collection: 0
	Content-Type: application/json
	BODY: null }
10-24 10:58:12.831    184 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=busy: connectionState=2, savingChkpt=0
10-24 10:58:12.832    185 I/CouchbaseLite/REPLICATOR: {RevFinder#21}==> litecore::repl::RevFinder {Repl#11} @0x4070143200
10-24 10:58:12.832    186 V/CouchbaseLite/REPLICATOR: {RevFinder#21} {Coll#0} Handling 'changes' REQ#1
10-24 10:58:12.833    187 I/CouchbaseLite/REPLICATOR: {RevFinder#21} {Coll#0} Caught up with remote changes
10-24 10:58:12.834    188 I/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} activityLevel=idle: pendingResponseCount=0, _caughtUp=1, _pendingRevMessages=0, _activeIncomingRevs=0, _waitingRevMessages=0, _unfinishedIncomingRevs=0
10-24 10:58:12.834    189 D/CouchbaseLite/NETWORK: [JAVA] ^C4Socket.completedReceive@40700d7c58(76)
10-24 10:58:12.835    129 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.coreAckReceive: 76
10-24 10:58:12.835    190 V/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} now idle
10-24 10:58:12.836    191 V/CouchbaseLite/REPLICATOR: {RevFinder#21} {Coll#0} now idle
10-24 10:58:12.837    192 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} pushStatus=stopped, pullStatus=idle, progress=0/0/0
10-24 10:58:12.838    193 V/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Replicator status collection-wise: pushStatus=stopped, pullStatus=idle, progress=0/0, docCount=0
10-24 10:58:12.838    194 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=busy: connectionState=2, savingChkpt=0
10-24 10:58:12.839    195 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} pushStatus=stopped, pullStatus=idle, progress=0/0/0
10-24 10:58:12.839    196 V/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Replicator status collection-wise: pushStatus=stopped, pullStatus=idle, progress=0/0, docCount=0
10-24 10:58:12.840    197 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=idle: connectionState=2, savingChkpt=0
10-24 10:58:12.840    198 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} now idle
10-24 10:58:12.841    199 I/CouchbaseLite/REPLICATOR: {C4Replicator#13} State: idle, progress=0.00%
10-24 10:58:12.842    200 D/CouchbaseLite/REPLICATOR: [JAVA] C4Replicator.statusChangedCallback(0x6f7a624c) C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}: C4ReplicatorStatus{level=3,completed=0,total=0,#docs=0,domain=0,code=0,info=0}
10-24 10:58:12.842     59 I/CouchbaseLite/REPLICATOR: [JAVA] status changed: (0, 0) @C4ReplicatorStatus{level=3,completed=0,total=0,#docs=0,domain=0,code=0,info=0} for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:12.843     59 I/CouchbaseLite/REPLICATOR: [JAVA] State changed BUSY => IDLE(0/0): null for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
23-10-24 10:58:14.207 [DefaultDispatcher-worker-4]  INFO  AssetBuilder - replication progress – completed:0, total:0, activity:IDLE
23-10-24 10:58:14.213 [DefaultDispatcher-worker-4]  INFO  AssetBuilder - finished replication
23-10-24 10:58:14.324 [DefaultDispatcher-worker-4]  INFO  o.c.plexus.archiver.zip.ZipArchiver - Building zip: /assetbuilder_1126052868787576212/com.tgsys.platform.compliance.zip
10-24 10:58:14.552     45 I/CouchbaseLite/REPLICATOR: [JAVA] Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}: Replicator is stopping (C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'})
URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:14.561    204 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Told to stop!
10-24 10:58:14.562    205 I/CouchbaseLite/NETWORK: {Connection#10} Closing with code 1000, msg ''
10-24 10:58:14.563    206 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=busy: connectionState=3, savingChkpt=0
10-24 10:58:14.564    207 I/CouchbaseLite/NETWORK: {C4SocketImpl#14} Requesting close with status=1000, message=''
10-24 10:58:14.565    208 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} now busy
10-24 10:58:14.565    209 D/CouchbaseLite/NETWORK: [JAVA] ^C4Socket.requestClose@40700d7c58(1000): 'null'
10-24 10:58:14.566    210 I/CouchbaseLite/REPLICATOR: {C4Replicator#13} State: busy, progress=0.00%
10-24 10:58:14.568    211 D/CouchbaseLite/REPLICATOR: [JAVA] C4Replicator.statusChangedCallback(0x6f7a624c) C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}: C4ReplicatorStatus{level=4,completed=0,total=0,#docs=0,domain=0,code=0,info=0}
10-24 10:58:14.568    103 I/CouchbaseLite/REPLICATOR: [JAVA] status changed: (0, 0) @C4ReplicatorStatus{level=4,completed=0,total=0,#docs=0,domain=0,code=0,info=0} for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:14.569    103 I/CouchbaseLite/REPLICATOR: [JAVA] State changed IDLE => BUSY(0/0): null for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:14.576    127 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.coreRequestsCloseCloseStatus{6:1000, null}
10-24 10:58:14.577    127 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.close: CloseStatus{6:1000, null}
10-24 10:58:14.583    104 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.onClosing(1000): ''
10-24 10:58:14.588    104 D/CouchbaseLite/NETWORK: [JAVA] CBLWebSocket@@0x3d628b88{vC4Socket@x40700d7c58 <=> OkHttpSocket@0x58f266ba(ws://sync-gateway:4984/main-content/_blipsync)}.remoteRequestsClose: CloseStatus{0:1000, }
10-24 10:58:14.588    104 D/CouchbaseLite/NETWORK: [JAVA] StateMachine@0x7beb0083: transition OPEN => CLOSING
10-24 10:58:14.589    104 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.requestCoreClose(1000): ''
10-24 10:58:14.591    104 V/CouchbaseLite/NETWORK: {C4SocketImpl#14} Calling requestClose when the socket is pending close
10-24 10:58:14.592    104 D/CouchbaseLite/NETWORK: [JAVA] OkHttpSocket@0x58f266ba.onClosed(1000): ''
10-24 10:58:14.596    104 D/CouchbaseLite/NETWORK: [JAVA] %s.remoteClosed(%d): %s
10-24 10:58:14.597    104 D/CouchbaseLite/NETWORK: [JAVA] StateMachine@0x7beb0083: transition CLOSING => CLOSED
10-24 10:58:14.597    104 D/CouchbaseLite/NETWORK: [JAVA] vC4Socket@x40700d7c58.closeCore(1, 0): ''
10-24 10:58:14.603    104 I/CouchbaseLite/NETWORK: {C4SocketImpl#14} WebSocket closed normally
10-24 10:58:14.604    104 I/CouchbaseLite/NETWORK: {C4SocketImpl#14} sent 316 bytes, rcvd 130, in 1.939 sec (163/sec, 67/sec)
10-24 10:58:14.605    212 I/CouchbaseLite/NETWORK: {Connection#10} Closed with WebSocket/HTTP status 1000:
10-24 10:58:14.606    213 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} Connection closed with WebSocket/HTTP status 1000: "" (state=3)
10-24 10:58:14.608    214 I/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} activityLevel=stopped: pendingResponseCount=0, _caughtUp=1, _pendingRevMessages=0, _activeIncomingRevs=0, _waitingRevMessages=0, _unfinishedIncomingRevs=0
10-24 10:58:14.609    215 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} activityLevel=stopped: connectionState=0, savingChkpt=0
10-24 10:58:14.610    216 V/CouchbaseLite/REPLICATOR: {Puller#20} {Coll#0} now stopped
10-24 10:58:14.612    217 I/CouchbaseLite/REPLICATOR: {Repl#11} {Coll#-1} now stopped
10-24 10:58:14.633    226 I/CouchbaseLite/REPLICATOR: {C4Replicator#13} State: stopped, progress=0.00%
10-24 10:58:14.635    227 D/CouchbaseLite/REPLICATOR: [JAVA] C4Replicator.statusChangedCallback(0x6f7a624c) C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}: C4ReplicatorStatus{level=0,completed=0,total=0,#docs=0,domain=0,code=0,info=0}
10-24 10:58:14.636    129 I/CouchbaseLite/REPLICATOR: [JAVA] status changed: (0, 0) @C4ReplicatorStatus{level=0,completed=0,total=0,#docs=0,domain=0,code=0,info=0} for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:14.636    228 I/CouchbaseLite/REPLICATOR: BLIP sent 2 msgs (316 bytes), rcvd 1 msgs (130 bytes) in 1.972 sec. Max outbox depth was 1, avg 1.00
10-24 10:58:14.637    129 I/CouchbaseLite/REPLICATOR: [JAVA] State changed BUSY => STOPPED(0/0): null for Replicator{@0x6856096(<*),Database{@@0x28479b6e: 'com-tgsys-platform-compliance'} => URLEndpoint{url=ws://sync-gateway:4984/main-content}}$C4CommonRepl{@0x1b3e1785/@x4070088c90, @0x6856096, SocketFactory{@URLEndpoint{url=ws://sync-gateway:4984/main-content}}'}
10-24 10:58:14.640    129 D/CouchbaseLite/DATABASE: [JAVA] Active processes com-tgsys-platform-compliance: 0
10-24 10:58:14.642     45 I/CouchbaseLite/QUERY: {Query#22} Closing query (db is closing)
10-24 10:58:14.643     45 I/CouchbaseLite/QUERY: {Query#5} Closing query (db is closing)
23-10-24 10:58:14.656 [DefaultDispatcher-worker-4]  INFO  AssetBuilder - database closed, directories cleaned up. exiting.
10-24 10:58:43.108      3 I/CouchbaseLite/REPLICATOR: {C4Replicator#13} Freeing C4BaseReplicator
2

Any inputs from couchbase lite experts would be appreciated on this. We are basically stuck in this. We have almost rebuilt the code going step by step but the behavior is the same.

Another possible log statement that could be of interest is the following:

10-24 10:58:12.782    150 I/CouchbaseLite/NETWORK: SENDING: REQ #0  {
	Profile: subChanges
	collection: 0
	continuous: true
	batch: 200
	versioning: rev-trees
	activeOnly: true
	revocations: true
	filter: sync_gateway/bychannel
	channels: com.tgsys.platform.compliance }

Should the value for collection be “content” instead of 0?

@blake.meike @priya.rajagopal your inputs would be great.

3

Hey @PShri : what I need to see is the failure. If this is failing consistently, it should be relatively easy to get the log from the failure.

The log that you pasted in there looks like a perfectly successful replication. Verifying that with the experts, now.

4

I am not sure about the technical definition of “success” and “failure”. In my thinking, if the replicator pulls the expected number of documents to the client, it is success. Here, the couchbase lite is empty after the replication. There could be some configuration issues on my side, but I have diligently followed the steps in the documentation and we have doubled checked, triple checked the code and logic to be sure. It appears to be correct.

Looking forward to hearing back from you.

5

Well, technically, In that log, the Replicator connects to the remote, sees no documents that need to be transferred, and stops. You can see that in this log message:

10-24 10:58:14.569    103 I/CouchbaseLite/REPLICATOR: [JAVA] State changed IDLE => BUSY(0/0): null

[key: <PREVIOUS_STATE> => <CURRENT_STATE>(<transferred>/<total>): <error>]

I suspect some confusion in the configuration of the replicator.

Can I see how you are saving documents in the Mobile db and how you are configuring the DB in your SG?

6

Here is the sync gateway configuration

{
  "bucket": "main",
  "name": "main-content",
  "guest": {
    "disabled": true
  },
  "import_docs": true,
  "num_index_replicas": 0,
  "enable_shared_bucket_access": true,
  "scopes": {
    "_default": {
      "collections": {
        "content": {
          "sync": `
            function(doc, oldDoc, meta) {
              channel(doc.module_identifier);
            }
          `,
          "import_filter": `
            function(doc) {
              return true;
            }
          `
        }
      }
    }
  }
}

Here is the Couchbase Lite replicator code:

      // Configure database
      val config = DatabaseConfiguration()

      // Copy and prepare existing app data asset builder to the local machine
      val directoryPair = setupDataAssetBuildDirectory(assetId) // get the directory
      config.directory = directoryPair.second.toAbsolutePath().toString()
      val database = Database(databaseName, config)
      val destinationCollection = database.createCollection("content", Scope.DEFAULT_NAME)

      // replication configurations
      val targetEndpoint: Endpoint = URLEndpoint(URI("ws://localhost:4984/main-content/"))

      val collectionConfiguration =
        CollectionConfiguration(listOf("com.tgsys.platform.compliance"), null, { _, _ -> true }, null, null)
      val replicatorConfig =
        ReplicatorConfiguration(targetEndpoint)
          // collection to be replicated from
          .addCollection(destinationCollection, collectionConfiguration)
          // pull replication
          .setType(ReplicatorType.PUSH_AND_PULL)
          // continuous replication
          .setContinuous(false)
          // add authentication
          .setAuthenticator(BasicAuthenticator("<un>", "<pwd>".toCharArray()))

      // Create replicator
      val replicator = Replicator(replicatorConfig)

      // Listen to replicator change events.
      replicator.addChangeListener { change: ReplicatorChange ->
        // This is run in a different thread. So, we have to put the MDC Value again (TODO : Figure
        // out a better way to do this)
        MDC.put("asset", assetId)
        if (change.status.error != null) {
          logger.error("Error Code::  " + change.status.error!!.code)
        }
      }

      // add a change listener to the collection
      destinationCollection.addChangeListener { collectionChange ->
        // This is run in a different thread. So, we have to put the MDC Value again
        // (TODO : Figure out a better way to do this)
        MDC.put("asset", assetId)
        logger.info("documents added")
        collectionChange.documentIDs.map { logger.info("id: $it") }
      }

      // Start replication
      replicator.start(true)

      // Check status of replication and wait till it is completed
      while (!isReplicationDone(replicator)) {
        delay(2000)
        logger.info(
          "replication progress – completed:${replicator.status.progress.completed}, " +
            "total:${replicator.status.progress.total}, activity:${replicator.status.activityLevel.name}"
        )
      }

      logger.info("finished replication")
      database.close()
7

Just FYI, there’s a bunch of unnecessary code here. Since you are using Kotlin, you might want to consider using the Kotlin extension package… It might make some of this stuff a lot easier.

Database.createCollection(<collection name>) is more reliable way of creating a collection in the default scope. Also, specifying a null pull filter will have the same affect – and run much faster – than specifying the filter { _, _ -> true }.
You can, also, specify an executor on which to deliver Replicator.addChangeListener callbacks, if that will help.

Your SGW config will prevent any documents, except those that have exactly the value com.tgsys.platform.compliance in the field module_identifier from being replicated.
Also, since all docs have been assigned a channel, the user “<un>” – configured in the replicator – will have to have have access to the channel com.tgsys.platform.compliance

8

I don’t see a static method in the Database class, so I assume you are referring to the instance method createCollection. I am using that method (database.createCollection). Am I missing something here?

I had it as null. These are just the things that I have tried. There are several other minor changes I have made just to see if something works. I will set this to null.

That is the intent in this setup.

Right. The user has access to that channel.

9

Ok… so, the replicator doesn’t see any documents to be replicated

  • There are no documents in the collection
  • There are no documents with the value com.tgsys.platform.compliance in the field module_identifier in the collection
  • The user <un> doesn’t have access to the channel

This may be a bit much to analyze in this forum. Perhaps, at this point, you should open a support ticket?

I am using that method database.createCollection

You are using a method that takes two parameters. I suggest you use the one that take only one, the name of the collection.

10

@blake.meike Is there a support for community version too?

11

Ah. I’m sorry. There is not.

12

Hmmm. Any ideas that I can try out? I’m stuck here for 3 days now.

In the document metadata, I see sync gateway related changes. Like channel names, revision etc. So, I think the sync gateway configuration and its initialisation has gone fine. Here is an example:

{
  "meta": {
    "id": "03d664b4-8011-4f4b-afb5-b6341933d25d",
    "rev": "3-17915d4df19d00000000000000000000",
    "expiration": 0,
    "flags": 0,
    "type": "json"
  },
  "xattrs": {
    "_sync": {
      "rev": "1-b411756d9efd394a5b17cf0bd7a009c9",
      "sequence": 7649,
      "recent_sequences": [
        7649
      ],
      "channels": {
        "com.tgsys.platform.compliance": null
      },
      "cas": "0x00009df14d5d9117",
      "value_crc32c": "0xd8b3b100",
      "channel_set": [
        {
          "name": "com.tgsys.platform.compliance",
          "start": 7649
        }
      ],
      "channel_set_history": null,
      "time_saved": "2023-10-25T13:38:43.841953883Z",
      "cluster_uuid": "0499bfdc8fa1b1d860a66bba8fb7dbf8",
      "history": {
        "revs": [
          "1-b411756d9efd394a5b17cf0bd7a009c9"
        ],
        "parents": [
          -1
        ],
        "channels": [
          [
            "com.tgsys.platform.compliance"
          ]
        ]
      }
    }
  }
}

Why does it say that checkpoint not found in the log? We are building a fresh database. So, there should be no checkpoint right?

13

@blake.meike

I wanted to test the setup using _default collection instead of custom and it worked. So, this problem happens only with custom collections. I tried two different things:

  1. Keep sync gateway running, ingest the data to _default collection and execute the pull replication
  2. take down sync gateway, ingest the data to _default collection, start sync gateway and execute the pull replication

Both of the above worked. So, the issue is only with custom collections. Here is the log for the successful replication. I wasn’t able to paste it here because of the character limits. If you have any clue about why the custom collection isn’t working or want me to try something out, please let me know. I’m looking forward to getting this issue fixed. We have taken loads of efforts to migrate our data to custom collection. So, getting it fixed would be great. Thanks in advance.

14

I am not expert at this: I’m the Android guy. I will see if there is anyone else around here that can provide a suggestion. I’ve already listed the things that seem likely to me. Let me reiterate them and add one:

  • Be sure you are using a SG that supports collection
  • Be sure that there are documents with a value in the field module_identifier that corresponds to the channel filter you are setting in your replicator, in the collection you are replicating
  • Be sure that the user has access to the channel

I might suggest trying to get replication to a custom channel to work, without the channel filter, as a first step. Once you are replicating the channel, add the channel filter.

15

How is your user configured? Since there are not dynamic channel grants in your sync function (use of access) I assume you have explicit channels in your user. Channels are scoped to collection, so you need to add the channels per collection.

When setting up a user, you need to use collection_access parameter, if you are using admin_channels. This could be better documented as per Loading....

If this is the issue, the payload for user creation would look something like:

{
  "collection_access": {
    "myscope": {
      "mycollection": {
        "admin_channels": [
          "com.tgsys.platform.compliance"
        ],
      },
  }
1 Like
16

This is how I have been creating user:

curl --location --request POST 'http://localhost:4985/main-content/_user/' --header 'Content-Type: application/json' --header 'Authorization: Basic <auth-token>' \ 
--data-raw \
'
{
  "name": "content-collector",
  "password": "password",
  "admin_channels": [
    "*"
  ],
  "disabled": false
}
'

I will use the one you shared and report any findings.

17

Success!

Thanks @torcolvin for pinpointing the issue. I never would have identified that as the root cause.

I didn’t come across any documentation addressing this particular point. Adding this in the Sync Gateway documentation would help users.

18

While we are at this, I want to learn about providing access using sync_functions too.

Could you please point me to some documentation/examples where I can learn about how to provide access to specific/all users on specific collections too?