Q: Some devices often cannot synchronize individual data

HI,

I have a testing env:
three Android devices. often occurred document sync on two devices, but another hasn’t synced the document. restart the app, haven’t sync data can be synced.
how can I inspect the sync? how can I resolve this issue?

my env:
cb: ee6.0.1
lite: 2.5
sg: 2.5

thanks!

angular

are all your 3 android devices syncing up with sync gateway server? or these 3 devices are configured to do peer to peer sync?
and in your app, how did you configure your replicator? is replicator continuous set to true?

HI @eunice.huang,
thanks for your reply.

are all your 3 android devices syncing up with sync gateway server?

yes

or these 3 devices are configured to do peer to peer sync?

no, I only use sync with sg.

and in your app, how did you configure your replicator?

yes. I use offical todo demo. the configure statement as follow:

    Endpoint endpoint = new URLEndpoint(uri);
    ReplicatorConfiguration config = new ReplicatorConfiguration(database, endpoint)
            .setReplicatorType(ReplicatorConfiguration.ReplicatorType.PUSH_AND_PULL)
            .setContinuous(true);

is replicator continuous set to true?

yes.

I want to know

  1. lite will send the heartbeat to sync gateway? if I use wireshark catch network package, I haven’t caught any heartbeat package, is it normal?
  2. three device sync normal, but occasionally docuement not sync to all device, for example:
    A B C is android client.
    A save documents 1 2 3 4
    B sync 1 2 3 4
    C sync 1 2 3
    how can I inspect this issue? increase channel_cache_max_length number of sg property?
  3. couchbase sync mechanism is Tripartite: lite <—> sg <---->cb
    how can I detect strong consistency, make sure all lite device synced new documents.

thanks!

angular

In this example, is C always miss doc 4? or device which has missing doc can be random? or a device miss a random document? I wonder if there is anything related to channel config, are all 3 devices subscripted the same channels which all the documents can be shared through the channels?

Hi @angular,
In general, it is probably a lot easier to crank up the level on the debug log, in order to diagnose problems like this. There is nothing that you are going to see on the wire that you won’t see in the log.

Is there any chance that there are different users logged in on the different devices? If so, this might have something to do with access/security.

HI @eunice.huang @blake.meike,

thanks for your reply.

  1. In this example, is C always miss doc 4? or device which has missing doc can be random? or a device miss a random document?

a device misses a random document. but I not ensure device which has missing doc be random.

  1. I wonder if there is anything related to channel config,

the sync gateway keypoint config statement as follows:
“log”: [“*”],
“adminInterface”: “0.0.0.0:4985”,
“interface”: “0.0.0.0:4984”,
“databases”: {
“kitchen”: {
“server”: “ydd-cbs-0000.ydd-cbs.default.svc:8091”,
“bucket”: “kitchen”,
“username”: “kitchen”,
“password”: “Ydd.app@609”,
“cache”: {
“channel_cache_max_length”: 2000
},
“users”: {
“GUEST”: { “disabled”: true },
“admin”: {
“password”: “Ydd.app@169”,
“admin_roles”: [ “admin” ]
}
},
“roles”: {
“admin”: {}
},
“serverReadTimeout”: 5,
“serverWriteTimeout”: 5,
“enable_shared_bucket_access”: true,
“import_docs”: “continuous”,
“sync”:
`
function sync(doc, oldDoc) {

  1. are all 3 devices subscripted the same channels which all the documents can be shared through the channels?

yes. 3 devices subscribed the same channels with the same sg gateway user. all the documents can be shared through the channel.
we developing a restaurant management system, now the design mind is per restaurant has one sg user, the user can register logical users(management system user), per user has an own channel, I use the channels control which document will be sync.

  1. In general, it is probably a lot easier to crank up the level on the debug log, in order to diagnose problems like this.

I set the android logs as verbose, the sg logs as log[*]. the logs as follows:
android logs:

2019-06-06 12:35:15.650 15926-16295/com.couchbase.todo W/CouchbaseLite/NETWORK: WebSocketListener.onFailure() response → null: java.net.SocketException: Connection reset
2019-06-06 12:35:15.651 15926-15985/com.couchbase.todo W/C4Socket: C4Socket.dispose() handle → 547980277552
2019-06-06 12:35:15.652 15926-15985/com.couchbase.todo E/CouchbaseLite/REPLICATOR: {Repl#2}==> N8litecore4repl10ReplicatorE /data/user/0/com.couchbase.todo/files/user1.cblite2/ ->ws://62.234.199.112:4984/todo/_blipsync @0x7f940659c8
2019-06-06 12:35:15.653 15926-15985/com.couchbase.todo E/CouchbaseLite/REPLICATOR: {Repl#2} Got LiteCore error: POSIX error 104 “Connection reset by peer”
sg logs same the lite logs.

I have a question:
lite will send the heartbeat to sync gateway? if I use wireshark catch network package, I haven’t caught any heartbeat package, is it normal?
I set the lite logs level to DEBUG, but I haven’t see the websocket PING message.

thanks again.

angular

HI:

MultiChangesFeed(channels: {*}, options: {Since:0 Limit:0 Conflicts:false IncludeDocs:true Wait:true Continuous:true Terminator:0xc0005d8240 HeartbeatMs:25000 TimeoutMs:300000 ActiveOnly:false Ctx:}) … (to user1)

as above is when I use
curl -X GET -H "Cache-Control: no-cache" "http://user1:pass@111.231.94.xx:4984/todo/_changes?xxfeed=continuous&include_docs=true&heartbeat=26000&active_only=false"
comand get _changes endpoint, the sg logs infomation. it’s contain heartbeat=26000.


] MultiChangesFeed(channels: {*}, options: {Since:25 Limit:0 Conflicts:false IncludeDocs:false Wait:true Continuous:true Terminator:0xc0005d8300 HeartbeatMs:0 TimeoutMs:0 ActiveOnly:false Ctx:context.Background.WithValue(base.LogContextKey{}, base.LogContext{CorrelationID:“#091”}).WithValue(base.LogContextKey{}, base.LogContext{CorrelationID:“[3c2ef4f4]”})}) …

as above is using the lite sync with sg, the sg logs infomation. it contain heartbeat=0, Is it means the lite will not sending heartbeat message to sg?

thanks

angular

You seem to have made quite a number of different threads all referring to the same error message, which indicates that a connection reset occurred (i.e. the other side “hung up”). There is not a standard way to debug this as it is usually environmental. Do you devices have direct connections to the server, or are they connecting through any other nodes such as reverse proxy / load balancer. The majority of the time this error is caused by a misbehaving intermediate node which doesn’t understand web sockets and thinks they are dead connections and closes them.

To give some info about “heartbeat,” this is a leftover term from 1.x as technically 2.0+ does not use them anymore but instead uses web socket pings and pongs to accomplish the same idea. If your wireshark is set up to catch web socket messages then you might be able to see them but you will need to have started from the beginning of the conversation (i.e. before the web socket handshake) and have a wireshark sufficiently new enough to understand the web socket protocol. In fact if you have Wireshark 3.0 or above it is also capable of decoding the packets sent by Couchbase Lite and Sync Gateway (BLIP protocol). However, the “heartbeat” here is just a standard websocket ping/pong response (no BLIP involved).

All that being said, you are saying that the documents do not sync even after restarting the application? Does the same error show up again?

HI @borrrden,

thanks for your reply.

I understood the HeartBeat settings is for lite 1.x.

All that being said, you are saying that the documents do not sync even after restarting the application? Does the same error show up again?

I have many times test restart the application that it can resync data.
I useing wireshark 3.0.1, but can’t catch websocket heartbeat.
lite connected to sg, the sg show logs as follows:

MultiChangesFeed(channel:{*},options:{Since:25 Limit:0 Conflicts:false IncludeDocs:false Wait:true Continuous:true Terminator:0xc0005d8300 HeartbeatMs:0 TimeoutMs:0 ActiveOnly:false Ctx:context.Background.WithValue( base.LogContextKey {},base.LogContext {CorrelationID:“ #091 ”})。WithValue(base.LogContextKey {},base.LogContext {CorrelationID:“[3c2ef4f4]”})})…

I refrence forum:
https://www.couchbase.com/forums/t/sg-not-propagating-changes-to-couchbase-lite-client/18710/5
it’s contain:
2018-10-10 13:10:35.368931-0500 xxxxxx[6264:158564] CouchbaseLite WS Info: {C4SocketImpl#1} Sending PING
my lite logs levels set to:
Database.log.getConsole().setLevel(LogLevel.INFO);
if I use DEBUG logs level in lite, can I see the PING and PONG message?

PING message will auto send by lite? it will send by interval time or when connecting send once?
I think the network env is very complex that I can’t debug it. if this phenomenon occurred, can I resolve by program code? for example: restart the lite replicator interval 2 mins.Does it have a negative effect?

thanks
angular