Sync Issue - Please Help!

Mobile Couchbase Lite
1

I am using Couchbase-Lite-PhoneGap-Plugin version 1.3.1 on Android 6.
I am creating the push/pull replication tasks using coax by passing session id as cookie, initially documents gets properly synced to and from sync gateway. But after few hours even with the valid session replication is not working and I am getting following error in adb logcat:

RemoteRequest: com.couchbase.lite.replicator.RemoteRequestRetry@d422b39: isTransientError, httpResponse: Response{protocol=http/1.1, code=401, message=Unauthorized, url=http://syncgateway-url/testdb/_local/97a82cc76d7a1c80674d7f9bc9625e289764c72d} e: com.couchbase.lite.replicator.RemoteRequestResponseException: Unauthorized
W Sync : com.couchbase.lite.replicator.ReplicationInternal$11@65e86d7: error getting remote checkpoint
W Sync : com.couchbase.lite.replicator.RemoteRequestResponseException: Unauthorized

D Listener: authHeader is null
D Listener: Unauthorized – requestCredentials not given or do not match allowed credentials

This error is part of https://github.com/couchbase/couchbase-lite-java-listener/blob/master/src/main/java/com/couchbase/lite/listener/LiteServlet.java#L56-L68
Part of LiteServlet.java code
String authHeader = req.getHeader(“Authorization”);
if (authHeader != null) {
// some code
}else{
Log.d(Log.TAG_LISTENER, “authHeader is null”);
}

Replication config that I am using:
pullRep = {
source: {
headers: {
Cookie: cookie,
},
url: config.site.syncUrl,
},
target: config.dbName,
};

This issue is similar to couchbase/couchbase-lite-android#817

2

As mentioned here, you can periodically check the validity of the cookie session against the SG REST API directly. When it returns a 401 Unauthorized, the client should request a new one and update the push/pull replications to use it.

3

The session is still valid, the whole problem is that, even thought the session is valid, the request returns Unauthorized. If we clear the cblite database, the replication works just fine. It looks like cblite is holding some invalid session even though the current/latest session is valid. We can see it through postman that the session is valid.

4

The session is still valid, the whole problem is that, even thought the session is valid, the request returns Unauthorized.

Which endpoint is that to? And is it a request to the Sync Gateway or Couchbase Lite REST API?

D Listener: authHeader is null
D Listener: Unauthorized – requestCredentials not given or do not match allowed credentials

Those logs point to a problem accessing the Listener, not running replications with Sync Gateway.

5

Thank you for the response. But not sure why we are getting this error. Seems like local checkpoint is failing to be authorized by sync gateway even though the session is valid

com.couchbase.lite.replicator.RemoteRequestRetry@e7b2c68: isTransientError, httpResponse: Response{protocol=http/1.1, code=401, message=Unauthorized, url=http://syncGatewayURL:4984/edc/_local/97a82cc76d7a1c80674d7f9bc9625e289764c72d} e: com.couchbase.lite.replicator.RemoteRequestResponseException: Unauthorized 04-03 09:33:01.412 21960 4498 D RemoteRequest: com.couchbase.lite.replicator.RemoteRequestRetry@e7b2c68: isTransientError, status code: 401 04-03 09:33:01.412 21960 4498 D RemoteRequest: com.couchbase.lite.replicator.RemoteRequestRetry@e7b2c68: isTransientError, return false 04-03 09:33:01.412 21960 4498 D RemoteRequest: com.couchbase.lite.replicator.RemoteRequestRetry$1@20aa714: RemoteRequestRetry failed, non-transient error. NOT retrying. url: http://syncGatewayURL:4984/edc/_local/97a82cc76d7a1c80674d7f9bc9625e289764c72d 04-03 09:33:01.412 21960 4498 W Sync : com.couchbase.lite.replicator.ReplicationInternal$11@ae96bbd: error getting remote checkpoint 04-03 09:33:01.412 21960 4498 W Sync : com.couchbase.lite.replicator.RemoteRequestResponseException: Unauthorized 04-03 09:33:01.412 21960 4498 W Sync : at com.couchbase.lite.replicator.RemoteRequest.executeRequest(RemoteRequest.java:265) 04-03 09:33:01.412 21960 4498 W Sync : at com.couchbase.lite.replicator.RemoteRequest.execute(RemoteRequest.java:165) 04-03 09:33:01.412 21960 4498 W Sync : at com.couchbase.lite.replicator.RemoteRequest.run(RemoteRequest.java:105) 04-03 09:33:01.412 21960 4498 W Sync : at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:423) 04-03 09:33:01.412 21960 4498 W Sync : at java.util.concurrent.FutureTask.run(FutureTask.java:237) 04-03 09:33:01.412 21960 4498 W Sync : at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:154) 04-03 09:33:01.412 21960 4498 W Sync : at java.util

6

We are making a request for sync gateway and it is returning us 401

INFO :: Sync Callback from restart Push Replication status : {“source”:“edc”,“task”:“repl002”,“status”:“Stopped”,“progress”:0,“transition_source”:“STOPPING”,“continuous”:true,“target”:“http://syncGatewayURL:4984/edc",“trigger”:“STOP_IMMEDIATE”,“transition_destination”:“STOPPED”,“error”:[401,"com.couchbase.lite.replicator.RemoteRequestResponseException: Unauthorized”],“type”:“Replication”} err : "needsLogin"
while starting the push replicator…

7

https://github.com/couchbase/sync_gateway/issues/1665 - This is the same issue we are seeing, any solution to this is very much appreciated.

8

Ok, I think we should define the scenario here + the expected behaviour and work our way from there rather than trying to troubleshoot in different directions. So far, this is what I understand the scenario to be:

  1. Create a session from the Sync Gateway REST API and retrieve the session id
  2. Set up a replication with that session id via the /_replicate endpoint
  3. Wait until the session id expires
  4. Request a new session id from Sync Gateway and update the replication object

Expected: The replication to use the latest session id and for documents to replicate.

Actual: The replication still uses the first session id instead which causes the 401 Unauthorized errors and documents are not replicated.

Does this sequence of steps match the operations performed in your code base?

9

Hi there, we are seeing a different issue. We are seeing the replication going in with the new session ID which is absolutely valid and sync gateway (via postman) is also returning valid when we pass that session.

There is an issue with the GET on local checkpoint returning. It returns unauthorized even if the session is valid.. The replicator stops at this point.

What ties sessionId to the Local check point?. Is that stored in CBLITE or the SyncGatway. Looks like even though the session is valid, the GET request for the local checkpoint fails.

10

The local checkpoint is used to keep track of up to where a particular Couchbase Lite database replicated to. So a device that starts a replication resumes from it stopped in the previous replication it ran. If it’s the first time a replication is setup, that checkpoint doc doesn’t exist so it returns a 404 Not Found which is fine.

The session id is only used for authentication.

So those two concepts are not directly related to one another. However I don’t know all the details of your scenario so perhaps it is something to look into. How are the session id 1 and 2 created? Do they correspond to the same Sync Gateway user?

11

We made some more progress, this is what is happening, From our code(Cordova) we are always sending the valid sessionId. We see in our chrome logs the valid sessionID Cblite within the replicator object going in the request, But CBLite is not sending the valid sessionID when it does a GET for the local checkpoint, it is sending the wrong session ID which is not valid anymore (f313**). How do we ensure that the request we pass to the continuous pull and push replication gets passed to SyncGateway from CBlite successfully.

Do we need to set an expire cookie so that cblite ignores it when that is not active anymore??

GET /edc/_local/486d3ddbfd029024a7a99287c6980b053dcc1c1f HTTP/1.1
Accept: multipart/related, application/json
User-Agent: CouchbaseLite/1.3 (1.3.1/5d96387011ce62d4d75269e4d40d285e8cf94ef8)
Accept-Encoding: gzip, deflate
Host: syncGatewayURL:4984
Connection: Keep-Alive
Cookie: SyncGatewaySession=f313725f810b7111bc78af49acafa84f6bfec1aa

12

Can you provide the code that sends the session id to Couchbase Lite? (the /_replicate endpoint)

13

==============
Actual Request
{
cancel: true,
continuous: true,
headers: “SyncGatewaySession=9ccc4d3430fdaa910dc35bee40cc0314ae7d4569”, //autogenerated
source: “edc”,
target: {
headers: {
Cookie: “SyncGatewaySession=9ccc4d3430fdaa910dc35bee40cc0314ae7d4569”, //autogenerated
url: “http://phx01-vm12.matrixhealth.net:4984/edc”
}
},
whatfunc: “pushVar”
}

===========================

function configureReplicators() {
logInfo(configureReplicators()...);
// cookie = cookie+";"+“Max-Age=30”;
pullRep = {
headers: cookie,
source: {
headers: {
Cookie: cookie,
},
url: config.site.syncUrl,
},
target: config.dbName,
continuous: true,
whatfunc: ‘pullVar’
};
pushRep = {
headers: cookie,
source: config.dbName,
target: {
headers: {
Cookie: cookie,
},
url: config.site.syncUrl,
},
continuous: true,
whatfunc: ‘pushVar’
};
}

/**

  • Cancel the replication, start once stopped

  • @param rep Push/Pull replicator

  • @param cb once replication is started again
    */
    // This runs when user logs in and also continuous push/pull
    function refreshSync(replicator, callback) {
    var cancel = JSON.parse(JSON.stringify(replicator));
    cancel.cancel = true;

    logInfo(Cancel Replicator cancel : ${JSON.stringify(cancel)});

    coax.post([config.server, “_replicate”], cancel, function(err) {
    if (err) {
    return logError(Nothing to cancel. err : ${JSON.stringify(err)});
    }

    logInfo(No error while cancelling replication. Starting ${replicator.whatfunc});
    coax.post([config.server, “_replicate”], replicator, callback);
    })
    }

Thank you again for helping with this. Please let me know if you need anything else.

14

The replicator algorithms that run within cblite make GET calls on Syncgateway giving the local check point correct?

When it does this, we can see in the headers, that the sessionId is wrong. How is is sending an invalid/inactive sessionID to syncgateway? Is this not setting the session ID properly or caching a bad invalid session?

We dont have control on that through our code. If we know how to make sure CBLite passes the right session/active sessionID (cblite -> syncgateway) we will be able to solve this problem.

tabletApp->cbliteRequest =>>> Valid Session ID (See it in the logs)
tabletAPP -> 4985 (the session that we got back from login service) -> returning valid.
cbliteRequest ===>> Syncgateway (Invalid session on local check point) to GET the local checkpoint
tabletAPP -> 4985 (the session that we got back from login service) -> returning valid even after the previous request says invalid.

15

Thanks for the detailed information. I think there are a couple areas to look into:

  • Is the request from cordova to CBL really sending the new cookie in the body and header? It looks like this is the case but I’m wondering if the webview is caching the old one. Furthermore, is the 1st session id invalid because it expired or is it being “destroyed” using one of the DELETE endpoints on the Public/Admin REST API?
  • How is the replication set up with the new session id. Is the first replication instance cancelled ({"cancel": true}) and then a new one is started with the new session id?
16

Hello there,

Is the request from cordova to CBL really sending the new cookie in the body and header? It looks like this is the case but I’m wondering if the webview is caching the old one. Furthermore, is the 1st session id invalid because it expired or is it being “destroyed” using one of the DELETE endpoints on the Public/Admin REST API?

**Ans : From the code we are sure that we are sending the new cookie in the body and header. How can we find out if the webview is really caching the old cookie? we are not sure where we can see that. **
The first session is invalid since it is being destroyed using one of the DELETE end points when user logs out of the app for some reason

How is the replication set up with the new session id. Is the first replication instance cancelled ({“cancel”: true}) and then a new one is started with the new session id?

Ans: The replication instance is cancelled and the new one is started, We took this from the Chat app. We thought this is cleaner to cancel all the replications and then start a new one.

Is this something we should not be doing? Please let us know. Is there a way to know what replication algorithm cblite that is getting invalid sessionId or how webview is caching the cookies?

17

Thanks for the further details.

Is the request from cordova to CBL really sending the new cookie in the body and header?

The cookie should be specified in the request body of the /_replicate request only. It’s not necessary to specify it in the header.

The replication instance is cancelled and the new one is started, We took this from the Chat app. We thought this is cleaner to cancel all the replications and then start a new one.

Updating the replication with the new cookie should be enough (a request with {"cancel": true} is not required but either way is fine).

18

I’ve seen this also, on both iOS and android.


19

We are sending the cookie in the body of _replicate. Also we will remove “cancel” : true from the replication and see if that helps.

We are looking at the recommended link to see if we are making any obvious mistakes.

http://docs-build.sc.couchbase.com/mobile/540/guides/couchbase-lite/rest-api.html#session-expiry

Thank you so much for the help.

20

Hi @preethi.minti,

CBL Android stores Cookie values into the database that the replicator belongs to. It means replicators, which belong to the same database, share the cookie value. I am wondering new replicator which has new session id starts before old replicator is still running. So old replicator’s HttpClient overwrites the cookie value with old one. This could happen because JavaScript is asynchronous.

Please make sure followings
After Logout:

  • Stop replicator(s)
  • Make sure all replicator stops by _active_tasks REST API
  • Cancel Session ID (Please not cancel Session ID before replicator completely stops)

After Login,

  • Not allow to login till all logout process completed.
  • Create new Session ID
  • Start new replicators (Make sure no replicators are running before starting this.)

Thanks,
Hideki