Replication doesn't always work with adding a new channel

Mobile
1

When I add a new channel for a pull replication, data for that channel is often not pulled to the client. Sometimes making a change to the original document seems to trigger a change, and then it’s pulled to the client - but not always. If all the documents do manage to fully replicate initially, then replication works flawlessly for that channel from then on.

It would appear that adding a channel doesn’t appear to trigger any changes.

I do only have one user specified in the gateway, and I am using channels names as a type of api_key that the use can specify. I wonder if this is my problem, and that if I used a different user for each channel my problem would fix itself?

Any thoughts guys? I’m hitting my head against the wall for this one!

  • I’m using Couchbase Lite .NET, in Xamarin for Mac.
2

Make sure you are running with the latest release of Sync Gateway or building from master, there was an issue in SG 1.0.2 that had symptoms similar to this.

Andy

3

What exactly are you doing to add a new channel on the client?

4

I’m using 1.0.3. Which appears to be the latest build available. Would it be worth building from master to try and solve this?

5

When a user adds a new channel, I stop the replication, and create a new replication object. I have also tried, just stopping it, adding the channel, and starting the replication again. But that had similar issues.

I wonder if my issue is anything to do with how I am doing user management (or not doing user management). I currently have a single user, that I’ve hard-coded into my app. I am using channels to decide which “libraries” of data is synced. To add a new “library”, the user adds the api key (which is actually the channel name). This way, I’m intending for multiple users to be able to sync the same library if they have the api key.

When I start a replication, a message such as “Puller: starting ChangeTracker with since=25018” is shown to the console. So, I wonder if it’s just not syncing data before that version, because other client (with the same username) has already pulled those versions? However, the doc’s lead me to believe that I can add and remove channels as I like, and it should all just work.

Anyway, here is my replication code:

My C# code for Initial replication:

var database = Manager.SharedInstance.GetDatabase ("mydb");
var channels = new List<string>();

// I have a utility function for fetching channels, but it's effectively the same as:
channels.Add("channel1");
channels.Add("channel2");

Replication pull = null;
if (channels.Count > 0) {
    var url = new Uri ("https://mydomain.com/mydb_sync_gateway");
    // Same username & password for all users.  Could this be my problem?
    var auth = AuthenticatorFactory.CreateBasicAuthenticator ("username", "password");
    var pull = database.CreatePullReplication (url);
    pull.Authenticator = auth;
    pull.Continuous = true;
    pull.Channels = channels;
    pull.Start ();
}

Once a user has added a channel, I’m restarting my replication like this:

channels.Add("channel3");
if (pull != null)
    pull.Stop ();

if (channels.Count > 0) {
    var url = new Uri ("https://mydomain.com/mydb_sync_gateway");
    var auth = AuthenticatorFactory.CreateBasicAuthenticator ("username", "password");
    var pull = database.CreatePullReplication (url);
    pull.Authenticator = auth;
    pull.Continuous = true;
    pull.Channels = channels;
    pull.Start ();
}
6

OK, Update to my problem. Issue appears to be with versioning. If I make a minor change to every document in the new channel, then all the documents instantly since down.

I think there could be two things going on.

  • Firstly, if I’ve added a channel in the past, and then removed it, how do I get those documents back if I add the channel again.
  • Secondly, is it possible that a different lite client with the same username has pull down that channel, and so my client isn’t pulling down all the doc’s because it thinks that it has them already?

So, really, my question is turning into this:

How do I force couchbase lite to ignore all previous version information, and pull every document for that channel?

7

For the second one, I don’t think that is possible. The reason that Couchbase Lite will start midway through a replication is if the puller information matches a previous replication that has been created on the same database. Two replications will be considered equal if the all the following conditions are met:

  1. Both are running against the same database (This is judged by the local database UUID, which is generated every time a new database is created so it is unique to a specific database file. This will prevent the second scenario you mentioned)
  2. The remote URL (i.e. sync gateway URL etc) of both is the same
  3. The two do not have differing directions (i.e. both are push or both are pull)
  4. The continuous property matches
  5. The filters are the same (this include channels)
  6. The parameters for the filters are the same
  7. Both have the same set of document IDs in DocIDs

I’m not quite clear on what you mean by “how do I get those documents back?”. Unless you’ve deleted them, they will still exist in the local database after you remove the channel. A puller simply takes documents from the remote that do not already exist in the local database and inserts them. If, perhaps, you are only relying on the puller’s Changed event for information about your documents, this could explain why you are seeing the behavior that you do. Could you show how you are determining whether or not a document is available?

8

If I make a minor change to every document in the new channel, then all the documents instantly [sync] down.

That’s consistent with the client replicator mistakenly not starting over from the beginning after you changed the channels.

is it possible that a different lite client with the same username has pull down that channel, and so my client isn’t pulling down all the doc’s because it thinks that it has them already?

No. The username doesn’t matter. The client keeps track of the latest sequence (pseudo-timestamp) of document it’s pulled from the server, and when it connects it requests the ones newer than that.

(I’m curious why you’re using a shared/fixed user account rather than just guest access. Hardcoding the credentials in the app isn’t secure.)

9

Thanks guys!

Looking further into this, it appears that replication randomly stalls after I add a new channel. Once all the documents have been pulled, it appears to stop stalling.

I know this is a BIG HACK, but I’ve managed to work around this by doing a few things after adding a new channel:

  1. Delete the local database. (hopefully reseting the revision)
  2. Start a pull replication, and pull all channels again.
  3. Use a system timer to “restart” the replication if no documents appear to have been updated in 20 seconds.
  4. Once the number of documents match the number I’m expecting, turn off the timer, and use normal replication.

Yes, it’s a bit horrible… but it works! Even if it takes a few minutes of stalling, restarting, waiting, until it’s fully sync’ed. However, I’m only replicating in one direction, and adding new channels is a very very rare event in my app.

I probably could just use guest access, given my currently scenario. As long as it doesn’t let somebody get to data if they don’t know a channel name. My data does need to be secure – so I’m using channel names as authentication. Eg. If you are able to know the channel name, then you are able to access any of the data in that channel.

10

I’m not fully grasping what you mean by “stall”. Do you have a test case that you can share? If so, please file an issue on the Couchbase Lite .NET github page.