Replicate only a part of the data

Mobile Sync Gateway
1

I have a couchbase server with data migrated from another data store. This data needs to be synced with mobile apps. How would I do this?
For clarity, I will use an example. I have a messages bucket on couchbase server. Each document in the bucket has some data plus a property userId. When a user is logged in to the app, their token has the userId. How can I make it that a user only replicates the data that they own (the messages that have their id’s)

2

Hi,

Can you tell us the tech you’re using ? The versions too.

Regards.

Steeve

3

You will need the Sync Gateway component fir syncing docs to mobile clients
How you would do it depends on version of Couchbase Mobile that you are using. If you are on Couchbase Mobile 1.5, you will follow the steps outlined here to get shared bucket access.

Pre-1.5, it’s trickier . You have the option of uploading all documents via the SGW’s REST API so the appropriate sync metadata gets added to it or you use shadow buckets. I would discourage the use of shadow buckets if you are starting off since its deprecated.

This is specified using the sync function that you will specify in your SGW config file. The doc has several examples that cover your scenario. At high level, every user has his own “channel” and docs for user are assigned to user specific channel. The sync gateway takes care of routing the docs and access control .
On client side, you can filter the documents that are pulled by specifying the channel.

4

hi.
I’m using Couchbase 5.0.0, Sync Gateway 1.5.0 and couchbase lite on the mobile apps.

5

I kinda understand using the Sync gateway from a really high level, I’ve even managed to sync data to and from the server/apps using the gateway. The biggest challenge i have is this. The sync gateway is already running and the sync function is already being run for each record. How then do I control or route that data to go to the correct user using a specific property?

6

What version of the platform are you running ?

Just so I follow your question - Are you saying that the documents have already been processed by the sync gateway and are now associated with a default channel that all clients have access to. And now you want to re-route the documents based on userId property?

7

I’m using Couchbase 5.0.0, Sync Gateway 1.5.0 and couchbase lite on the mobile apps.

So I guess that is another level of complexity added now :slight_smile: yes the documents have been processed by the gateway and now are associated with a default channel.

If we can kindly ignore that fact for now, what would be the best (and correct even) way to handle this use-case I have?

8

You will have to update the sync function to route documents based on the property. Assume you are familiar with writing sync functions else please check out this doc link that I shared earlier.

Once you update your sync function, restart sync gateway and do a resync so documents get re-processed. You can read how to do that in this guide

9

Okay thanks. Last question, I still can’t wrap my head around this: its the app that knows the id which is to be used in the replication process. Do I still write the sync function without any additional config on the app?

10

The sync function would the source of truth in terms of what gets routed to what channel. So your sync function will assign the docs to channels and the client can filter documents that get replicated based on the channel.

Trivially, you would do something like this (assuming userId is a property in the doc)

if (doc.userId) {
      channel ("channel" + doc.userId);
   }

So all documents for a given userId are assigned to a channel for that user .

Again, I would recommend reviewing the documentation that i shared in earlier links. That explains the objective of the sync function and routing and so on.

11

thank you very much. Let me try this out

12

I created the sync function like this:

function(doc, oldDoc) {
 if (doc.formedById) {
   channel (doc.formedById);
  } else {
   channel ("blank");
  }
 }

this syncs and takes way too long. I have 48000 docs and I presume this is why it takes too long. The question I have is how can I access the channels on the gateway. Accessing http://localhost:4985/_admin/db shows the db name but when I open it, chrome becomes unresponsive

13

The initial sync may take some time. How long did it take ? Subsequently, only changes are processed.

The Admin UI for Sync Gateway is defunct for now (well - it was never quite officially supported). AFAIK, there is currently no way to get just channel info but you could use the _raw REST API on SGW to get details of the documents including channel info associated with the document.

14

Sorry its actually 480,000. Took a little over an hour but that’s to be expected I guess. Thanks for the info on the sync gateway. I guess I’ll do the tests on the app to see if everything is ‘synced’ correctly

15

Yeah- that sounds quite reasonable for an initial sync (or re-sync) of the bucket by the SGW. Sync times to the device itself depends on factors like network , document size and such.

16

There isn’t a way to access all the channels directly. You can, though, get information about channels a user has access to. Take a look at /{db}/_user/{name} under https://developer.couchbase.com/documentation/mobile/1.5/references/sync-gateway/admin-rest-api/index.html#/user

So to see all relevant channels you could iterate over users and list out the channels they have access to.

The admin console will show you channels as well. However, it does it by literally pulling every change in the history of the database and running documents through its own copy of the sync function. You can see the code for this at https://github.com/couchbaselabs/sync_gateway_admin_ui/blob/master/src/js/syncModel.js

17

Thank you all for your help. Everything is working well

1 Like