Replication not working as expected with CBL and Couchbase Server

[ikiugualf]

I have data in couchbase server. The data in this case is a list of products, the bucket name is products as well. On a mobile app (Xamarin) , a user on first launch gets all the documents in the products bucket and they are saved on couchbase lite on a products database. This works perfectly fine but when I create totally different data - a user profile to be stored on the device, something weird happens. The user data is stored but also on couchbase server all the documents in the products bucket now have all the data that is the user data. My questions are the following:

• How do I stop the user data from replicating on Couchbase server? Especially to all the other documents. Is this because the only replicator set up points to the products bucket?
• I have set push.continuous to be false but the user info still gets synced to Couchbase, does this have no effect on the data getting replicated both directions?
• If I need to replicate the user data from Couchbase Lite to Couchbase server, do I create a new bucket for this and do I create new replicator objects with different URLS?

For reference, here is my sync gateway code…I am not sure if the issue is at this point.

{
	"log": [
		"HTTP+"
	],
	"adminInterface": "127.0.0.1:4985",
	"interface": "0.0.0.0:4984",
	"databases": {
		"products": {
			"server": "http://127.0.0.1:8091/",
			"bucket": "products",
            "import_docs": "continuous",
            "username": "my-user",
            "password": "my-password",
            "enable_shared_bucket_access": true,
		}
	}
}

[priya.rajagopal]

@ikiugualf

How do I stop the user data from replicating on Couchbase server? Especially to all the other documents. Is this because the only replicator set up points to the products bucket?

Which version of Couchbase Lite are you using ? You can filter by documentIds or set up push filters on replicator which determines what gets pushed up to server. It looks like you have already posted this question here which was responded to

@ikiugualf

I have set push.continuous to be false but the user info still gets synced to Couchbase, does this have no effect on the data getting replicated both directions?

Push mode of continuous does not impact what gets synced. It determines if replication happens continuously (i.e. any time changes happen on either end subject to network connectivity ) or if it is one shot (syncs the changes and closes the replicator)
The direction of replication is determined using config.replicatorType (Push or Pull or PushAndPull)

@ikiugualf

If I need to replicate the user data from Couchbase Lite to Couchbase server, do I create a new bucket for this and do I create new replicator objects with different URLS?

You don’t have to do that. Unless you want the data synced up (pushed) from the clients to be stored in a bucket different from the rest of the server side documents.

[ikiugualf]

Thank you @priya.rajagopal for the quick and detailed response…I actually wanted to use this as the more contextual question instead of the other one.

@priya.rajagopal

You don’t have to do that. Unless you want the data synced up (pushed) from the clients to be stored in a bucket different from the rest of the server side documents.

For clarification on this, I come from an SQL background and best practice would be to have different tables with each holding specific data for example a products table and a users table.
For Couchbase, what would be the rationale? Use one bucket per application and use a type (or any other property) to differentiate users and products for example?
What would be the implications performance wise and is there any limit for the number of documents per bucket?
The reason I ask this is that I have around 7 different sets of data which in SQL would be 7 different tables. In Couchbase, do I use one bucket for all?

[jens]

Buckets are entirely independent and you can’t, for example, query across buckets.

Most cases where you’d use multiple tables in SQL are done using a property like type that can distinguish different types of documents.

[ikiugualf]

Hi @jens
Yes I noticed buckets are independent of each other but if I did not need cross bucket queries would it be possible to run several buckets or I’d end up with a mess on the mobile app?

[jens]

“bucket” is a Couchbase Server term — on client they’re called databases.

It sounds like you’re asking this question again … or is it something different?

[ikiugualf]

It’s something different but I also got the answer. If I should use multiple buckets or a single one. I guess the answer is I should use one

[priya.rajagopal]

Yes. A single bucket should be used to hold all documents related to an application. As mentioned by Jens, this is equivalent to a database.
A table in SQL would be equivalent to a logical group of Couchbase Documents. Very loosely, every row in your SQL table would be equivalent to a Document. The documents can be logically grouped using a “type” property . You may want to check out this doc on the basics of Data Modeling.

[natgross]

@priya.rajagopal

A single bucket should be used to hold all documents related to an application. As mentioned by Jens, this is equivalent to a database.

The Couchbase advise on (these) issues changes as newer [server] versions are introduced. Couchbase videos/lectures/webinars on YouTube (over the years) actually advise different methodologies, both in design, query, and coding. For example, once upon a time it was great to have all your [related] data in the same document. Now, it is advised to store just the related id’s, like in a normalized sql db. Of course this is due to new features for the server (like n1ql, which we had on Oracle since 1990). No big deal, just saying… maybe someone should remove the old videos!

Now, with the bucket situation, they (server team) clearly green-light using multiple buckets for multiple tables. Are you saying that when it comes to mobile and sync, we are better of with one bucket on the server due to mobile apps?

[natgross]

@jens

Buckets are entirely independent and you can’t, for example, query across buckets.

Can’t query across buckets??

Querying documents from 2 different buckets (HELP)

I am trying to retrieve data from 2 buckets, no error but nothing shows up (I do have documents I need in these buckets). 1st bucket: a_bucket here is the document I am interested in (I do have 3 different docs) author_ID document: { “author_ID”: 1, “profil_creation_date”: “2017/01/01/01:23:05/+5”, “prefix”: “Mr.”, “first_name”: “Dylan”, “middle_name_s”: “Alfred”, “last_name”: “Kerr”, “date_of_birth”: “1974/01/02”, “sex”: “M”, “marital_status”: “Single”, “mobile_phone”: “(860) 231-…

Please clarify.
Thanks
-nat

[priya.rajagopal]

@natgross

Now, with the bucket situation, they (server team) clearly green-light using multiple buckets for multiple tables. Are you saying that when it comes to mobile and sync, we are better of with one bucket on the server due to mobile apps?

Where do you see such a recommendation? Again, a couchbase server bucket is analogous to a database in SQL world. You don’t create a database for every table, do you ?

[natgross]

@priya.rajagopal

Where do you see such a recommendation?

Well, I’d have to re-watch webinars posted in the last 18 months. But here it is in server documentation, last portion of the page, sub-titled Buckets. Couchbase SDKs
Again, I have definitely heard/watched cb [server] folks mention the benefits of multiple buckets in a variety of use-cases.
The one thing that bothers me about a single bucket, is that it’s really one huge table (in sql terms) with a type differentiator, causing extra indexes and index segments.

@priya.rajagopal

Again, a couchbase server bucket is analogous to a database in SQL world. You don’t create a database for every table, do you ?

True. But Oracle and IBM db2 (as well as others) allow me to (flip the script) place tables into ‘buckets’ (using cb terminology) as I wish, and then assign different rules per bucket for replication and exact placement on physical drives, among other things.

It might sound like that I am advocating sql over cb. I am not. If I would, I wouldn’t be here. But it is natural to compare with past experience.

All the best,
-nat

[priya.rajagopal]

Yes- the number of buckets is an application level concern that depends on the use case. So it’s definitely not the case of

@natgross

they (server team) clearly green-light using multiple buckets for multiple tables.

Security, multi-tenancy are typically the reasons to have multiple buckets . That said, one should be wary of the number of buckets - there are performance implications to having too many of them.

You may be interested in this post that advocates against creating too many buckets. This is from our own server team. It also provides some relevant links that would be good reads.

So specifically in the use case discussed in the original question there is no reason to have more than one bucket.

@natgross

It might sound like that I am advocating sql over cb. I am not. If I would, I wouldn’t be here. But it is natural to compare with past experience.

The choice of database comes down to the application needs. NoSql is best suited for certain kinds of applications and Sql for others. There is also a paradigm shift in the approach to data modeling and data management - all of which is discussed in our Couchbase server docs…

[jens]

@natgross

Can’t query across buckets??

Sorry, I was referring to mobile databases, not server buckets.