I am trying to figure out if Couchbase Lite and Capella database is the correct approach for an application.
The application is developed in Flutter.
On the user device I would like to have partial sync. Only the last month worth of data is available on the user device. Is this possible with Couchbase Lite?
Also when internet connection is available the user device needs to be able to query older data from the remote server. I did not fully understand the documentation but this seems to be possible using Sync Gateway REST API is this the correct approach? I could not find any documentation on how to query the database free tier couchbase. I am able to get documents from the Sync Gateway through REST API but how do I run a query?
Syncing partial databases is done through the channel mechanism of Sync Gateway (App Services). Channels are sort of like tags applied to documents.
One way to sync recent data would be to assign docs a channel name based on the month, like “2025_01”, and then have the client pull the channel for the desired months. The client can set expiration dates on documents as it receives them, so they’ll expire after a month.
We don’t have a mechanism to allow the client to submit queries to run server-side, although the REST API can look up documents based on ranges of keys. So if you named the docs with the date as a prefix you could use the _all_docs REST API to get a list of doc IDs in a date range.
(However, I don’t know a lot about Capella and I’m not sure it exposes the REST API to clients.)
Can you please explain the distinction between Capella and Couchbase server. On the free tier when I create a cluster as far as I understand it is running a Couchbase server that I can query using the SDK. Can I just wrap the C sdk through flutter ffi and use it? Is that available from Couchbase?
Also I noticed that as soon as I start an App Service and link a collection the collection changes on the server.
For example I am using the default travel-sample and linked the hotel collection.
Original content of hotels collection is as expected:
hotel:
address null | string
alias null | string
checkin null | string
.
.
.
After linking in the app service
hotel:
Schema 1:
metadataID string
Schema 2:
address null | string
alias null | string
checkin null | string
.
.
.
I do not understand what it means to have the two Schemas. Also before linking I can query as expected using SQL++ in the data tools/query tab. For example I can query for hotel name.
After linking app service I can only query on metadataID using the query tab so it seems I can only query Schema 1.
Is it possible for me to set up channels as you described above to automatically sync the data for the last month and use the Couchbase C SDK through ffi bindings to query any other data I want from the server using SQL++ queries?
Capella is basically a cloud-hosted Couchbase Server. (With some limitations since you don’t ‘own’ the entire server, just individual collections.)
it is running a Couchbase server that I can query using the SDK.
If you mean the Couchbase Server SDK, that is intended only for use by server-side code; in the on-prem world, code that runs in your data center. In Capella I assume it requires authorization with your Capella credentials. For security reasons it’s never accessible to the public.
Public/client devices can access Capella using Couchbase Lite and/or the App Services REST API.
I’m not sure what you mean by “schemas” on the server. The existing documents should not change. App Services does create its own metadata documents; in general you should ignore any keys beginning with _sync:.
Thanks Jens for the reply. I am fairly new to DBs and Servers and anything cloud. I figured out the Schemas issue, when you add an application service each linked collections adds a _sync:syncInfo document. Since I was querying with a limit 1 it was always only returning that document and not an actual one I was expecting.
I still cannot figure out the correct setup for an application where I don’t want one user to have access to all the data.
Here is a simplified version of the scenario:
2 users u1 and u2
Each user has 2 devices
u1: [du1_a, du1_b]
u2: [du2_a, du2_b]
I would like to achieve the following
each user has local access to their data for 1 month on all their devices
each user can query the server and get older data as needed when they are online.
u1 cannot access data from u2 locally or by querying
The problem I see with using channels is I can only have 1000 channels. What if there are more than 1000 users. Then I cannot make a channel for each user to only access their data.
My current thinking is Couchbase Lite on device with sync enabled but only with Push Replication. This will allow the users to store their own data locally and push it to the server when internet connection is available.
The main problem is I don’t understand the best path to query older data for a user from the server. Here I would like to run an SQL++ query, get the results, and display them in the app.
I understand the security issue with including the server credentials in the app but am not familiar enough with server access topologies to figure out the best process.
I found the following design reading through AWS documentation, is this the correct approach with the only change being instead of Amazon DynamoDB I would access the Couchbase Server? Also given I am currently using free tier couchbase managed cluster. Does this mean that accessing it through AWS would be slow since they are not on the same virtual network? Will I have to deploy and manage my own capella on AWS?
Huh. I am not familiar with Capella, but that seems a low limit. Is that only for the free tier? If you want each document to be accessible to only one user, you are going to need a channel per user; that’s a very common setup.
My current thinking is Couchbase Lite on device with sync enabled but only with Push Replication.
That won’t propagate data between multiple devices belonging to a user. For that you need push and pull.
In my first reply on this thread I described how to set up chronological channels, so only recent data is synced. In this case you’d need channels that combine a user ID and a month.
the best path to query older data for a user from the server. Here I would like to run an SQL++ query, get the results, and display them in the app.
You’d need to run some kind of service, like an AWS Lambda, that has the credentials to access the Couchbase Server itself (via its SDK.) Then a client can send it a request for old docs; the service would authenticate the user and generate a SQL++ query to send to Couchbase, then return the results to the client. This shouldn’t be too slow.
