The Sync Gateway 3.0 release introduces a modular, cluster-aware approach to Sync Gateway configuration that supersedes the node-level JSON configuration file based approach from earlier versions. In this post, we will take a deeper look at the new approach to Sync Gateway configuration and provide context around the motivation behind the enhancement. For details, refer to the Sync Gateway documentation.
Configuration Workflow
We begin with a discussion of the logical stages in setting up and configuring a Sync Gateway node as shown in this illustration:
Bootstrap: The first step is to bootstrap the Sync Gateway to connect to the Couchbase Server cluster with appropriate authentication credentials.
Database Configuration: Once the Sync Gateway is up and running, the next step is to set up the Sync Gateway database. This is where one would provide the database name, the backing Couchbase Server bucket, configure cache management policies etc.
Access Control: Once the database is configured, the access control policies that control the read and write access to documents is defined via the Sync Function.
User Management: Finally, sync gateway users are created/registered with Sync Gateway via a suitable authentication scheme. These are the clients that sync data or access data via the Sync Gateway public REST endpoint.
Starting 3.0, Sync Gateway is bootstrapped via a JSON formatted configuration file. Subsequently, the rest of the configuration is handled via a RESTful interface.
Static File-Based Configuration (Legacy)
For context, before we dig deeper into how we configure the Sync Gateway in 3.0, let’s take a look at the pre–3.0 way of doing things. This configuration model continues to be supported in 3.0 and can be enabled via a configuration option that disables persistent config based approach.
In the legacy model, every node is independently configured with a JSON configuration file that is scoped to a Sync Gateway node. The config file includes bootstrap settings, database-level config, access control policies and more. The only exceptions are users and inter-Sync Gateway replications which can also be managed via the users and replications REST endpoints respectively.Â
Anytime the database config needs to be updated, the config file needs to be updated on every sync gateway node. After updating the file, the Sync Gateway nodes need to be restarted in order for the updated config to take effect.
Centralized Persistent Configuration of Sync GatewayÂ
Let’s take a look at the centralized persistent configuration Referring to the Configuration workflow model described earlier:
Bootstrap
In the centralized persistent config model, every Sync Gateway node is configured with a static JSON formatted bootstrap config file that includes minimal information that will allow the Sync Gateway to securely connect to the Couchbase Server cluster.Â
Any change made to the config file will require Sync Gateway restart to take effect.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
{ Â "bootstrap": { Â Â Â "group_id": "group1", Â Â Â "server": "couchbases://cb-server", Â Â Â "username": "Administrator", Â Â Â "password": "password" Â }, Â "logging": { Â Â Â "console": { Â Â Â Â Â "log_level": "trace", Â Â Â Â Â "log_keys": ["*"] Â Â Â } Â } } |
Database Configuration
Sync Gateway databases can be created and managed via the Database Management Admin endpoint. The config made through the REST API is persisted in the corresponding Couchbase Server bucket and automatically propagated to the other Sync Gateway nodes in the cluster.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
curl --location --request PUT 'localhost:4985/travel-sample/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic c2d3X2NsdXN0ZXI6cGFzc3dvcmQ=' \ --data-raw '{     "bucket": "travel-sample",     "name": "travel-sample",     "guest": {         "disabled": true     },     "import_docs": true,     "num_index_replicas": 0,     "enable_shared_bucket_access": true }' |
Access Control
Similarly, the Javascript Sync Function, that defines the read/write access control policies, can be configured via REST endpoint. It can be handled as part of the database configuration endpoint or via the sync endpoint.
|
1 2 3 4 5 |
curl --location --request PUT \ 'localhost:4985/travel-sample/_config/sync' \ --header 'Content-Type: application/javascript' \ --header 'Authorization: Basic c2d3X2FkbWluOnBhc3N3b3Jk'\ --data-raw 'function(doc, oldDoc) {if (doc.type){ channel("channel:"+ doc.type);}}' |
User Management
Users are configured via the user creation endpoint.
|
1 2 3 4 5 6 7 8 9 10 |
curl --location --request POST 'localhost:4985/travel-sample/_user/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic c2d3X2FkbWluOnBhc3N3b3Jk' \ --data-raw '{ Â Â Â Â "name": "demo", Â Â Â Â "password": "password", Â Â Â Â "admin_channels": ["*"], Â Â Â Â "email": "demo@example.com", Â Â Â Â "disabled": false }' |
All config changes made through the REST API are persisted in the corresponding Couchbase Server buckets and survives any Sync Gateway restarts.
Scoping Database Configuration
The database config is applied to one Sync Gateway node and automatically propagated to all nodes in the cluster.
Now, what if you don’t want all the Sync Gateway nodes to share the same database config. Introducing the concept of database config groups. At a high level, a database config group specifies the subset of sync gateway nodes to which a database config will apply. By default, all the nodes belong to a default config group and users can define config groups to narrow the scope of the database config.
For example, consider a five node Sync Gateway cluster wherein three of the nodes are configured for Couchbase Lite client replications and the remaining two are configured for inter-Sync Gateway replications. In this case, the three nodes can be configured to belong to the default group where the remaining two nodes are configured to belong to a config group named ISGR.
You can learn more about config groups in our documentation pages.
Summary
Here are the salient features of the configuration model:
Secure Remote Administration: Securely administer Sync Gateway clusters deployed in the cloud or edge data centers.
Modular: Dynamically configure and manage the various aspects of database configuration in a modular fashion.
Cluster-Aware: Database configuration changes made against one Sync Gateway node are automatically propagated to all or relevant subset of Sync Gateway nodes in the cluster.
Persistence: All database changes made through the REST endpoint are persisted and survive restarts.
Next Steps
Check out the video session that discusses the configuration approach and shows a demonstration.
Download Sync Gateway and try it for free. The documentation pages are a good resource for more information.
If you have questions or feedback, please leave a comment below or feel free to reach out to me via Twitter or email me.
Acknowledgements
I would like to thank Ben Brooks, Engineer on the Sync Gateway team, for his input on this blog post.
