Sync Gateway via the Command Line

Introduction

This continues my my “Getting Comfortable with Couchbase Mobile” series (post #5). This time we’ll look at exploring Sync Gateway from the command line. This can be really helpful, both for understanding and for testing/debugging. (Links to other posts in the series provided at the end.)

Background

To get comfortable with the whole Couchbase Mobile stack, it’s great to install and run everything on one machine. In this series of posts, I’ll walk through the steps get to started with each component. I’ll show how to do a little extra exploring along the way, too.

I’ll only do minimal configuration. This is not intended to explain what you need for a production environment. I assume you’re familiar with some basics of NoSQL, have some understanding of Couchbase, and know how to build apps in something like Java, Android, or iOS. If you want to read up on NoSQL databases or Couchbase, you can find lots of resources on the Couchbase site.

Couchbase is open-source. Everything I’ll use here is free to try out. See the end of the post for more resources if you need help.

Sync Gateway

Sync Gateway is a secure web gateway application with synchronization, REST, stream, batch and event APIs for accessing and synchronizing data over the web. Sync Gateway enables, among other things, secure data replication between Couchbase Server and Couchbase Lite.

Sync Gateway has few dependencies, and can run on most Linux distributions (even on the Raspberry Pi), Windows, and OSX/macOS. The specific steps for installation vary with platform. See the downloads site for all the available packages. To install on Linux distributions other than the supported ones, see this post.

Configuration and running

Once you have Sync Gateway installed, open up a command line shell.

Next, I have two configurations listed. You can either find these in the examples folder of your installation, or copy and paste the text here into your own.

The first you can use if you have Couchbase Server running on your machine. This is the configuration you’ll find in examples/basic-couchbase-bucket.json.

{
  "log": ["*"],
  "databases": {
    "db": {
      "server": "http://localhost:8091",
      "bucket": "default",
      "users": { "GUEST": { "disabled": false, "admin_channels": ["*"] } }
    }
  }
}

{

"log": ["*"],

"databases": {

"db": {

"server": "http://localhost:8091",

"bucket": "default",

"users": { "GUEST": { "disabled": false, "admin_channels": ["*"] } }

}

Or to just use Sync Gateway stand-alone (usually for testing only), use this configuration (found in examples/basic-walrus-bucket.json):

{
  "log": ["*"],
  "databases": {
    "db": {
      "server": "walrus:",
      "users": { "GUEST": { "disabled": false, "admin_channels": ["*"] } },
      "allow_empty_password": true
    }
  }
}

{

"log": ["*"],

"databases": {

"db": {

"server": "walrus:",

"users": { "GUEST": { "disabled": false, "admin_channels": ["*"] } },

"allow_empty_password": true

}

Creating a document

From the command line, run Sync Gateway and supply the path to the configuration file.

$ /path/to/sync_gateway your_config.json

1	$ /path/to/sync_gateway your_config.json

At this point the special GUEST user is enabled. We can go ahead and create a document.

$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "test": "entry" }' localhost:4984/db/a_doc

1	$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "test": "entry" }' localhost:4984/db/a_doc

The response shows the write succeeded. You can also see this in the Sync Gateway log output, and by looking at the backing bucket in the Couchbase Server Web Console.

Here’s the response we see on the command line.

{"id":"a_doc","ok":true,"rev":"1-93996db139ad4024d0c7c982d4f69122"}

1	{"id":"a_doc","ok":true,"rev":"1-93996db139ad4024d0c7c982d4f69122"}

Updating a document

Next, let’s try to make a change in the document.

$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "test": "changed" }' localhost:4984/db/a_doc

1	$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "test": "changed" }' localhost:4984/db/a_doc

The output shows this failed.

{"error":"conflict","reason":"Document exists"}

1	{"error":"conflict","reason":"Document exists"}

Why? If you recall from other parts of this blog series, Couchbase Mobile uses something called Multiversion Concurrency Control to track revisions. That means when we make changes to a document, we have to indicate what the “parent” document is.

Let’s try again, this time specifying the parent revision.

$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "test": "changed" }' 'localhost:4984/db/a_doc?rev=1-93996db139ad4024d0c7c982d4f69122'

1	$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "test": "changed" }' 'localhost:4984/db/a_doc?rev=1-93996db139ad4024d0c7c982d4f69122'

That works. The response is

{"id":"a_doc","ok":true,"rev":"2-8d7051e4510c47c5e0aa74aed1fbd788"}

1	{"id":"a_doc","ok":true,"rev":"2-8d7051e4510c47c5e0aa74aed1fbd788"}

Monitoring the changes feed

The changes feed gives you a great way to track the database. You can use this to write event-driven server side logic, or implement an automated conflict resolution system.

We’ll show a version that keeps the feed open. You might want to open another command line window to watch the output.

$ curl -X GET 'localhost:4984/db/_changes?feed=continuous&heartbeat=26000&since=0'

1	$ curl -X GET 'localhost:4984/db/_changes?feed=continuous&heartbeat=26000&since=0'

Here we’ve set the feed type to continuous. Various elements along a network path may close a connection that appears idle. Sync Gateway can send a heartbeat to prevent that. We set a timeout of 26000 milliseconds.

The last parameter, since, tells Sync Gateway where we want to start in the overall sequence of changes. You should normally treat this as an opaque value. Typically an app will start without specifying this parameter (getting all changes), then track the value sent back to use in future requests.

Here’s what a set of changes looks like.

{"seq":1,"id":"_user/","changes":[{"rev":""}]}
{"seq":3,"id":"a_doc","changes":[{"rev":"2-8d7051e4510c47c5e0aa74aed1fbd788"}]}

1 2	{"seq":1,"id":"_user/","changes":[{"rev":""}]} {"seq":3,"id":"a_doc","changes":[{"rev":"2-8d7051e4510c47c5e0aa74aed1fbd788"}]}

Adding a user

Now, let’s add a new user, jdoe. This is an admin function, so the request has to go through the admin port. (Port 4985 if you’re using one of the listed configurations.)

$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "password": "pass", "admin_channels": [ "ch9" ] }' localhost:4985/db/_user/jdoe

1	$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "password": "pass", "admin_channels": [ "ch9" ] }' localhost:4985/db/_user/jdoe

We’ll also disable anonymous (GUEST) access.

$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "disabled": true }' localhost:4985/db/_user/GUEST

1	$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "disabled": true }' localhost:4985/db/_user/GUEST

We can no longer access Sync Gateway from the public port (4984) without supplying a username and password.

Channels

Let’s add a doc with a special field, channels.

$ curl -s -g -u 'jdoe:pass' -X PUT -H 'Content-Type: application/json' -d '{ "my": "doc", "channels":"abc" }' localhost:4984/db/jd_doc

1	$ curl -s -g -u 'jdoe:pass' -X PUT -H 'Content-Type: application/json' -d '{ "my": "doc", "channels":"abc" }' localhost:4984/db/jd_doc

You should see this returned.

{"id":"jd_doc","ok":true,"rev":"1-4407b06b44eccdf1a9ef8cbe6f5dbe7f"}

1	{"id":"jd_doc","ok":true,"rev":"1-4407b06b44eccdf1a9ef8cbe6f5dbe7f"}

Sync Gateway passes every document through a special function, called the sync function, whenever a document gets updated. We didn’t specify a sync function, so the default is automatically used.

The default function looks for channels in the document properties. If found, the document is assigned to that channel.

A user must have access to a channel to receive a document assigned to that channel. You can read more about channels here. We’ll see the impact coming up.

Just to see what happens when we don’t authenticate, try this command. The request gets rejected

$ curl -X GET localhost:4984/db/jd_doc

1	$ curl -X GET localhost:4984/db/jd_doc

with the expected response.

{"error":"Unauthorized","reason":"Login required"}

1	{"error":"Unauthorized","reason":"Login required"}

How about if we try again, supplying the right username and password?

$ curl -u 'jdoe:pass' -X GET localhost:4984/db/jd_doc

1	$ curl -u 'jdoe:pass' -X GET localhost:4984/db/jd_doc

This sometimes catches people by surprise. We get rejected again, but for a different reason.

{"error":"Forbidden","reason":"forbidden"}

1	{"error":"Forbidden","reason":"forbidden"}

What happened? Remember when we wrote the doc, we assigned it to channel abc. Even though user jdoe wrote the doc, we haven’t given access to the abc channel. Without that, jdoe isn’t allowed to read the doc back.

Change the channels jdoe has access to

$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "admin_channels": [ "abc" ] }' localhost:4985/db/_user/jdoe

1	$ curl -s -g -X PUT -H 'Content-Type: application/json' -d '{ "admin_channels": [ "abc" ] }' localhost:4985/db/_user/jdoe

and see that the user can now access the doc.

$ curl -u 'jdoe:pass' -X GET localhost:4984/db/jd_doc

1	$ curl -u 'jdoe:pass' -X GET localhost:4984/db/jd_doc

Success!

{"_id":"jd_doc","_rev":"1-4407b06b44eccdf1a9ef8cbe6f5dbe7f","channels":"abc","my":"doc"}

1	{"_id":"jd_doc","_rev":"1-4407b06b44eccdf1a9ef8cbe6f5dbe7f","channels":"abc","my":"doc"}

Wrapping up

Hopefully that gives some insight into Sync Gateway. I encourage you to check out the REST API further. You can find more interesting information looking at the Sync Gateway built-in web admin utility (http://localhost:4985/_admin/) and the Couchbase Server Web Console.

For an interesting example of using Sync Gateway stand-alone, see our talk from CouchBase Connect 2016: https://www.youtube.com/watch?v=6Ra4NeY7TNA

[buttongroup][button style=”btn-link btn-lg” icon=”fa fa-arrow-left” align=”left” iconcolor=”#dd3333″ type=”link” target=”false” title=”Previous: Installing Sync Gateway” link=”https://www.couchbase.com/blog/getting-comfortable-couchbase-mobile-installing-sync-gateway/” linkrel=””][button style=”btn-link btn-lg” icon=”fa fa-arrow-right” align=”left” iconcolor=”#dd3333″ type=”link” target=”false” title=”Next: TBD” link=”” linkrel=””][/buttongroup]

Postscript

Check out more resources on our developer portal and follow us on Twitter @CouchbaseDev.

You can post questions on our forums. And we actively participate on Stack Overflow.

You can follow me personally at @HodGreeley

Hod Greeley, Developer Advocate, Couchbase

2 Comments

Couchbase Mobile Changes Explorer - Part. 1 - The Couchbase Blog February 28, 2017 at 3:58 pm

[…] can try out. You can also use command line tools and the Sync Gateway admin interface. Take a look here for more on […]

Log in to Reply
Getting Comfortable with Couchbase Mobile: Installing Sync Gateway - The Couchbase Blog April 5, 2017 at 8:55 pm

[…] Previous: Couchbase Server via the Command Line Next: Sync Gateway via the Command Line […]

Log in to Reply

Platform

Self-Managed

Services

Capabilities

Why Couchbase?

Migrate to Capella

By Use Case

By Industry

By Application Need

Popular Docs

By Developer Role

COMMUNITY

Join the Developer Community

Resource Center

Education

Compare

About

Partnerships

Our Services

Partners: Register a Deal

Ready to register a deal with Couchbase?

Marriott

All Posts

Getting Comfortable with Couchbase Mobile: Sync Gateway via the Command Line