Sync Gateway Not Starting

Mobile Sync Gateway
1

We have Couchbase with 3 data nodes, 1 query node and 2 index node.

we have created couchbase bucket and using following sync gateway configuration

{
“log”: [
“"
],
“databases”: {
“bucketname”: {
“server”: “data.couchbase-flex.l4lb.thisdcos.directory:8091”,
“bucket”: “bucketname”,
“username”: “bucketname”,
“password”: “123456”,
“enable_shared_bucket_access”: true,
“import_docs”: “continuous”,
“num_index_replicas”: 1,
“users”: {
“GUEST”: {
“disabled”: true,
“admin_channels”: [
"”
]
},
“flex”: {
“password”: “123456”,
“admin_channels”: [
“"
]
}
},
“sync”: function(doc, oldDoc) { if (doc.taskType!='Load' && doc.taskType!='Unload' ) { channel(doc.clientId+'-'+doc.terminalId); }else{ channel('Invalid'); } },
“rev_cache_size”: 0
}
},
“CORS”: {
“Origin”: ["”],
“LoginOrigin”: ["*"],
“Headers”: [“Content-Type”],
“MaxAge”: 17280000
}
}

but we are getting following error

2020-02-17T15:24:14.297Z [INF] Set query timeouts for bucket flexoffline to cluster:1m15s, bucket:1m15s
2020-02-17T15:24:14.297Z [INF] Initializing indexes with numReplicas: 1…
2020-02-17T15:24:16.896Z [INF] Verifying index availability for bucket flexoffline…
2020-02-17T15:25:31.896Z [INF] Timeout waiting for index “sg_channels_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:25:31.897Z [INF] Timeout waiting for index “sg_roleAccess_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:25:31.897Z [INF] Timeout waiting for index “sg_access_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:26:46.897Z [INF] Timeout waiting for index “sg_channels_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:26:46.897Z [INF] Timeout waiting for index “sg_access_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:26:46.898Z [INF] Timeout waiting for index “sg_roleAccess_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:28:01.897Z [INF] Timeout waiting for index “sg_channels_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:28:01.898Z [INF] Timeout waiting for index “sg_roleAccess_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:28:01.898Z [INF] Timeout waiting for index “sg_access_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:29:16.898Z [INF] Timeout waiting for index “sg_channels_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:29:16.898Z [INF] Timeout waiting for index “sg_access_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:29:16.898Z [INF] Timeout waiting for index “sg_roleAccess_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:30:31.898Z [INF] Timeout waiting for index “sg_channels_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:30:31.899Z [INF] Timeout waiting for index “sg_roleAccess_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:30:31.899Z [INF] Timeout waiting for index “sg_access_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:31:46.898Z [INF] Timeout waiting for index “sg_channels_x1” to be ready for bucket “flexoffline” - retrying…
2020-02-17T15:31:46.899Z [INF] Timeout waiting for index “sg_access_x1” to be ready for bucket “flexoffline” - retrying…

2

Hi @chvishwakarma,

Are you able to verify that the indexes are created and online in Couchbase Server through the UI?
If they are not online, you may be able to determine what’s happening by looking at the indexer logs:
/opt/couchbase/var/lib/couchbase/logs/indexer.log

If they appear to be online, the issue may be that Sync Gateway cannot connect to the query service correctly.

3

@bbrks yes indexes are being created in ADMIN UI, however for index nodes and query nodes I see
disk used 0 %. is that okay ?

4

Also we are using DC/OS for couchbase server

5

Do they all have status:ready?

E.g

Screenshot 2020-02-17 at 16.33.13
Screenshot 2020-02-17 at 16.33.131137×316 28.7 KB

7

I see from your screenshot you’re using Couchbase Server version before 6.5.0. These older versions show index build status at 100% when the index is ready and online.

At this point, it looks like Sync Gateway cannot connect to the query service correctly.
Make sure your query ports (8093 and 18093) are accessible from the machine running Sync Gateway.

You can verify this by issuing a query from the Sync Gateway machine using curl:

 curl -kv  https://cbsusername:cbspassword@cbs-address:18093/query/service -H "Content-Type: application/json"  -d '{"statement": "SELECT * FROM bucketname"}'
8

image
image1219×264 15.3 KB

9

@bbrks we have tried CURL command following are the out put

About to connect() to host port 8091 (#0)

  • Trying IP…
  • Connected to host IP) port 8091 (#0)
  • Server auth using Basic with user ‘Administrator’

POST /query/service HTTP/1.1
Authorization: Basic QWRtaW5pdfsfc3RyYXRvcjpwYfsdfXNzd29yZA==
User-Agent: curl/7.29.0
Host: host:8091
Accept: /
Content
-Type: application/json
Content-Length: 42
Content-Type: application/x-www-form-urlencoded

  • upload completely sent off: 42 out of 42 bytes
    < HTTP/1.1 400 Bad Request
    < Server: MochiWeb/1.0 (Any of you quaids got a smint?)
    < Date: Tue, 18 Feb 2020 04:02:50 GMT
    < Content-Length: 0

this output is for port 8091 and on port 18093/8093 we are getting connection refused.

We have out couchbase on DC/OS and it is configured with PORT 8093

10

Couchbase exposes many different ports. 8091 is only one of them. 8093/10893 is the query port.

You’ll have to open all ports listed under “Client-to-node” in the following table.

https://docs.couchbase.com/server/current/install/install-ports.html#ports-listed-by-communication-path

11

@bbrks Thanks for the quick response. I will check with DevOps team and confirm.