Couchbase Server 5.0 + SG 1.5 fresh install causes 'TAP connection lost'

gtur · June 27, 2017, 10:41pm

Hello,

I am trying out the new combination of Couchbase Server 5.0 Beta and Sync Gateway 1.5, with CBLite 2.0 to follow.

Couchbase Server 5.0 Beta was successul. I added travel-sample, and then a user that has full access to that bucket. I added this user to the new sync gateway config file:

{
        "log": ["HTTP+"],
        "adminInterface": "127.0.0.1:4985",
        "interface": "0.0.0.0:4984",
        "databases": {
                "db": {
                        "server":"http://localhost:8091",
                        "username":"***",
                        "password":"***",
                        "bucket":"travel-sample",
                        "users": {
                                "GUEST": {"disabled": false, "admin_channels": ["*"] }
                        }
                }
        }
}

The sync gateway starts up with the admin port working, but no visibility into the documents in the buckets. The message in the start up logs look like this:

$ sync_gateway /Users/sync_gateway/sync_gateway.json

2017-06-26T16:39:30.187-07:00 Enabling logging: [HTTP+]
2017-06-26T16:39:30.187-07:00 ==== Couchbase Sync Gateway/1.1.0(28;86f028c) ====
2017-06-26T16:39:30.187-07:00 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this
2017-06-26T16:39:30.187-07:00 Opening db /db as bucket "travel-sample", pool "default", server <http://localhost:8091>
2017-06-26T16:39:30.187-07:00 Opening Couchbase database travel-sample on <http://localhost:8091> as user "travel-sample-gw"
2017/06/26 16:39:30  Trying with selected node 0
2017/06/26 16:39:30  Trying with selected node 0
2017-06-26T16:39:30.421-07:00 Using default sync function 'channel(doc.channels)' for database "db"
2017/06/26 16:39:30 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017-06-26T16:39:30.446-07:00     Reset guest user to config
2017-06-26T16:39:30.446-07:00 Starting admin server on 127.0.0.1:4985
2017-06-26T16:39:30.449-07:00 Starting server on 0.0.0.0:4984 ...
2017/06/26 16:39:31 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/26 16:39:32 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/26 16:39:33 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/26 16:39:34 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/26 16:39:35 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/26 16:39:36 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s

* another 2 dozen times! *

2017/06/26 16:40:24 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017-06-27T08:50:33.248-07:00 HTTP:  #001: GET /db/_config  (ADMIN)
2017-06-27T08:50:33.248-07:00 HTTP+: #001:     --> 200   (0.1 ms)
2017-06-27T08:50:33.253-07:00 HTTP:  #002: GET /db/_changes?limit=10&include_docs=true  (ADMIN)
2017-06-27T08:50:33.274-07:00 HTTP+: #002:     --> 200 OK  (0.0 ms)
2017-06-27T08:50:33.276-07:00 HTTP:  #003: GET /db/_all_docs  (ADMIN)
2017-06-27T08:50:33.277-07:00 HTTP:  #004: GET /db/_changes?since=0&include_docs=true&feed=longpoll  (ADMIN)
2017-06-27T08:50:33.277-07:00 HTTP:  #005: GET /db/_changes?since=0&include_docs=true&limit=10  (ADMIN)
2017-06-27T08:50:33.278-07:00 HTTP+: #005:     --> 200 OK  (0.0 ms)

I see three _sync entries in the bucket, in addition to travel-sample docs:

_sync:seq, _sync:syncdata, _sync:user

What do I do next so that SG can see all the docs in CB?

Thank you!

priya.rajagopal · June 27, 2017, 10:55pm

What do you see when you query for the documents via the Sync Gateway REST API? You can use curl , postman or any http client for testing this

gtur · June 28, 2017, 12:04am

Doing a GET on http://127.0.0.1:4984/db/_all_docs

results in

{"rows":[
],
"total_rows":0,"update_seq":1}

Then asking for a specific doc http://127.0.0.1:4984/db/airline_10 resulted in this response after a long delay (a minute+?):

{
"error":
"not_found",
"reason":
"Not imported"
}

Console shows:

2017/06/27 16:52:18 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/27 16:52:19 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/27 17:01:19 go-couchbase: call to Do("_sync:user:") in github.com/couchbase/go-couchbase.(*Bucket).casNext took 5m56.860243401s
2017/06/27 17:01:19 go-couchbase: call to Do("_sync:user:") in github.com/couchbase/go-couchbase.(*Bucket).casNext took 5m26.894956221s
2017/06/27 17:01:19 go-couchbase: call to Do("_sync:user:") in github.com/couchbase/go-couchbase.(*Bucket).casNext took 6m28.888380487s
2017-06-27T17:01:19.789-07:00 HTTP:  #004: GET /db/airline_10
2017-06-27T17:01:19.789-07:00 HTTP:  #003: GET /db/airline_10
2017-06-27T17:01:19.789-07:00 HTTP:  #002: GET /db/_all_docs
2017/06/27 17:01:19 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017-06-27T17:01:19.792-07:00 HTTP: #004:     --> 404 Not imported  (326898.1 ms)
2017-06-27T17:01:19.792-07:00 HTTP: #003:     --> 404 Not imported  (356863.4 ms)
2017-06-27T17:01:19.831-07:00 HTTP+: #002:     --> 200   (388930.5 ms)
2017/06/27 17:01:20 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/27 17:01:21 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s

gtur · June 28, 2017, 12:09am

And then, SG crashed:

2017/06/27 17:03:07 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/27 17:03:08 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
failed MSpanList_Insert 0xe37f68 0x173607e14986 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0xd0bcd0, 0xe37f68)
	/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Alloc(0xd0bc60, 0x1, 0x10000000023, 0xda09)
	/usr/local/go/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0xd14cf8, 0xc208641748)
	/usr/local/go/src/runtime/mcentral.c:85 +0x167
runtime.MCache_Refill(0xe0de10, 0x23, 0x800000000)
	/usr/local/go/src/runtime/mcache.c:90 +0xa0

goroutine 38 [running]:
runtime.switchtoM()
	/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc208641708 sp=0xc208641700
runtime.mallocgc(0x600, 0x515820, 0x1, 0xc20823a890)
	/usr/local/go/src/runtime/malloc.go:178 +0x849 fp=0xc2086417b8 sp=0xc208641708
runtime.newarray(0x515820, 0x600, 0x136456)
	/usr/local/go/src/runtime/malloc.go:365 +0xc1 fp=0xc2086417f0 sp=0xc2086417b8
runtime.makeslice(0x4ffe40, 0x200, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/runtime/slice.go:32 +0x15c fp=0xc208641838 sp=0xc2086417f0
encoding/json.(*Decoder).readValue(0xc20823a780, 0xc207fdc587, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:117 +0x479 fp=0xc208641930 sp=0xc208641838
encoding/json.(*Decoder).Decode(0xc20823a780, 0x4ea480, 0xc2080f9160, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b fp=0xc2086419d0 sp=0xc208641930
github.com/couchbase/go-couchbase.queryRestAPI(0xc208010620, 0xc208060420, 0x51, 0xe1fbc8, 0xc208109130, 0x6d3ee0, 0xc2084604e0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/pools.go:478 +0x799 fp=0xc208641b90 sp=0xc2086419d0
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc2081640b8, 0xc208060420, 0x51, 0x6d3ee0, 0xc2084604e0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/pools.go:485 +0x75 fp=0xc208641be0 sp=0xc208641b90
github.com/couchbase/go-couchbase.(*Bucket).Refresh(0xc2081ac820, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/pools.go:660 +0xf2 fp=0xc208641e80 sp=0xc208641be0
github.com/couchbase/go-couchbase.(*TapFeed).run(0xc208174480)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:71 +0x267 fp=0xc208641fd8 sp=0xc208641e80
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208641fe0 sp=0xc208641fd8
created by github.com/couchbase/go-couchbase.(*Bucket).StartTapFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:42 +0x222

goroutine 1 [IO wait]:
net.(*pollDesc).Wait(0xc20824ef40, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20824ef40, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc20824eee0, 0x0, 0xe1ed60, 0xc208310250)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208176398, 0xc2080d4aa8, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208176398, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208212d20, 0xe2bc40, 0xc208176398, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/sync_gateway/base.ListenAndServeHTTP(0xc20802b9f0, 0xe, 0x0, 0x0, 0x0, 0xe1efa8, 0xc2081c0940, 0x0, 0x0, 0x0, ...)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/http_listener.go:52 +0x5ab
github.com/couchbase/sync_gateway/rest.(*ServerConfig).serve(0xc2080320e0, 0xc20802b9f0, 0xe, 0xe1efa8, 0xc2081c0940)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:455 +0xa6
github.com/couchbase/sync_gateway/rest.RunServer(0xc2080320e0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:495 +0x82a
github.com/couchbase/sync_gateway/rest.ServerMain()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:510 +0x34
main.main()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:33 +0x195

goroutine 5 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 7 [chan receive]:
main.func·001()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:27 +0x5c
created by main.main
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:31 +0x18e

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 171 [select]:
net/http.(*persistConn).writeLoop(0xc2085100b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 170 [select]:
net/http.(*persistConn).readLoop(0xc2085100b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 37 [sleep]:
github.com/couchbase/sync_gateway/db.func·004()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:120 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:122 +0x3c3

goroutine 36 [sleep]:
github.com/couchbase/sync_gateway/db.func·003()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:113 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:115 +0x371

goroutine 13 [IO wait]:
net.(*pollDesc).Wait(0xc2080fe060, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080fe060, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080fe000, 0x0, 0xe1ed60, 0xc20823a010)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208176000, 0xc2080ce000, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208176000, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc2080a4000, 0xe2bc40, 0xc208176000, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/sync_gateway/base.ListenAndServeHTTP(0xc20802b9d0, 0xe, 0x0, 0x0, 0x0, 0xe1efa8, 0xc2080e8200, 0x0, 0x0, 0x0, ...)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/http_listener.go:52 +0x5ab
github.com/couchbase/sync_gateway/rest.(*ServerConfig).serve(0xc2080320e0, 0xc20802b9d0, 0xe, 0xe1efa8, 0xc2080e8200)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:455 +0xa6
created by github.com/couchbase/sync_gateway/rest.RunServer
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:493 +0x6eb

goroutine 39 [chan receive]:
github.com/couchbase/sync_gateway/base.func·002()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:120 +0x6d
created by github.com/couchbase/sync_gateway/base.CouchbaseBucket.StartCouchbaseTapFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:130 +0x27a

goroutine 40 [chan receive]:
github.com/couchbase/sync_gateway/db.func·010()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:50 +0x10b
created by github.com/couchbase/sync_gateway/db.(*changeListener).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:67 +0x400

goroutine 41 [chan receive]:
github.com/couchbase/sync_gateway/db.(*DatabaseContext).watchDocChanges(0xc20820c000)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/assimilator.go:16 +0xd0
created by github.com/couchbase/sync_gateway/db.NewDatabaseContext
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:120 +0x65b

goroutine 156 [select]:
net/http.(*persistConn).writeLoop(0xc208088210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 155 [IO wait]:
net.(*pollDesc).Wait(0xc20826ad80, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20826ad80, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20826ad20, 0xc208261000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc208310f68)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038040, 0xc208261000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20d38, 0xc208038040, 0xc208088268, 0xc208261000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208268480, 0xc208261000, 0x1000, 0x1000, 0xc2080cb200, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081ec480)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081ec480, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208088210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

priya.rajagopal · June 28, 2017, 2:23pm

Tagging @traun … Also, one more question . How did you install the server and SG ? Docker? Basically , anything specific about your install that we should be aware of.

gtur · June 28, 2017, 4:41pm

Just barebones installation on my laptop. No docker. All default configurations, other than activating travel-sample, adding a user with pull permissions to the bucket, and then adding that user to SG config. No other change.

priya.rajagopal · June 28, 2017, 5:09pm

I just scrolled up to your original message - Did you follow the instructions in the document for configuring the sync gateway for import? Please include the following in your config file and restart sync gateway and try again.

"db": {
 "import_docs": "continuous",
      "unsupported": {
        "enable_extended_attributes": true,
        "replicator_2":true
      },

gtur · June 28, 2017, 6:44pm

Hi Priya. I did that, and the SG crashes on start up.

$ sync_gateway /Users/sync_gateway/sync_gateway.json
2017-06-28T11:19:08.910-07:00 Enabling logging: [HTTP+]
2017-06-28T11:19:08.910-07:00 ==== Couchbase Sync Gateway/1.1.0(28;86f028c) ====
2017-06-28T11:19:08.910-07:00 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this
2017-06-28T11:19:08.910-07:00 Opening db /db as bucket "travel-sample", pool "default", server <http://localhost:8091>
2017-06-28T11:19:08.911-07:00 Opening Couchbase database travel-sample on <http://localhost:8091> as user "travel-sample-gw"
2017/06/28 11:19:08  Trying with selected node 0
2017/06/28 11:19:09  Trying with selected node 0
2017-06-28T11:19:09.160-07:00 Importing documents...
2017/06/28 11:19:09 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017/06/28 11:19:09 go-couchbase: call to ViewCustom("sync_housekeeping", "import") in github.com/couchbase/sync_gateway/base.CouchbaseBucket.View took 203.882885ms
2017-06-28T11:19:09.366-07:00 Re-running sync function on all 31591 documents...
1969/12/31 16:00:00 go-couchbase: call to Do("_sync:seq") in github.com/couchbase/go-couchbase.(*Bucket).Incr took 24.020784s
[signal panic during panic
[signal stack trace unavailable

and then again:

$ sync_gateway /Users/sync_gateway/sync_gateway.json
2017-06-28T11:39:40.461-07:00 Enabling logging: [HTTP+]
2017-06-28T11:39:40.462-07:00 ==== Couchbase Sync Gateway/1.1.0(28;86f028c) ====
2017-06-28T11:39:40.462-07:00 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this
2017-06-28T11:39:40.462-07:00 Opening db /db as bucket “travel-sample”, pool “default”, server http://localhost:8091
2017-06-28T11:39:40.462-07:00 Opening Couchbase database travel-sample on http://localhost:8091 as user “travel-sample-gw”
2017/06/28 11:39:40 Trying with selected node 0
2017/06/28 11:39:40 Trying with selected node 0
2017-06-28T11:39:40.702-07:00 Importing documents…
2017/06/28 11:39:40 go-couchbase: TAP connection lost; reconnecting to bucket “travel-sample” in 1s
2017/06/28 11:39:41 go-couchbase: call to ViewCustom(“sync_housekeeping”, “import”) in github.com/couchbase/sync_gateway/base.CouchbaseBucket.View took 314.81997ms
2017-06-28T11:39:41.019-07:00 Re-running sync function on all 30714 documents…
1969/12/31 16:00:00 go-couchbase: call to Do(“airport_7141”) in github.com/couchbase/go-couchbase.(*Bucket).casNext took 24.020784s
fatal error: unexpected signal during runtime execution
[signal 0xa code=0x2 addr=0xc208e00000 pc=0x2c8c7]

goroutine 1 [running]:
runtime.gothrow(0x7cfc30, 0x2a)
/usr/local/go/src/runtime/panic.go:503 +0x8e fp=0xc2081054d0 sp=0xc2081054b8
runtime.sigpanic()
/usr/local/go/src/runtime/sigpanic_unix.go:14 +0x5e fp=0xc208105520 sp=0xc2081054d0
runtime.memmove(0xc208105570, 0xc20803a8f0, 0xffffffffc45a9251)
/usr/local/go/src/runtime/memmove_amd64.s:82 +0x87 fp=0xc208105528 sp=0xc208105520
runtime.deferreturn(0xe7791f700)
/usr/local/go/src/runtime/panic.go:250 +0x83 fp=0xc208105570 sp=0xc208105528
github.com/couchbase/go-couchbase.(*Bucket).casNext(0x0, 0x0, 0x0, 0x0, 0x0, 0x65ef40)
/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:633 +0x1a2 fp=0xc208105618 sp=0xc208105570

goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 7 [chan receive]:
main.func·001()
/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:27 +0x5c
created by main.main
/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:31 +0x18e

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 18 [IO wait]:
runtime.gopark(0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/runtime/proc.go:130 +0x105
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 19 [select]:
net/http.(*persistConn).writeLoop(0xc2080ee000)
/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 0 [idle]:

goroutine 0 [idle]:
panic during panic
[signal 0xb code=0x1 addr=0x0 pc=0x1f79b]

runtime stack:
runtime.startpanic_m()
/usr/local/go/src/runtime/panic.c:110 +0x108
runtime.onM(0x98f9a8)
/usr/local/go/src/runtime/asm_amd64.s:273 +0x9a
runtime.startpanic()
/usr/local/go/src/runtime/panic.go:469 +0x14
runtime.gothrow(0x7cfc30, 0x2a)
/usr/local/go/src/runtime/panic.go:501 +0x35
runtime.sigpanic()
/usr/local/go/src/runtime/sigpanic_unix.go:14 +0x5e
runtime.gentraceback(0x0, 0x0, 0x0, 0xc2081b0240, 0x0, 0x0, 0x64, 0x0, 0x0, 0x0, …)
/usr/local/go/src/runtime/traceback.go:160 +0x1ab
runtime.traceback1(0xffffffffffffffff, 0xffffffffffffffff, 0x0, 0xc2081b0240, 0x0)
/usr/local/go/src/runtime/traceback.go:517 +0xc8
runtime.traceback(0xffffffffffffffff, 0xffffffffffffffff, 0x0, 0xc2081b0240)
/usr/local/go/src/runtime/traceback.go:494 +0x4f
runtime.tracebackothers(0xc208000120)
/usr/local/go/src/runtime/traceback.go:644 +0x23f
runtime.dopanic_m()
/usr/local/go/src/runtime/panic.c:153 +0x189
runtime.onM(0xc208106000)
/usr/local/go/src/runtime/asm_amd64.s:257 +0x68
runtime.mstart()
/usr/local/go/src/runtime/proc.c:818

The config now looks like:

{
        "log": ["HTTP+"],
        "adminInterface": "127.0.0.1:4985",
        "interface": "127.0.0.1:4984",
        "databases": {
                "db": {
                        "server":"http://localhost:8091",
                        "username":"***",
                        "password":"***",
                        "bucket":"travel-sample",
                        "unsupported": {
                            "enable_extended_attributes": true
                        },
                        "import_docs": "continuous",
                        "users": {
                            "GUEST": {"disabled": false, "admin_channels": ["*"] }
                        }
                }
        }
}

gtur · June 28, 2017, 6:51pm

Adding “replicator_2”: true also results in a crash:

$ sync_gateway /Users/sync_gateway/sync_gateway.json
2017-06-28T11:49:40.817-07:00 Enabling logging: [HTTP+]
2017-06-28T11:49:40.817-07:00 ==== Couchbase Sync Gateway/1.1.0(28;86f028c) ====
2017-06-28T11:49:40.817-07:00 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this
2017-06-28T11:49:40.817-07:00 Opening db /db as bucket "travel-sample", pool "default", server <http://localhost:8091>
2017-06-28T11:49:40.817-07:00 Opening Couchbase database travel-sample on <http://localhost:8091> as user "travel-sample-gw"
2017/06/28 11:49:40  Trying with selected node 0
2017/06/28 11:49:40  Trying with selected node 0
2017-06-28T11:49:41.063-07:00 Importing documents...
2017/06/28 11:49:41 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
failed MSpanList_Insert 0xe13018 0x1ff97626e128 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0xd0be70, 0xe13018)
	/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Free(0xd0bc60, 0xe13018, 0xc200000000)
	/usr/local/go/src/runtime/mheap.c:500 +0x5b
runtime.MCentral_FreeSpan(0xd16498, 0xe13018, 0x1, 0xc2080fc000, 0xc2080fc000, 0xc208020e00, 0x64)
	/usr/local/go/src/runtime/mcentral.c:181 +0x1bb
runtime.MSpan_Sweep(0xe13018, 0x1900000000, 0xc200000001)
	/usr/local/go/src/runtime/mgc0.c:1096 +0x477

goroutine 3 [running]:
runtime.switchtoM()
	/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc20801dfa8 sp=0xc20801dfa0
runtime.gosweepone(0x0)
	/usr/local/go/src/runtime/mgc0.c:1165 +0x1e fp=0xc20801dfc0 sp=0xc20801dfa8
runtime.bgsweep()
	/usr/local/go/src/runtime/mgc0.go:85 +0x36 fp=0xc20801dfe0 sp=0xc20801dfc0
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20801dfe8 sp=0xc20801dfe0
created by gc
	/usr/local/go/src/runtime/mgc0.c:1383

goroutine 1 [runnable]:
encoding/json.stateInString(0xc208174020, 0x61, 0x0)
	/usr/local/go/src/encoding/json/scanner.go:324
encoding/json.(*decodeState).scanWhile(0xc208174000, 0x0, 0x1)
	/usr/local/go/src/encoding/json/decode.go:238 +0xc9
encoding/json.(*decodeState).object(0xc208174000, 0x65fbc0, 0xc2080f6780, 0xd9)
	/usr/local/go/src/encoding/json/decode.go:539 +0x82e
encoding/json.(*decodeState).value(0xc208174000, 0x4e80e0, 0xc208778320, 0xd6)
	/usr/local/go/src/encoding/json/decode.go:288 +0x3ce
encoding/json.(*decodeState).array(0xc208174000, 0x515d20, 0xc20813c588, 0xd7)
	/usr/local/go/src/encoding/json/decode.go:439 +0xa9b
encoding/json.(*decodeState).value(0xc208174000, 0x515d20, 0xc20813c588, 0xd7)
	/usr/local/go/src/encoding/json/decode.go:285 +0x3ee
encoding/json.(*decodeState).object(0xc208174000, 0x65fa80, 0xc20813c580, 0xd9)
	/usr/local/go/src/encoding/json/decode.go:605 +0x1199
encoding/json.(*decodeState).value(0xc208174000, 0x655340, 0xc20813c580, 0x16)
	/usr/local/go/src/encoding/json/decode.go:288 +0x3ce
encoding/json.(*decodeState).unmarshal(0xc208174000, 0x655340, 0xc20813c580, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:149 +0x1e7
encoding/json.Unmarshal(0xc2083e4000, 0x1db604, 0x1ffe00, 0x655340, 0xc20813c580, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:78 +0x145
github.com/couchbase/go-couchbase.(*Bucket).ViewCustom(0xc208037a00, 0x76bef0, 0x11, 0x6f5e60, 0x6, 0xc2081425a0, 0x655340, 0xc20813c580, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/views.go:183 +0xa25
github.com/couchbase/sync_gateway/base.CouchbaseBucket.View(0xc208037a00, 0xc20800b9c0, 0x15, 0x6f1600, 0x7, 0xc20802bbd0, 0xd, 0x0, 0x0, 0xe1f9e0, ...)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:86 +0xcf
github.com/couchbase/sync_gateway/base.(*CouchbaseBucket).View(0xc208060c00, 0x76bef0, 0x11, 0x6f5e60, 0x6, 0xc2081425a0, 0x0, 0x0, 0x0, 0x0, ...)
	<autogenerated>:74 +0x120
github.com/couchbase/sync_gateway/db.(*Database).UpdateAllDocChannels(0xc2081405a0, 0x100, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:559 +0x49a
github.com/couchbase/sync_gateway/rest.(*ServerContext).getOrAddDatabaseFromConfig(0xc20808a2c0, 0xc208108d10, 0xc208133e00, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:244 +0xc22
github.com/couchbase/sync_gateway/rest.(*ServerContext).AddDatabaseFromConfig(0xc20808a2c0, 0xc208108d10, 0xc208133ea8, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:344 +0x49
github.com/couchbase/sync_gateway/rest.RunServer(0xc2080320e0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:479 +0x335
github.com/couchbase/sync_gateway/rest.ServerMain()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:510 +0x34
main.main()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:33 +0x195

goroutine 5 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 11 [sleep]:
github.com/couchbase/sync_gateway/db.func·004()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:120 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:122 +0x3c3

goroutine 7 [chan receive]:
main.func·001()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:27 +0x5c
created by main.main
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:31 +0x18e

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 18 [IO wait]:
net.(*pollDesc).Wait(0xc2080a0060, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080a0060, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080a0000, 0xc2080e2000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc20813e518)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208144000, 0xc2080e2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20b78, 0xc208144000, 0xc2080fa058, 0xc2080e2000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081400e0, 0xc2080e2000, 0x1000, 0x1000, 0xc208014d00, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208067800)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208067800, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080fa000)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 19 [select]:
net/http.(*persistConn).writeLoop(0xc2080fa000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 10 [sleep]:
github.com/couchbase/sync_gateway/db.func·003()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:113 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:115 +0x371

goroutine 12 [select]:
github.com/couchbase/go-couchbase.(*TapFeed).run(0xc20808ac80)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:74 +0x392
created by github.com/couchbase/go-couchbase.(*Bucket).StartTapFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:42 +0x222

goroutine 13 [chan receive]:
github.com/couchbase/sync_gateway/base.func·002()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:120 +0x6d
created by github.com/couchbase/sync_gateway/base.CouchbaseBucket.StartCouchbaseTapFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:130 +0x27a

goroutine 14 [chan receive]:
github.com/couchbase/sync_gateway/db.func·010()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:50 +0x10b
created by github.com/couchbase/sync_gateway/db.(*changeListener).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:67 +0x400

goroutine 15 [chan receive]:
github.com/couchbase/sync_gateway/db.(*DatabaseContext).watchDocChanges(0xc2081c6000)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/assimilator.go:16 +0xd0
created by github.com/couchbase/sync_gateway/db.NewDatabaseContext
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:120 +0x65b

goroutine 16 [IO wait]:
net.(*pollDesc).Wait(0xc2080a0c30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080a0c30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080a0bd0, 0xc2081de000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc2080f4120)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080381c8, 0xc2081de000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20b78, 0xc2080381c8, 0xc208088528, 0xc2081de000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20818b3e0, 0xc2081de000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081ca5a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081ca5a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080884d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 50 [select]:
net/http.(*persistConn).writeLoop(0xc2080884d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

priya.rajagopal · June 28, 2017, 7:43pm

To scope the problem can you do the following:-

Update “db” in your sync gateway to “travel-sample” and restart SG.
If that does not work,
Provide the user full access to the bucket - basically “Admin” level access and retry
Unrelated to the issue, you may also want to configure some users in the SG config so you can query for docs via SG public interface. This is a moot issue now since the SG is crashing but once you have docs imported through SG, you would want to have a user configured so you can query.

Example
:"users":{ "<my_user>": {"password": "<my_password>", "admin_channels": ["*"]},

matt.carabine · June 28, 2017, 8:54pm

Couchbase Server 5.0 has officially removed TAP (after it was deprecated in earlier releases), which is why your TAP connection is probably failing.

There is an open issue to handle this at https://github.com/couchbase/sync_gateway/issues/2523.

That said, I thought that DCP was the default connection method in Sync Gateway now, so perhaps it is falling back to TAP as the DCP has failed.

What exact RBAC roles have you assigned to the user you’re using for Sync Gateway?

gtur · June 28, 2017, 9:35pm

Only one role, that shows up in the CB Security tab as: Bucket Full Access[travel-sample]

gtur · June 28, 2017, 10:13pm

Renaming ‘db’ to ‘travel-sample’ and restarting also causes a crash:

$ sync_gateway /Users/sync_gateway/sync_gateway.json
2017-06-28T15:10:16.616-07:00 Enabling logging: [HTTP+]
2017-06-28T15:10:16.617-07:00 ==== Couchbase Sync Gateway/1.1.0(28;86f028c) ====
2017-06-28T15:10:16.617-07:00 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this
2017-06-28T15:10:16.617-07:00 Opening db /travel-sample as bucket "travel-sample", pool "default", server <http://localhost:8091>
2017-06-28T15:10:16.618-07:00 Opening Couchbase database travel-sample on <http://localhost:8091> as user "travel-sample-gw"
2017/06/28 15:10:16  Trying with selected node 0
2017/06/28 15:10:16  Trying with selected node 0
2017-06-28T15:10:16.860-07:00 Importing documents...
2017/06/28 15:10:16 go-couchbase: TAP connection lost; reconnecting to bucket "travel-sample" in 1s
2017-06-28T15:10:17.049-07:00 Re-running sync function on all 30161 documents...
2017-06-28T15:10:17.049-07:00 WARNING: Error updating doc "airport_7141": invalid character 'Î' looking for beginning of value -- db.(*Database).UpdateAllDocChannels() at database.go:635
failed MSpanList_Insert 0xe47198 0x29575a100a7e 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0xd0bed8, 0xe47198)
	/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Alloc(0xd0bc60, 0x1, 0x10000000004, 0xda09)
	/usr/local/go/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0xd12a18, 0xc208106df8)
	/usr/local/go/src/runtime/mcentral.c:85 +0x167
runtime.MCache_Refill(0xe0d960, 0x4, 0xe47268)
	/usr/local/go/src/runtime/mcache.c:90 +0xa0

goroutine 1 [running]:
runtime.switchtoM()
	/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc2081072f0 sp=0xc2081072e8
runtime.mallocgc(0x30, 0x615700, 0x0, 0x10)
	/usr/local/go/src/runtime/malloc.go:178 +0x849 fp=0xc2081073a0 sp=0xc2081072f0
runtime.newobject(0x615700, 0xc208401a01)
	/usr/local/go/src/runtime/malloc.go:353 +0x49 fp=0xc2081073c8 sp=0xc2081073a0
runtime.makemap(0x6b6620, 0x0, 0xc208107480)
	/usr/local/go/src/runtime/hashmap.go:227 +0x36e fp=0xc208107418 sp=0xc2081073c8
github.com/couchbase/sync_gateway/db.(*Database).initializeSyncData(0xc2081dc0a0, 0xc208b560d0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/crud.go:353 +0x172 fp=0xc2081074f0 sp=0xc208107418
github.com/couchbase/sync_gateway/db.func·023(0xc208b56004, 0xc4, 0xc4, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:594 +0x2b1 fp=0xc208107618 sp=0xc2081074f0
github.com/couchbase/go-couchbase.(*Bucket).update(0xc208041860, 0xc208149dd0, 0xc, 0x0, 0xc208b39cb0, 0x14cc68a11a680000, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:663 +0xd1 fp=0xc208107690 sp=0xc208107618
github.com/couchbase/go-couchbase.(*Bucket).Update(0xc208041860, 0xc208149dd0, 0xc, 0x0, 0xc208b39cb0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:654 +0x62 fp=0xc2081076d8 sp=0xc208107690
github.com/couchbase/sync_gateway/base.CouchbaseBucket.Update(0xc208041860, 0xc20800b9c0, 0x15, 0x6f1600, 0x7, 0xc20802bbe0, 0xd, 0x0, 0x0, 0xe1f918, ...)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:72 +0x74 fp=0xc208107718 sp=0xc2081076d8
github.com/couchbase/sync_gateway/base.(*CouchbaseBucket).Update(0xc2080c7d40, 0xc208149dd0, 0xc, 0x0, 0xc208b39cb0, 0x0, 0x0)
	<autogenerated>:72 +0xfd fp=0xc2081077a8 sp=0xc208107718
github.com/couchbase/sync_gateway/db.(*Database).UpdateAllDocChannels(0xc2081dc0a0, 0x100, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:631 +0x94e fp=0xc208107a70 sp=0xc2081077a8
github.com/couchbase/sync_gateway/rest.(*ServerContext).getOrAddDatabaseFromConfig(0xc2081022c0, 0xc2080320b0, 0xc208107e00, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:244 +0xc22 fp=0xc208107dc8 sp=0xc208107a70
github.com/couchbase/sync_gateway/rest.(*ServerContext).AddDatabaseFromConfig(0xc2081022c0, 0xc2080320b0, 0xc208107ea8, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:344 +0x49 fp=0xc208107e00 sp=0xc208107dc8
github.com/couchbase/sync_gateway/rest.RunServer(0xc2081240e0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:479 +0x335 fp=0xc208107f00 sp=0xc208107e00
github.com/couchbase/sync_gateway/rest.ServerMain()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:510 +0x34 fp=0xc208107f10 sp=0xc208107f00
main.main()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:33 +0x195 fp=0xc208107f98 sp=0xc208107f10
runtime.main()
	/usr/local/go/src/runtime/proc.go:63 +0xf3 fp=0xc208107fe0 sp=0xc208107f98
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208107fe8 sp=0xc208107fe0

goroutine 5 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 7 [chan receive]:
main.func·001()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:27 +0x5c
created by main.main
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:31 +0x18e

goroutine 14 [sleep]:
github.com/couchbase/sync_gateway/db.func·003()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:113 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:115 +0x371

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 9 [IO wait]:
net.(*pollDesc).Wait(0xc208092530, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208092530, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080924d0, 0xc2080e2000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc2081e60a8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208042120, 0xc2080e2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20ab0, 0xc208042120, 0xc208032268, 0xc2080e2000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20810a220, 0xc2080e2000, 0x1000, 0x1000, 0xc2080de900, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208072780)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208072780, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208032210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 10 [select]:
net/http.(*persistConn).writeLoop(0xc208032210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 15 [sleep]:
github.com/couchbase/sync_gateway/db.func·004()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:120 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:122 +0x3c3

goroutine 16 [select]:
github.com/couchbase/go-couchbase.(*TapFeed).run(0xc208102e00)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:74 +0x392
created by github.com/couchbase/go-couchbase.(*Bucket).StartTapFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:42 +0x222

goroutine 34 [chan receive]:
github.com/couchbase/sync_gateway/base.func·002()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:120 +0x6d
created by github.com/couchbase/sync_gateway/base.CouchbaseBucket.StartCouchbaseTapFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:130 +0x27a

goroutine 35 [chan receive]:
github.com/couchbase/sync_gateway/db.func·010()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:50 +0x10b
created by github.com/couchbase/sync_gateway/db.(*changeListener).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:67 +0x400

goroutine 36 [chan receive]:
github.com/couchbase/sync_gateway/db.(*DatabaseContext).watchDocChanges(0xc2081b0000)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/assimilator.go:16 +0xd0
created by github.com/couchbase/sync_gateway/db.NewDatabaseContext
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:120 +0x65b

goroutine 52 [IO wait]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc208230000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc208212120)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208208008, 0xc208230000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20ab0, 0xc208208008, 0xc208098478, 0xc208230000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081fa060, 0xc208230000, 0x1000, 0x1000, 0xc208015600, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081f4120)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081f4120, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208098420)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 53 [select]:
net/http.(*persistConn).writeLoop(0xc208098420)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

priya.rajagopal · June 29, 2017, 3:27am

And did you try this ? There should be a checkbox at the bottom of the roles form that says “Admin” .
There may be a problem if you didn’t provide the user with replication roles. The crash is probably a red herring because of falling back to TAP …

priya.rajagopal · June 29, 2017, 5:21pm

@gtur: Can you please update your config file to specify “DCP” as the feed type instead of TAP. That should get us past the crash (which is because of TAP) and we can then focus on the other user RBAC related issues
"feed_type":"DCP"

Reference : https://developer.couchbase.com/documentation/mobile/1.4/guides/sync-gateway/config-properties/index.html (this is for 1.4 but you can look it up for a description of this field)

Note that in future releases, this will default to DCP so you wouldn’t have to set this up on config file.

gtur · June 29, 2017, 10:36pm

Reinstalled Couchbase Server. Added user permission set to Admin, added feed_type = DCP. Still crashes. I am on MacOS 10.12.5.

If you’ve gotten this to work on a Mac with travel-sample, can I borrow your config file?

The config file looks like this:

{
        "log": ["HTTP+"],
        "adminInterface": "127.0.0.1:4985",
        "interface": "127.0.0.1:4984",
        "databases": {
                "travel-sample": {
                        "server":"http://localhost:8091",
                        "username":"***",
                        "password":"***",
                        "bucket":"travel-sample",
                        "feed_type":"DCP",
                        "unsupported": {
                            "enable_extended_attributes": true,
                            "replicator_2": true
                        },
                        "import_docs": "continuous",
                        "users": {
                            "GUEST": {"disabled": false, "admin_channels": ["*"] }
                        }
                }
        }
}

Crash log:

$ sync_gateway /Users/sync_gateway/sync_gateway.json
2017-06-29T15:31:06.400-07:00 Enabling logging: [HTTP+]
2017-06-29T15:31:06.400-07:00 ==== Couchbase Sync Gateway/1.1.0(28;86f028c) ====
2017-06-29T15:31:06.400-07:00 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this
2017-06-29T15:31:06.400-07:00 Opening db /travel-sample as bucket "travel-sample", pool "default", server <http://localhost:8091>
2017-06-29T15:31:06.400-07:00 Opening Couchbase database travel-sample on <http://localhost:8091> as user "travel-sample-gw"
2017/06/29 15:31:06  Trying with selected node 0
2017/06/29 15:31:06  Trying with selected node 0
2017-06-29T15:31:06.644-07:00 Importing documents...
2017-06-29T15:31:06.652-07:00 WARNING: Feed%!(EXTRA string=Error processing DCP stream: %v, *errors.errorString=UPROpen receive, err: EOF) -- base.(*DCPReceiver).OnError() at dcp_feed.go:103
2017-06-29T15:31:06.753-07:00 WARNING: Feed%!(EXTRA string=Error processing DCP stream: %v, *errors.errorString=UPROpen receive, err: EOF) -- base.(*DCPReceiver).OnError() at dcp_feed.go:103
2017-06-29T15:31:06.908-07:00 WARNING: Feed%!(EXTRA string=Error processing DCP stream: %v, *errors.errorString=UPROpen receive, err: EOF) -- base.(*DCPReceiver).OnError() at dcp_feed.go:103
2017/06/29 15:31:06 go-couchbase: call to ViewCustom("sync_housekeeping", "import") in github.com/couchbase/sync_gateway/base.CouchbaseBucket.View took 277.029646ms
2017-06-29T15:31:06.922-07:00 Re-running sync function on all 30715 documents...
2017-06-29T15:31:07.134-07:00 WARNING: Feed%!(EXTRA string=Error processing DCP stream: %v, *errors.errorString=UPROpen receive, err: EOF) -- base.(*DCPReceiver).OnError() at dcp_feed.go:103
failed MSpanList_Insert 0xeb2bb8 0x39abb222a21f 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0xd0bcd0, 0xeb2bb8)
	/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Free(0xd0bc60, 0xeb2bb8, 0x0)
	/usr/local/go/src/runtime/mheap.c:500 +0x5b
runtime.MCentral_FreeSpan(0xd13e58, 0xeb2bb8, 0x13, 0xc208b42000, 0xc208b43d40, 0xc208021500, 0x64)
	/usr/local/go/src/runtime/mcentral.c:181 +0x1bb
runtime.MSpan_Sweep(0xeb2bb8, 0x2100000000, 0xc200000401)
	/usr/local/go/src/runtime/mgc0.c:1096 +0x477

goroutine 3 [running]:
runtime.switchtoM()
	/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc20801dfa8 sp=0xc20801dfa0
runtime.gosweepone(0x1)
	/usr/local/go/src/runtime/mgc0.c:1165 +0x1e fp=0xc20801dfc0 sp=0xc20801dfa8
runtime.bgsweep()
	/usr/local/go/src/runtime/mgc0.go:85 +0x36 fp=0xc20801dfe0 sp=0xc20801dfc0
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20801dfe8 sp=0xc20801dfe0
created by gc
	/usr/local/go/src/runtime/mgc0.c:1383

goroutine 1 [IO wait]:
net.(*pollDesc).Wait(0xc20808b720, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20808b720, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20808b6c0, 0xc20817f700, 0x18, 0x18, 0x0, 0xe1ed60, 0xc2089e2000)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038218, 0xc20817f700, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0xe20b78, 0xc208038218, 0xc20817f700, 0x18, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xf1
io.ReadFull(0xe20b78, 0xc208038218, 0xc20817f700, 0x18, 0x18, 0x1e8, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x6d
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc208186060, 0xe20b78, 0xc208038218, 0xc20817f700, 0x18, 0x18, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/mc_res.go:154 +0xee
github.com/couchbase/gomemcached/client.getResponse(0xe20b78, 0xc208038218, 0xc20817f700, 0x18, 0x18, 0xc208186060, 0x10, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Send(0xc208178d80, 0xc2080688a0, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/client/mc.go:81 +0x10d
github.com/couchbase/gomemcached/client.(*Client).CASNext(0xc208178d80, 0x29, 0xc2081b0230, 0xc, 0x0, 0xc20803b270, 0xc208021c00)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/client/mc.go:458 +0x559
github.com/couchbase/go-couchbase.func·015(0xc208178d80, 0xc208170029, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:630 +0x7b
github.com/couchbase/go-couchbase.func·002(0xc2080d4700, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:102 +0x146
github.com/couchbase/go-couchbase.(*Bucket).Do(0xc208037860, 0xc2081b0230, 0xc, 0xc2081d35e8, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:108 +0x30c
github.com/couchbase/go-couchbase.(*Bucket).casNext(0xc208037860, 0xc2081b0230, 0xc, 0x0, 0xc20803b270, 0x100)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:632 +0x143
github.com/couchbase/go-couchbase.(*Bucket).update(0xc208037860, 0xc2081b0230, 0xc, 0x0, 0xc208179230, 0x14ccb858c7fb0000, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:661 +0x8b
github.com/couchbase/go-couchbase.(*Bucket).Update(0xc208037860, 0xc2081b0230, 0xc, 0x0, 0xc208179230, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:654 +0x62
github.com/couchbase/sync_gateway/base.CouchbaseBucket.Update(0xc208037860, 0xc20800b9c0, 0x15, 0x6f1600, 0x7, 0xc20802bbe0, 0xd, 0xc20802bf65, 0x3, 0xe1f9e0, ...)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:72 +0x74
github.com/couchbase/sync_gateway/base.(*CouchbaseBucket).Update(0xc2080bd8c0, 0xc2081b0230, 0xc, 0x0, 0xc208179230, 0x0, 0x0)
	<autogenerated>:72 +0xfd
github.com/couchbase/sync_gateway/db.(*Database).UpdateAllDocChannels(0xc20817e120, 0x100, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:631 +0x94e
github.com/couchbase/sync_gateway/rest.(*ServerContext).getOrAddDatabaseFromConfig(0xc2080f82c0, 0xc2081040b0, 0xc2080fde00, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:244 +0xc22
github.com/couchbase/sync_gateway/rest.(*ServerContext).AddDatabaseFromConfig(0xc2080f82c0, 0xc2081040b0, 0xc2080fdea8, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:344 +0x49
github.com/couchbase/sync_gateway/rest.RunServer(0xc20811c0e0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:479 +0x335
github.com/couchbase/sync_gateway/rest.ServerMain()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:510 +0x34
main.main()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:33 +0x195

goroutine 5 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 7 [chan receive]:
main.func·001()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:27 +0x5c
created by main.main
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:31 +0x18e

goroutine 38 [select]:
net/http.(*persistConn).writeLoop(0xc2080fa160)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 18 [IO wait]:
net.(*pollDesc).Wait(0xc2080101b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080101b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010150, 0xc2080d8000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc2080e6e40)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080f4000, 0xc2080d8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20b78, 0xc2080f4000, 0xc2080e8058, 0xc2080d8000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080d6100, 0xc2080d8000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080682a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080682a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080e8000)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 19 [select]:
net/http.(*persistConn).writeLoop(0xc2080e8000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 15 [chan receive]:
github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).refreshCluster(0xc20809f200, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:527 +0x9ab
github.com/couchbase/go-couchbase/cbdatasource.func·001(0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:454 +0x2a
github.com/couchbase/go-couchbase/cbdatasource.ExponentialBackoffLoop(0x7795f0, 0x1b, 0xc208017fc8, 0x64, 0x3fc00000, 0x3e8)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:1518 +0x2e
github.com/couchbase/go-couchbase/cbdatasource.func·002()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:455 +0xbd
created by github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:460 +0x3f0

goroutine 13 [sleep]:
github.com/couchbase/sync_gateway/db.func·004()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:120 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:122 +0x3c3

goroutine 12 [sleep]:
github.com/couchbase/sync_gateway/db.func·003()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:113 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:115 +0x371

goroutine 37 [IO wait]:
net.(*pollDesc).Wait(0xc20808a450, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20808a450, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20808a3f0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc2080d40a0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080c2050, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20b78, 0xc2080c2050, 0xc2080fa1b8, 0xc20800f000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20817e660, 0xc20800f000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081862a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081862a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080fa160)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 16 [chan receive]:
github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).refreshWorkers(0xc20809f200)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:563 +0xa1
created by github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:462 +0x40d

goroutine 50 [chan receive]:
github.com/couchbase/sync_gateway/base.func·003()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:199 +0x8d
created by github.com/couchbase/sync_gateway/base.CouchbaseBucket.StartDCPFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:202 +0xafc

goroutine 51 [chan receive]:
github.com/couchbase/sync_gateway/db.func·010()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:50 +0x10b
created by github.com/couchbase/sync_gateway/db.(*changeListener).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:67 +0x400

goroutine 52 [chan receive]:
github.com/couchbase/sync_gateway/db.(*DatabaseContext).watchDocChanges(0xc2081b6000)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/assimilator.go:16 +0xd0
created by github.com/couchbase/sync_gateway/db.NewDatabaseContext
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:120 +0x65b

goroutine 53 [sleep]:
github.com/couchbase/go-couchbase/cbdatasource.ExponentialBackoffLoop(0xc2080ed440, 0x23, 0xc2080187b8, 0x64, 0xc23fc00000, 0x3e8)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:1527 +0x64
github.com/couchbase/go-couchbase/cbdatasource.func·004()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:682 +0x185
created by github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).workerStart
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:685 +0x270

gtur · June 29, 2017, 11:22pm

Can’t imagine this being the issue, but thought I’d mention it.

priya.rajagopal · June 30, 2017, 2:59am

FWIW , this is my config file . My RBAC user “admin” is created on server and has full admin rights. I have this running on Sierra. (10.12.4). I have enabled full logging on= Try that out and see if the error logs give more insight.
The one difference is that the Couchbase Server 5.0 that I have is the previous Beta Release , not the latest from a week ago.
Maybe there are some incompatibility issues with the latest Beta and SG1.5 .I’d have to defer to @traun for insights

{
  "interface":":4984",
  "log": ["*"],
  "databases": {
    "travel-sample": {
      "import_docs": "continuous",
      "unsupported": {
        "enable_extended_attributes": true,
        "replicator_2":true
      },
      "bucket":"travel-sample",
      "server": "http://localhost:8091",
      "username": "admin",
      "password": "password",
      "users":{
          "admin": {"password": "password", "admin_channels": ["*"]}
      }

    }
  }
}

gtur · June 30, 2017, 5:30pm

Used your config file and added feed_type = DCP. Made some progress, but it crashes nonetheless. Same MSpanList_Insert fatal error:

$sync_gateway /Users/sync_gateway/sync_gateway.json
2017-06-30T10:24:50.466-07:00 Enabling logging: [*]
2017-06-30T10:24:50.466-07:00 ==== Couchbase Sync Gateway/1.1.0(28;86f028c) ====
2017-06-30T10:24:50.466-07:00 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this
2017-06-30T10:24:50.466-07:00 Opening db /travel-sample as bucket "travel-sample", pool "default", server <http://localhost:8091>
2017-06-30T10:24:50.466-07:00 Opening Couchbase database travel-sample on <http://localhost:8091> as user "Administrator"
2017/06/30 10:24:50  Trying with selected node 0
2017/06/30 10:24:50  Trying with selected node 0
2017-06-30T10:24:50.708-07:00 Cache: Initializing changes cache with options {CachePendingSeqMaxWait:5s CachePendingSeqMaxNum:10000 CacheSkippedSeqMaxWait:1h0m0s}
2017-06-30T10:24:50.710-07:00 Feed+: Seeding seqnos: map[3:609 5:635 20:605 34:613 44:606 59:608 14:601 17:613 27:572 28:598 62:597 4:602 9:591 18:610 54:587 8:7995 49:580 51:630 55:585 39:626 21:602 45:613 26:585 29:610 32:607 36:607 37:600 47:626 63:582 19:597 48:582 0:607 6:596 24:597 33:603 41:613 52:624 58:609 7:584 22:619 46:600 16:641 38:605 43:613 57:620 10:617 15:594 31:602 1:586 23:646 30:663 35:598 50:577 61:631 2:642 11:650 12:663 13:609 25:655 40:631 42:595 53:590 56:567 60:595]
2017-06-30T10:24:50.711-07:00 Feed+: Connecting to new bucket datasource.  URLs:[http://localhost:8091], pool:default, name:travel-sample, auth:&{Administrator password travel-sample}
2017-06-30T10:24:50.711-07:00 Feed: Using DCP feed for bucket: "travel-sample"
2017-06-30T10:24:50.711-07:00 Shadow: Watching doc changes...
2017-06-30T10:24:50.711-07:00 Importing documents...
2017-06-30T10:24:50.717-07:00 WARNING: Feed%!(EXTRA string=Error processing DCP stream: %v, *errors.errorString=UPROpen receive, err: EOF) -- base.(*DCPReceiver).OnError() at dcp_feed.go:103
2017-06-30T10:24:50.822-07:00 WARNING: Feed%!(EXTRA string=Error processing DCP stream: %v, *errors.errorString=UPROpen receive, err: EOF) -- base.(*DCPReceiver).OnError() at dcp_feed.go:103
2017/06/30 10:24:50 go-couchbase: call to ViewCustom("sync_housekeeping", "import") in github.com/couchbase/sync_gateway/base.CouchbaseBucket.View took 233.864914ms
2017-06-30T10:24:50.945-07:00 Re-running sync function on all 24249 documents...
2017-06-30T10:24:50.945-07:00 CRUD: 	Importing document "landmark_8384" --> rev "1-49c498e00ed04beaf32339f6f5d5898a"
2017-06-30T10:24:50.945-07:00 CRUD+: Invoking sync on doc "landmark_8384" rev 1-49c498e00ed04beaf32339f6f5d5898a
2017-06-30T10:24:50.945-07:00 Access: Saving updated channels and access grants of "landmark_8384"

** Many such CRUD/CRUD+/Access logs**

2017-06-30T10:24:51.004-07:00 CRUD: 	Importing document "landmark_8623" --> rev "1-44079fca373d1247809c173fcad91183"
2017-06-30T10:24:51.004-07:00 CRUD+: Invoking sync on doc "landmark_8623" rev 1-44079fca373d1247809c173fcad91183
2017-06-30T10:24:51.004-07:00 Access: Saving updated channels and access grants of "landmark_8623"
2017-06-30T10:24:51.005-07:00 CRUD: 	Importing document "landmark_8624" --> rev "1-232878fcb358b2322ea9bd889b88366f"
2017-06-30T10:24:51.005-07:00 CRUD+: Invoking sync on doc "landmark_8624" rev 1-232878fcb358b2322ea9bd889b88366f
2017-06-30T10:24:51.005-07:00 Access: Saving updated channels and access grants of "landmark_8624"
failed MSpanList_Insert 0xea0eb8 0x41dec2bd8a28 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0xd0dbb0, 0xea0eb8)
	/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Alloc(0xd0bc60, 0x1, 0x10000000002, 0xda09)
	/usr/local/go/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0xd127d8, 0xc20817ac80)
	/usr/local/go/src/runtime/mcentral.c:85 +0x167
runtime.MCache_Refill(0xe0de10, 0x2, 0x800000000)
	/usr/local/go/src/runtime/mcache.c:90 +0xa0

goroutine 1 [running]:
runtime.switchtoM()
	/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc20817ab60 sp=0xc20817ab58
runtime.mallocgc(0x8, 0x651820, 0xc200000001, 0x23)
	/usr/local/go/src/runtime/malloc.go:147 +0x1bb fp=0xc20817ac10 sp=0xc20817ab60
runtime.newobject(0x651820, 0xe1ed60)
	/usr/local/go/src/runtime/malloc.go:353 +0x49 fp=0xc20817ac38 sp=0xc20817ac10
runtime.convT2I(0x651820, 0x5c1b40, 0xd03ee8, 0xc20817acb8, 0x0, 0x0)
	/usr/local/go/src/runtime/iface.go:163 +0x130 fp=0xc20817ac80 sp=0xc20817ac38
net.(*netFD).Read(0xc2081be2a0, 0xc2081aa440, 0x18, 0x18, 0x0, 0xe1ed60, 0xc208b5bf98)
	/usr/local/go/src/net/fd_unix.go:241 +0x391 fp=0xc20817ad20 sp=0xc20817ac80
net.(*conn).Read(0xc2081bc038, 0xc2081aa440, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc fp=0xc20817ad68 sp=0xc20817ad20
io.ReadAtLeast(0xe20bc0, 0xc2081bc038, 0xc2081aa440, 0x18, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xf1 fp=0xc20817adc0 sp=0xc20817ad68
io.ReadFull(0xe20bc0, 0xc2081bc038, 0xc2081aa440, 0x18, 0x18, 0x25, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x6d fp=0xc20817ae10 sp=0xc20817adc0
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc208b35f20, 0xe20bc0, 0xc2081bc038, 0xc2081aa440, 0x18, 0x18, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/mc_res.go:154 +0xee fp=0xc20817af50 sp=0xc20817ae10
github.com/couchbase/gomemcached/client.getResponse(0xe20bc0, 0xc2081bc038, 0xc2081aa440, 0x18, 0x18, 0xc208b35f20, 0x10, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1 fp=0xc20817afa8 sp=0xc20817af50
github.com/couchbase/gomemcached/client.(*Client).Send(0xc2081a63c0, 0xc208b35ec0, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/client/mc.go:81 +0x10d fp=0xc20817b000 sp=0xc20817afa8
github.com/couchbase/gomemcached/client.(*Client).Get(0xc2081a63c0, 0x34, 0xc2080f07b0, 0xd, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/client/mc.go:110 +0xc7 fp=0xc20817b038 sp=0xc20817b000
github.com/couchbase/gomemcached/client.(*Client).CASNext(0xc2081a63c0, 0x34, 0xc2080f07b0, 0xd, 0x0, 0xc208b52a00, 0xc2080ce380)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/gomemcached/client/mc.go:472 +0x1d1 fp=0xc20817b0d8 sp=0xc20817b038
github.com/couchbase/go-couchbase.func·015(0xc2081a63c0, 0xc2081a0034, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:630 +0x7b fp=0xc20817b128 sp=0xc20817b0d8
github.com/couchbase/go-couchbase.func·002(0xc2080f4800, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:102 +0x146 fp=0xc20817b2f8 sp=0xc20817b128
github.com/couchbase/go-couchbase.(*Bucket).Do(0xc20816eb60, 0xc2080f07b0, 0xd, 0xc20817b5e8, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:108 +0x30c fp=0xc20817b570 sp=0xc20817b2f8
github.com/couchbase/go-couchbase.(*Bucket).casNext(0xc20816eb60, 0xc2080f07b0, 0xd, 0x0, 0xc208b52a00, 0x400)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:632 +0x143 fp=0xc20817b618 sp=0xc20817b570
github.com/couchbase/go-couchbase.(*Bucket).update(0xc20816eb60, 0xc2080f07b0, 0xd, 0x0, 0xc208b4d1a0, 0x14ccf636bf4d0000, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:661 +0x8b fp=0xc20817b690 sp=0xc20817b618
github.com/couchbase/go-couchbase.(*Bucket).Update(0xc20816eb60, 0xc2080f07b0, 0xd, 0x0, 0xc208b4d1a0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/client.go:654 +0x62 fp=0xc20817b6d8 sp=0xc20817b690
github.com/couchbase/sync_gateway/base.CouchbaseBucket.Update(0xc20816eb60, 0xc20800b9e0, 0x15, 0x6f1600, 0x7, 0xc20802bbc0, 0xd, 0xc20802bf2d, 0x3, 0xe1fa28, ...)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:72 +0x74 fp=0xc20817b718 sp=0xc20817b6d8
github.com/couchbase/sync_gateway/base.(*CouchbaseBucket).Update(0xc2080acd20, 0xc2080f07b0, 0xd, 0x0, 0xc208b4d1a0, 0x0, 0x0)
	<autogenerated>:72 +0xfd fp=0xc20817b7a8 sp=0xc20817b718
github.com/couchbase/sync_gateway/db.(*Database).UpdateAllDocChannels(0xc208194140, 0x100, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:631 +0x94e fp=0xc20817ba70 sp=0xc20817b7a8
github.com/couchbase/sync_gateway/rest.(*ServerContext).getOrAddDatabaseFromConfig(0xc2080922c0, 0xc2081116b0, 0xc208133e00, 0x0, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:244 +0xc22 fp=0xc20817bdc8 sp=0xc20817ba70
github.com/couchbase/sync_gateway/rest.(*ServerContext).AddDatabaseFromConfig(0xc2080922c0, 0xc2081116b0, 0xc208133ea8, 0x0, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:344 +0x49 fp=0xc20817be00 sp=0xc20817bdc8
github.com/couchbase/sync_gateway/rest.RunServer(0xc2080320e0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:479 +0x335 fp=0xc20817bf00 sp=0xc20817be00
github.com/couchbase/sync_gateway/rest.ServerMain()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/rest/config.go:510 +0x34 fp=0xc20817bf10 sp=0xc20817bf00
main.main()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:33 +0x195 fp=0xc20817bf98 sp=0xc20817bf10
runtime.main()
	/usr/local/go/src/runtime/proc.go:63 +0xf3 fp=0xc20817bfe0 sp=0xc20817bf98
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20817bfe8 sp=0xc20817bfe0

goroutine 5 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 53 [sleep]:
github.com/couchbase/go-couchbase/cbdatasource.ExponentialBackoffLoop(0xc20817ccf0, 0x23, 0xc2089bffb8, 0x64, 0xc23fc00000, 0x3e8)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:1527 +0x64
github.com/couchbase/go-couchbase/cbdatasource.func·004()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:682 +0x185
created by github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).workerStart
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:685 +0x270

goroutine 7 [chan receive]:
main.func·001()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:27 +0x5c
created by main.main
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/main.go:31 +0x18e

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 18 [IO wait]:
net.(*pollDesc).Wait(0xc2080a8060, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080a8060, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080a8000, 0xc2080ea000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc2081dee20)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208100000, 0xc2080ea000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20bc0, 0xc208100000, 0xc208102058, 0xc2080ea000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080f2100, 0xc2080ea000, 0x1000, 0x1000, 0xc2080d2000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080682a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080682a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208102000)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 19 [select]:
net/http.(*persistConn).writeLoop(0xc208102000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 15 [chan receive]:
github.com/couchbase/sync_gateway/db.func·010()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:50 +0x10b
created by github.com/couchbase/sync_gateway/db.(*changeListener).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:67 +0x400

goroutine 37 [sleep]:
github.com/couchbase/sync_gateway/db.func·003()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:113 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:115 +0x371

goroutine 14 [chan receive]:
github.com/couchbase/sync_gateway/base.func·003()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:199 +0x8d
created by github.com/couchbase/sync_gateway/base.CouchbaseBucket.StartDCPFeed
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/base/bucket.go:202 +0xafc

goroutine 13 [chan receive]:
github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).refreshWorkers(0xc2080a1200)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:563 +0xa1
created by github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:462 +0x40d

goroutine 38 [sleep]:
github.com/couchbase/sync_gateway/db.func·004()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:120 +0x66
created by github.com/couchbase/sync_gateway/db.(*changeCache).Init
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_cache.go:122 +0x3c3

goroutine 12 [chan receive]:
github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).refreshCluster(0xc2080a1200, 0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:527 +0x9ab
github.com/couchbase/go-couchbase/cbdatasource.func·001(0x0)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:454 +0x2a
github.com/couchbase/go-couchbase/cbdatasource.ExponentialBackoffLoop(0x7795f0, 0x1b, 0xc20801cfc8, 0x64, 0x3fc00000, 0x3e8)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:1518 +0x2e
github.com/couchbase/go-couchbase/cbdatasource.func·002()
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:455 +0xbd
created by github.com/couchbase/go-couchbase/cbdatasource.(*bucketDataSource).Start
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/go-couchbase/cbdatasource/cbdatasource.go:460 +0x3f0

goroutine 16 [chan receive]:
github.com/couchbase/sync_gateway/db.(*DatabaseContext).watchDocChanges(0xc2081da000)
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/assimilator.go:16 +0xd0
created by github.com/couchbase/sync_gateway/db.NewDatabaseContext
	/Users/jenkins/jenkins/workspace/sync-gateway-unix-builds/release/1.1.0/enterprise/app-under-test/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:120 +0x65b

goroutine 51 [IO wait]:
net.(*pollDesc).Wait(0xc2081be370, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081be370, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081be310, 0xc20800f000, 0x1000, 0x1000, 0x0, 0xe1ed60, 0xc20802ab70)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038088, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0xe20bc0, 0xc208038088, 0xc2081101b8, 0xc20800f000, 0x1000, 0x1000, 0x5a9180, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081946a0, 0xc20800f000, 0x1000, 0x1000, 0xc208014d00, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20806fb60)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20806fb60, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208110160)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 52 [select]:
net/http.(*persistConn).writeLoop(0xc208110160)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

priya.rajagopal · June 30, 2017, 7:11pm

Where did you download the SG from ? You mentioned that you are using 1.5 but this is 1.1.0 which is a very old version and that’s probably why its failing!
You should see something this version in your logs
Couchbase Sync Gateway/1.5.0(

(Thanks to @traun for noticing the discrepancy )

Topic		Replies	Views
'tap connection failed' error in Sync_gateway log file Sync Gateway	2	1311	January 12, 2018
Sync Gateway gets stuck connecting to Couchbase Server 5.5 Sync Gateway connections	3	1755	August 3, 2018
Not able to connect Sync Gateway with Couchbase server Sync Gateway	21	4824	April 27, 2023
Sync Gateway hangs during startup (during auth?) to CB 5.5 beta Sync Gateway	2	973	June 21, 2018
Authentication Error when trying to connect Sync Gateway to Couchbase Server Sync Gateway	6	5307	April 2, 2019

Couchbase Server 5.0 + SG 1.5 fresh install causes 'TAP connection lost'

Related topics