We have been having a pretty consistent indexer issue with the latest community build ( 4.5.1-2844 Community Edition (build-2844) )
Environment:
3 Nodes
Ubuntu 14.04
RAM Allocation: 12gb in use for CB, 4 gb free for OS
---------------------
2x8GB Kingston 8GB DDR3 2rx8
3.5GHz Intel Xeon-Haswell (E3-1270-V3-Quadcore)
each
---------------------
2 buckets per node ( `content` && `scores` )
content bucket: 7gb * 3
scores bucket: 2gb * 3
----------------------
Data RAM quota: 9gb
Index RAM quota: 3gb
Indexer Threads: 6
Circular Write Mode ( 30% Fragmentation )
----------------------
29 Indexes ( 28 GSI && 1 primary )
-----------------------
Document inserts for the past 7 days:
2017-06-05 | 28328
2017-06-06 | 26009
2017-06-07 | 34013
2017-06-08 | 56880
2017-06-09 | 50714
2017-06-10 | 52170
2017-06-11 | 30814
------------------------
bucket item counts:
`content`: 3,274,788
`scores`: 767,505
------------------------
Service 'indexer' exited with status 1. Restarting. Messages: runtime.goexit()
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc824ce7f90 sp=0xc824ce7f88
created by github.com/couchbase/indexing/secondary/queryport.(*Server).listener
/home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/server.go:119 +0xc7
[goport] 2017/06/11 06:27:06 /opt/couchbase/bin/indexer terminated: signal: aborted (core dumped)
Service 'indexer' exited with status 1. Restarting. Messages: runtime.goexit()
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82401cfb0 sp=0xc82401cfa8
created by net/http.(*Transport).dialConn
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/net/http/transport.go:853 +0x10a6
[goport] 2017/06/11 04:38:38 /opt/couchbase/bin/indexer terminated: signal: aborted (core dumped)
Service 'indexer' exited with status 1. Restarting. Messages: runtime.goexit()
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82114cfb8 sp=0xc82114cfb0
created by github.com/couchbase/indexing/secondary/indexer.(*statsManager).tryUpdateStats
/home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/stats_manager.go:463 +0x2b2
[goport] 2017/06/10 19:27:00 /opt/couchbase/bin/indexer terminated: signal: aborted (core dumped)
crash times from the last day:
6:27:06 AM Sun Jun 11, 2017
4:38:38 AM Sun Jun 11, 2017
3:33:59 AM Sun Jun 11, 2017
3:09:25 AM Sun Jun 11, 2017
2:08:39 AM Sun Jun 11, 2017
9:56:26 PM Sun Jun 11, 2017
7:27:00 PM Sat Jun 10, 2017
7:18:15 PM Sat Jun 10, 2017
indexer.zip (180.8 KB)
error.zip (54.7 KB)
babysitter.zip (1.2 MB)
It seems to be only the one node out of 3.
The only hardware difference I can see is that this node has 4x4GB Kingston 4GB DDR3 1Rx8 instead of 2x8GB Kingston 8GB DDR3 2rx8 which shouldn’t be an issue.
I’ve been up for days already trying to debug this while stopping and starting the couchbase-server service to get the node back up when it falls. I’d love some help so I can get some sleep 
Please let me know if there’s any other information I can provide.