Server Down on Windows 10

Hello,

I’ve been playing around with Couchbase Server (4.1.1-5914 CE) today. It’s the first time I’ve used it, and it’s been working nicely until only recently. For no apparent reason, the server went down (“Servers Down: 1” in the Cluster Overview, and the server marked as “Pend” in Server Nodes). I haven’t been taxing anything, just experimenting and looking at the SDKs.

Couchbase is installed on my laptop. I haven’t connected any other nodes to the cluster, it’s just a single instance.

Can anybody suggest a fix? I’m loathe to just reinstall, I’d like to know what caused the problem and whether it’s indicative of a serious issue.

Thanks in advance,

Graham

You’re not running Windows 10 Anniversary Update are you? That is incompatible with Couchbase 4.1.1.

There’s a dev preview of 4.5.1 EE for Windows 10 AU on the download page.

Yes, I’m running the anniversary update. I don’t have a Couchbase license, so can’t use the EE. We’re hoping that what we’re intending to develop will be able to use either the CE or the EE, depending on customer requirements.

But after rebooting the machine, the Couchbase server became active and is no longer down. It seems that restarting the service wasn’t sufficient, the machine itself needed restarting.

Thanks,

Graham

Version: 4.1.1-5914 Community Edition (build-5914)

I have a similar problem, but restarting the windows system can only work for a while, and then the couchbase Server will go down itself.

Output from cbcollect_info.exe, it seemed the memcached service crashed

C:\Program Files\Couchbase\Server\bin>cbcollect_info.exe output_file.zip
System information (systeminfo) - OK
Computer system (wmic computersystem) - OK
Computer OS (wmic os) - OK
Service list (wmic service where state="running" GET caption, name, state) - OK
Process list (wmic process) - OK
Process usage (tasklist /V /fo list) - OK
Swap settings (wmic pagefile) - OK
Disk partition (wmic partition) - OK
Disk volumes (wmic volume) - OK
Network configuration (ipconfig /all) - OK
Taking sample 2 after 10.000000 seconds - OK
Network status (netstat -ano) - OK
Network routing table (netstat -rn) - OK
Arp cache (arp -a) - OK
Network Interface Controller (wmic nic) - OK
Network Adapter (wmic nicconfig) - OK
Active network connection (wmic netuse) - OK
Protocols (wmic netprotocol) - OK
Hosts file (type %SystemRoot%\system32\drivers\etc\hosts) - OK
Cache memory (wmic memcache) - OK
Physical memory (wmic memphysical) - OK
Physical memory chip info (wmic memorychip) - OK
Local storage devices (wmic logicaldisk) - OK
Event log (wmic ntevent where "(LogFile='application' or LogFile='system') and EventType<3 and TimeGenerated>'20210706000000.000000-000'" get TimeGenerated,LogFile,SourceName,EventType,Message /FORMAT:list) - OK
Found dump-guts: .\dump-guts
Checking for server guts in C:\Program Files\Couchbase\Server\var\lib\couchbase\initargs...
product diag header () - OK
Found escript-wrapper: .\escript-wrapper
Found dump-stats: .\dump-stats
Result of query statement 'SELECT * FROM system:datastores' (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Adatastores']) - OK
Result of query statement 'SELECT * FROM system:namespaces' (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Anamespaces']) - OK
Result of query statement 'SELECT * FROM system:keyspaces' (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Akeyspaces']) - OK
Result of query statement 'SELECT * FROM system:indexes' (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Aindexes']) - OK
Index definitions are:  (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:9102/getIndexStatus']) - OK
Database directory structure (dir /s "C:\Program Files\Couchbase\Server\var\lib\couchbase\data") - OK
Index directory structure (dir /s "C:\Program Files\Couchbase\Server\var\lib\couchbase\data") - OK
Version file (type "."\..\VERSION.txt) - OK
Manifest file (type "."\..\manifest.txt) - OK
Manifest file (type "."\..\manifest.xml) - Exit code 1
Couchbase config () - OK
Couchbase static config () - OK
Raw ns_log () - OK
Memcached logs (cd "c:\Program Files\Couchbase\Server\var\lib\couchbase\logs" && for /f %a IN ('dir /od /b memcached.log.*') do type %a) - OK
Ini files (c:/Program Files/Couchbase/Server/bin/../etc/couchdb/default.ini) (type "c:\Program Files\Couchbase\Server\bin\..\etc\couchdb\default.ini") - OK
Ini files (c:/Program Files/Couchbase/Server/bin/../etc/couchdb/local.ini) (type "c:\Program Files\Couchbase\Server\bin\..\etc\couchdb\local.ini") - OK
Ini files (c:/Program Files/Couchbase/Server/bin/../etc/couchdb/default.d/geocouch.ini) (type "c:\Program Files\Couchbase\Server\bin\..\etc\couchdb\default.d\geocouch.ini") - OK
Ini files (c:/Program Files/Couchbase/Server/bin/../etc/couchdb/default.d/capi.ini) (type "c:\Program Files\Couchbase\Server\bin\..\etc\couchdb\default.d\capi.ini") - OK
Ini files (c:/Program Files/Couchbase/Server/bin/../etc/runtime.ini) (type "c:\Program Files\Couchbase\Server\bin\..\etc\runtime.ini") - OK
couchbase diags (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:8091/diag?noLogs=1']) - OK
master events (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:8091/diag/masterEvents?o=1']) - OK
ale configuration (['curl', '-sS', '--proxy', '', '-u', '@:*****', 'http://127.0.0.1:8091/diag/ale']) - OK
couchbase logs (debug.log) (cbbrowse_logs debug.log) - OK
couchbase logs (info.log) (cbbrowse_logs info.log) - OK
couchbase logs (error.log) (cbbrowse_logs error.log) - OK
couchbase logs (couchdb.log) (cbbrowse_logs couchdb.log) - OK
couchbase logs (xdcr.log) (cbbrowse_logs xdcr.log) - OK
couchbase logs (xdcr_errors.log) (cbbrowse_logs xdcr_errors.log) - OK
couchbase logs (views.log) (cbbrowse_logs views.log) - OK
couchbase logs (mapreduce_errors.log) (cbbrowse_logs mapreduce_errors.log) - OK
couchbase logs (stats.log) (cbbrowse_logs stats.log) - OK
couchbase logs (babysitter.log) (cbbrowse_logs babysitter.log) - OK
couchbase logs (ssl_proxy.log) (cbbrowse_logs ssl_proxy.log) - OK
couchbase logs (reports.log) (cbbrowse_logs reports.log) - OK
couchbase logs (xdcr_trace.log) (cbbrowse_logs xdcr_trace.log) - OK
couchbase logs (http_access.log) (cbbrowse_logs http_access.log) - OK
couchbase logs (http_access_internal.log) (cbbrowse_logs http_access_internal.log) - OK
couchbase logs (ns_couchdb.log) (cbbrowse_logs ns_couchdb.log) - OK
couchbase logs (goxdcr.log) (cbbrowse_logs goxdcr.log) - OK
couchbase logs (query.log) (cbbrowse_logs query.log) - OK
couchbase logs (projector.log) (cbbrowse_logs projector.log) - OK
couchbase logs (indexer.log) (cbbrowse_logs indexer.log) - OK
couchbase logs (metakv.log) (cbbrowse_logs metakv.log) - OK
memcached stats all (['cbstats', '-a', '127.0.0.1:11210', 'all', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats allocator (['cbstats', '-a', '127.0.0.1:11210', 'allocator', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats checkpoint (['cbstats', '-a', '127.0.0.1:11210', 'checkpoint', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats config (['cbstats', '-a', '127.0.0.1:11210', 'config', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats dcp (['cbstats', '-a', '127.0.0.1:11210', 'dcp', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats dcpagg (['cbstats', '-a', '127.0.0.1:11210', 'dcpagg', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats ['diskinfo', 'detail'] (['cbstats', '-a', '127.0.0.1:11210', 'diskinfo', 'detail', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats ['dispatcher', 'logs'] (['cbstats', '-a', '127.0.0.1:11210', 'dispatcher', 'logs', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats failovers (['cbstats', '-a', '127.0.0.1:11210', 'failovers', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats ['hash', 'detail'] (['cbstats', '-a', '127.0.0.1:11210', 'hash', 'detail', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats kvstore (['cbstats', '-a', '127.0.0.1:11210', 'kvstore', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats kvtimings (['cbstats', '-a', '127.0.0.1:11210', 'kvtimings', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats memory (['cbstats', '-a', '127.0.0.1:11210', 'memory', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats prev-vbucket (['cbstats', '-a', '127.0.0.1:11210', 'prev-vbucket', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats runtimes (['cbstats', '-a', '127.0.0.1:11210', 'runtimes', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats scheduler (['cbstats', '-a', '127.0.0.1:11210', 'scheduler', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats tap (['cbstats', '-a', '127.0.0.1:11210', 'tap', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats tapagg (['cbstats', '-a', '127.0.0.1:11210', 'tapagg', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats timings (['cbstats', '-a', '127.0.0.1:11210', 'timings', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats uuid (['cbstats', '-a', '127.0.0.1:11210', 'uuid', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats vbucket (['cbstats', '-a', '127.0.0.1:11210', 'vbucket', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats vbucket-details (['cbstats', '-a', '127.0.0.1:11210', 'vbucket-details', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats vbucket-seqno (['cbstats', '-a', '127.0.0.1:11210', 'vbucket-seqno', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats warmup (['cbstats', '-a', '127.0.0.1:11210', 'warmup', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached stats workload (['cbstats', '-a', '127.0.0.1:11210', 'workload', '-b', '_admin', '-p', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX']) - Exit code 255
memcached mcstat connections (['mcstat', '-h', '127.0.0.1:11210', '-u', '_admin', '-P', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', 'connections']) - Exit code 1
ddocs for default (C:/Program Files/Couchbase/Server/var/lib/couchbase/data\default\master.couch.2) (['couch_dbdump', 'C:/Program Files/Couchbase/Server/var/lib/couchbase/data\\default\\master.couch.2']) - OK
ddocs for gamesim-sample (C:/Program Files/Couchbase/Server/var/lib/couchbase/data\gamesim-sample\master.couch.1) (['couch_dbdump', 'C:/Program Files/Couchbase/Server/var/lib/couchbase/data\\gamesim-sample\\master.couch.1']) - OK
replication docs (C:/Program Files/Couchbase/Server/var/lib/couchbase/data\_replicator.couch.1) (['couch_dbdump', 'C:/Program Files/Couchbase/Server/var/lib/couchbase/data\\_replicator.couch.1']) - OK
mctimings (['mctimings', '-u', '_admin', '-P', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', '-h', '127.0.0.1:11210', '-v']) - Exit code 1
stats archives (['escript.exe', '.\\escript-wrapper', '--initargs-path', 'C:\\Program Files\\Couchbase\\Server\\var\\lib\\couchbase\\initargs', '--', '.\\dump-stats', 'c:/Program Files/Couchbase/Server/var/lib/couchbase/stats']) - OK
Traceback (most recent call last):
  File "cbcollect_info", line 1016, in <module>
    main()
  File "cbcollect_info", line 985, in main
    runner.collect_file(dump)
  File "cbcollect_info", line 167, in collect_file
    self.files[filename] = open(filename, 'r')
IOError: [Errno 13] Permission denied: 'c:/Program Files/Couchbase/Server/var/lib/couchbase/crash\\XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.dmp'

Error logs in the log panel

==Event==
Control connection to memcached on 'ns_1@127.0.0.1' disconnected: 
{
  {badmatch,{error,timeout}},
  [
    {mc_client_binary,cmd_vocal_recv,5,[{file,"src/mc_client_binary.erl"},{line,156}]},
    {mc_client_binary,select_bucket,2,[{file,"src/mc_client_binary.erl"},{line,351}]},
    {ns_memcached,ensure_bucket,2,[{file,"src/ns_memcached.erl"},{line,1299}]},
    {ns_memcached,handle_info,2,[{file,"src/ns_memcached.erl"},{line,748}]},
    {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,604}]},
    {ns_memcached,init,1,[{file,"src/ns_memcached.erl"},{line,177}]},
    {gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},
    {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}
  ]
} (repeated 1 times) | 

==Module Code==
ns_memcached000

==Server Node==
ns_1@127.0.0.1

Just find a workaround:

  1. kill the memcached service(if any)
  2. restart it via command below, this is just like reboot your system, it should be able to work for a while.
C:\Program Files\Couchbase\Server\bin>memcached.exe -C "C:\Program Files\Couchbase\Server\var\lib\couchbase\config\memcached.json"

A better workaround may be:

  1. run is via docker Couchbase Server and Windows 10 Anniversary Edition Problems - RandomDev
  2. uninstall 4.1-CE, install 4.5-CE or higher