I am interested in knowing / getting some inputs on how we get started with debugging an issue with Couchbase cluster, what is my first log file to look at.
How do I know that a particular problem is related to a particular service e.t.c. and look into this particular log will give me more details. Any inputs and experiences on this is highly appreciated.
Are there any Error Messages / Search Terms in particular that I can feed into a Log Monitoring system like Splunk etc generating an alert. I tried looking into documentation & on google but its not of much help.
Hi,
When you collect logs from a CB server, you will end up with a zip file with many logs in them. The best place to start will be diag.log. This log is pretty verbose and captures almost every issue that occurs in the server. You could search for terms like “Rebalance exited with reason” and “Service exited with status” in here to look for issues. We have a separate location where crashes and coredumps are collected: /opt/couchbase/var/lib/couchbase/crash. Every service has its own log files - ns_server.goxdcr.log, ns_server. indexer.log, ns_server.query.log etc. If you are running into issues with a specific services, you can look at that service specific log and search for terms like “panic” or “exited”. Here is some documentation on how to interpret logs: https://docs.couchbase.com/server/5.5/clustersetup/ui-logs.html
Hope this helps.
Best,
Arun