We still see extremely high connections on our nodes. This is just on one node. We have close to 200 pods. What is the expected number of connections per pod including all the Couchbase internal services that hold connections?
Your netstat command is counting all connections - even non-couchbase ones. The couchbase ports are listed in the connections on ports listed in Couchbase Server Ports | Couchbase Docs
The couchbase kv engine alone can support up to 60,000 connections, so 1000 connections should not an issue. According to the exchange above, the solution to the warning messages from your monitoring would be to increase the point where it emits warning messages.
What is the expected number of connections per pod including all the Couchbase internal services that hold connections
It depends on the number of nodes in the cluster, on the number of query, index, search services, xdcr, backups etc. I’m not sure it is useful to count them up for the purpose of reducing the number of connections as they aren’t configurable (as far as I know). To see what they are, one could start all the services without any external clients connecting, and count the connections on ports listed in Couchbase Server Ports | Couchbase Docs