We are using Couchbase .net SDK for retrieving a value from a bucket, and the following exception is occurring intermittently.
The node ****** that the key was mapped to is either down or unreachable. The SDK will continue to try to connect every 1000ms. Until it can connect every operation routed to it will fail with this exception.
Questions
Where can i configure the SDK to not retry that is happening for
every 1000ms, if we are failing we want to fail fast without that
retry happening for every 1000ms. Is the 1000ms configurable in the
client configuration ?
What version of the SDK are you using? There were some improvements related to connectivity in 2.2. I had similar issues on 2.1 that were corrected with an upgrade.
In this case, the operation will fail-fast; however, the SDK will try to reconnect with node every 1000ms by sending noop’s until it connects. The reason it does this is to not spend time sending operations to a node that is offline until it has gone back online.
That was the theory, in-practice TCP resets due to flakey networks and net-appliances (firewalls, LB’s, etc) can happen frequently creating a “false” positive. So, the behavior was changed in 2.2.X so that a certain number of failures will have to happen within a timespan before the node is considered offline or unreachable from the client (this doesn’t actually mean the physical node is down, it just means the client cannot connect).