Cluster, ConnectTimeoutException for not connect to endpoint

Couchbase Server
1

Hi,
I have external machine that insert a document on my Couchbase cluster (with java sdk).
For a few minute one server node of the cluster was down (reboot for upgrade hw) and the log reported this exception:

18:09:41.910 [RxComputationScheduler-2] WARN Endpoint: [couchbase01-xxx/xxx:11210][KeyValueEndpoint]: Could not connect to endpoint, retrying with delay 4096 MILLISECONDS:
com.couchbase.client.deps.io.netty.channel.ConnectTimeoutException: Connect callback did not return, hit safeguarding timeout.
at com.couchbase.client.core.endpoint.AbstractEndpoint$3.call(AbstractEndpoint.java:317) ~[core-io-1.3.6.jar:na]
at com.couchbase.client.core.endpoint.AbstractEndpoint$3.call(AbstractEndpoint.java:310) ~[core-io-1.3.6.jar:na]
at rx.internal.operators.SingleOperatorOnErrorResumeNext$2.onError(SingleOperatorOnErrorResumeNext.java:69) [rxjava-1.1.8.jar:1.1.8]
at rx.Single$18.onError(Single.java:1883) [rxjava-1.1.8.jar:1.1.8]
at rx.observers.SafeSubscriber._onError(SafeSubscriber.java:152) [rxjava-1.1.8.jar:1.1.8]
at rx.observers.SafeSubscriber.onError(SafeSubscriber.java:115) [rxjava-1.1.8.jar:1.1.8]
at rx.observers.SerializedObserver.onError(SerializedObserver.java:153) [rxjava-1.1.8.jar:1.1.8]
at rx.observers.SerializedSubscriber.onError(SerializedSubscriber.java:78) [rxjava-1.1.8.jar:1.1.8]
at rx.internal.operators.OperatorTimeoutBase$TimeoutSubscriber$1.onError(OperatorTimeoutBase.java:187) [rxjava-1.1.8.jar:1.1.8]
at rx.Single$1$1.onError(Single.java:85) [rxjava-1.1.8.jar:1.1.8]
at rx.Single$3.call(Single.java:505) [rxjava-1.1.8.jar:1.1.8]
at rx.Single$3.call(Single.java:501) [rxjava-1.1.8.jar:1.1.8]
at rx.Single$1.call(Single.java:90) [rxjava-1.1.8.jar:1.1.8]
at rx.Single$1.call(Single.java:70) [rxjava-1.1.8.jar:1.1.8]
at rx.Observable.unsafeSubscribe(Observable.java:9861) [rxjava-1.1.8.jar:1.1.8]
at rx.internal.operators.OperatorTimeoutBase$TimeoutSubscriber.onTimeout(OperatorTimeoutBase.java:200) [rxjava-1.1.8.jar:1.1.8]
at rx.internal.operators.OperatorTimeout$1$1.call(OperatorTimeout.java:41) [rxjava-1.1.8.jar:1.1.8]
at rx.internal.schedulers.EventLoopsScheduler$EventLoopWorker$2.call(EventLoopsScheduler.java:189) [rxjava-1.1.8.jar:1.1.8]
at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) [rxjava-1.1.8.jar:1.1.8]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]

I think it’s normal, but the is problem that exception has blocked my external machine and I had to restart it.

2

It’s normal that operations destined for one machine can’t be completed while that machine is down. Other operations for other nodes will flow through just fine. We test that this behavior is correct for every release. Of course, there can be things at the application level that prevent you from issuing other ops to available nodes.

I think that’s what you’re seeing here, but let us know if you’re concerned about something else.

3

Thanks for the reply