We are seeing a lot of timeouts at application startup. I have checked our file handle limits (they’re set at 100k and we have nowhere near that many opened (about 8k on the system). We also have 13GB available of RAM.
This occurs at startup, as I mentioned, and the application needs to be restarted to make it work. It’s hit or miss when it works successfully or not. It does not recover.
We saw this problem before but it largely went away - pretty sure I posted here and we never really resolved it. We are running Couchbase Server 4.5.1 and a variety of Couchbase Java client versions, but this most recently occurred with 2.4.6.
Thanks - H
Below is a stack trace:
java.util.concurrent.TimeoutException
at com.couchbase.client.core.utils.Blocking.blockForSingle(Blocking.java:74)
at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:343)
at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:327)
at com.couchbase.client.java.CouchbaseCluster.openBucket(CouchbaseCluster.java:316)
at com.company.couch2.CouchbaseClientManager.lambda$initializeBuckets$3(CouchbaseClientManager.java:160)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at com.company.couch2.CouchbaseClientManager.initializeBuckets(CouchbaseClientManager.java:154)
at com.company.couch2.CouchbaseClientManager.lambda$start$1(CouchbaseClientManager.java:87)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at com.company.couch2.CouchbaseClientManager.start(CouchbaseClientManager.java:71)
at io.dropwizard.lifecycle.JettyManaged.doStart(JettyManaged.java:27)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:118)
at org.eclipse.jetty.server.Server.start(Server.java:342)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:100)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:60)
at org.eclipse.jetty.server.Server.doStart(Server.java:290)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:43)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:76)
at io.dropwizard.cli.Cli.run(Cli.java:70)
at io.dropwizard.Application.run(Application.java:72)
at com.company.application.service.main(Service.java:292)