Operation times out

.NET SDK
1

I’m getting a lot of timeouts in my logs. (approx 1 every 20-30 requests)
I’m using SDK 2.5.8.0 on windows 10 .

This is my web.config:

  <couchbase username="xxxx" password="xxxx">
    <servers>
      <add uri="http://127.0.0.1:8091/pools"/>
    </servers>
  </couchbase>

These are the error messages:

2018-03-28 17:37:52.6924 DEBUG Couchbase.IO.SendTimeoutExpiredException: The operation has timed out., kv, 974e6a2de12a8b5c/88454c85f7240f5c/fa2, 127.0.0.1:60319, 15000, 127.0.0.1:11210
   at Couchbase.IO.MultiplexingConnection.Send(Byte[] request)
   at Couchbase.IO.Services.PooledIOService.Execute[T](IOperation`1 operation)
2018-03-28 17:37:52.6924 INFO Checking if node 127.0.0.1:11210 should be down - last: 17:37:38.1784636, current: 17:37:52.7013688, count: 4
2018-03-28 17:37:52.6924 DEBUG Operation doesn't support retries for key arrMenuIDs__UserID_1
2018-03-28 17:37:52.6924 DEBUG Operation for key arrMenuIDs__UserID_1 failed after 1 retries using vb327 from rev71 and opaque4002. Reason: The operation has timed out.
2

Hi @alon.schachter,

What version of Couchbase are you using? Are you running it on Windows or are you running within docker or a VM?

3

I’m on 5.0.1-5003-community
I’m running it on Windows.

I’ll just note in case it matters that i’m using an ASP interop wrapper for your .NET SDK

4

@alon.schachter -

Any chance that you can provide more log info? The exception is likely just the effect of something else happening before hand; also, read through this post - it sounds similar to what you are experiencing: Couchbase.IO.SendTimeoutExpiredException: The operation has timed out - #23 by clinton1ql

-Jeff

5

Here is an entire log collected in about 5 seconds with a few timeouts:

https://drive.google.com/file/d/0BwoJABE2FogsN20yLUh2RExsSHMzSk4yS0ZKOVZmMVpJT1FV/view?usp=drivesdk

6

@alon.schachter -

The problem is that the Cluster object is being created and disposed every couple of milliseconds. Both Cluster and Bucket objects should be scoped to the application’s lifetime: created and initialized when the app starts up and disposed/closed when the application shuts down. A single instance can be used concurrently by multiple threads.

If you are doing this:

foreach(var key in keys)
{
      using(var cluster = new Cluster())
      {
            using(var bucket = cluster.OpenBucket())
            {
                   //do work
            }
      }
}

Then you are using the SDK incorrectly; the cluster and bucket objects should be global to the loop. In the simplest case created just before the loop and disposed just after. In a real world application you’ll want to create/initialize from the main thread - in ASP.NET you can use Application_Start and Application_End in Globals.asax for example (there are other ways as well).

Note that the SDK will create connections up-front and cache them; this helps improve performance, in this case it looks like the cluster/bucket are being disposed/closed after every use or per request. You can read more about how to manage connections in the Couchbase Developer Documentation.

Hopefully this helps!

-Jeff

7

I am using classic ASP against the .NET SDK.
There is a ‘thread apartment’ issue when trying to store .NET com interops references in asp’s global.asa’s ‘Application_OnStart’ Application variables. So I don’t know if i’ll be successful creating a global instance.

How do I increase the ‘send operation timeout’ via my web.config? (my web.config is at the top of the thread)

Another “solution” i’m considering is skipping the COM interop all together and setting up a .NET web service instead, and communicating with it directly from ASP using http requests. That might be more efficient then recreating the cluster on every request

8

This is controlled by PoolConfiguration.SendTimeout; however this won’t fix your problem since you’ll still be in a bad state.

That definitly is an option.

Note that there is an older blog post that discusses using the SDK with COM Interop, so it’d definitely possible. Unfortunately the images in the blog didn’t seem to survive a conversion from our old blog system to our newer one. While not officially supported by Couchbase, there is a Github project here.

9

That’s the SDK I based my solution on and just updated it to the latest .NET SDK, but they also didn’t offer a way to solve the global instance issue. Their implementation method requires you to recreate the cluster on each request.

Anyway, I decided to go with the web service solution.

1 Like