Time out and Key was mapped issues

I have been trying to setup a 3 node couchbase server. I have indexs on all three servers and I’m still seeing a lot of connection errors when trying to upsert and select.

Here are the most common errors:
The operation has timed out.

and

The node XXX.XXX.XXX.XXX:11210 that the key was mapped to is either down or unreachable. The SDK will continue to try to connect every 1000ms. Until it can connect every operation routed to it will fail with this exception.

I have tried countless web config changes and have implmented the following code:

public class DataAccess : IDisposable
{
    /// <summary>Bucket that is being used for the application</summary>
    private readonly string bucketName = WebConfigurationManager.AppSettings["bucketName"];
    private LogHelper appLog = new LogHelper();

public int InsertUpdateCompany(Company companyDocument, out string error)
    {
IBucket bucket = null;
try
{
    bucket = ClusterHelper.GetBucket(bucketName);

var document = new Document<dynamic>
{
    Id = String.Format(CouchbaseConstants.DocumentID.Company, companyDocument.CleanName(), companyDocument.CompanyID),
    Content = companyDocument
};

var result = bucket.Upsert(document);
if (result.Success)
{
    success = 0;
}
else if (result.Status == ResponseStatus.OperationTimeout)
{
    result = bucket.Upsert(document, ReplicateTo.One);
    if (result.Success)
    {
        success = 0;
    }
    else
    {
        error = ServiceConstants.Response.InsertFail;
        success = -2;
    }
}
else
{
    error = ServiceConstants.Response.InsertFail;
    success = -2;
}
}
catch (Exception ex)
{
    error = String.Format(ServiceConstants.Response.Exception, ex.Message);
    success = -1;
}
return success
}

Here is how I access the code:

using (var dbAccess = new DataAccess())
{
    string _error = "";
    sucess = dbAccess.InsertCompany(this, out _error);
}

Currently the couchbase server is on Joyent and the WebServer is on Azure. I have the SetTCPKeepAlive and ClusterHelper.Initialize so I’m curious to know why I’m seeing this kind of performance and ways I can fix this issue

@schuranator -

What is your Dispose method doing (it doesn’t appear to be implemented in the snippet)? More so, what is the purpose of implementing IDisposable here?

Your DAL should probably take a IBucket reference (as a ctor parameter) and your ClusterHelper should be initialized within Application_Start and closed in Application_End (in Global.asax). So your DAL looks like this:

public class DataAccess
{
    IBucket _bucket;
    public DataAccess(IBucket bucket)
    {
         _bucket=bucket;    
    }

    ///use the bucket here
    public bool DoSomething(key, val){}
}

The OperationTimeout and NodeUnavailableException’s could be related to your app architecture or likely to your environment; I need more information like client logs to help with that.

-Jeff