How to set operationTimeout in Couchbase NodeJS SDK 3X?

Node.js SDK
, ,
1

Hi. I used CouchBase NodeJS SDK 2.6.10 previously and setting operationTimeout was something like this:

const couchbase = require(‘couchbase’)
var cluster = new couchbase.Cluster(‘couchbase://XXXXXXXXXXXXXXXXXXX’);
cluster.authenticate(‘USERNAME’, ‘PASSWORD’)
var bucket = cluster.openBucket(‘statistics’)
bucket.operationTimeout = 3600000

But, now in NodeJS SDK 3.0.4, it’s a bit different, like:

const couchbase = require(‘couchbase’)
const cluster = new couchbase.Cluster(‘couchbase://XXXXXXXXXXXXXXXXXXX’, {
username: ‘USERNAME’,
password: ‘PASSWORD’
})
const bucket = cluster.bucket(‘statistics’)
const collection = bucket.defaultCollection()

Here, I’m not finding any option to set operationTimeout from. Can anybody help me about this?

2 Likes
2

you seem to be correct, I do see query level timeouts but dont see a cluster level timeout options. I have reached out to the engineers to see if I am missing something. If not will open a ticket to get it addressed.

3

Thank you @AV25242 sir for your consideration and please keep me updated if you find something. I’ll let you know if I find a solution myself.

4

Hello @kaiserindrajit first of all apologies for the delay as far as Timeout Settings is considered use the following ,

const myCluster = couchbase.connect('couchbase://…', {
    operationTimeout: ...,
})
OR
const myCluster = couchbase.connect('couchbase://...?operation_timeout=2500');

For other timeouts in the meanwhile you can check the client settings for C SDK

1 Like
5

Thanks Arun. I’m using collection.get instead of couchbase.connect. Is this the proper usage for collection.get? Also, this might warrant another thread, but are there reasons (e.g. security or performance) to use couchbase.connect over collection.get?

const cluster = new couchbase.Cluster(“couchbase:///”, {
username:[userid],
password: [password],
operationTimeout: [timeout-in-millis]
});
const bucket = cluster.bucket([bucket]);
const collection = bucket.defaultCollection();
const key = [key];
const result = await collection.get(key);

6

Hi @brankin welcome to the forums.

On your first question about correct usage I believe you are asking if a timeout can be passed as an option upon connect vs get. ?

The default timeout values are suitable for most environments, and should be adjusted only after profiling the expected latencies in your deployment environment. If you get a timeout exception, it may be a symptom of another issue; increasing the timeout duration is sometimes not the best long-term solution.

Most timeouts can be overridden on a per-operation basis (for example, by passing a custom options block to a “get” or “query” method)

Also, this is how you set the timeout on a get operation.

https://docs.couchbase.com/nodejs-sdk/current/howtos/kv-operations.html#retrieving-full-documents

In nutshell the reason for a Data Operation to support timeout options is to override the global settings.
Hope this helps !

1 Like
7

Than you @AV25242 sir. I really appreciate it. Last time I felt so bad about this problem. I tried to make the timeout work for analytics query for several days, so it doesn’t throw timeout error after 75sec. It didn’t work. So I stuck with the NodeJS SDK 2.6.10, where the timeouts actually worked. Today suddenly I started working again with SDK 3.1.1 for a solution. I found an amazing one. I opened this forum to share that and found your reply. Thank you very much. Yes, the solution you have shared here surely works with SDK 3.1.1. But looks like Couchbase SDKs no longer work with operationTimeout in 3.1.1. I’ll share my solution here as well

8

OK. So finally the solution is here with Couchbase NodeJS SDK 3.1.1. The coding approach isn’t much different than Couchbase NodeJS SDK 3.0.4, but it surely deals with that timeout problem. It has a problem though, but that can be easily handled.

The first notable change in SDK 3.1.1 is that ‘couchbase’ is no longer a constructor. So, we no longer connect to the cluster like this:

const couchbase = require(‘couchbase’)
const cluster = new couchbase.Cluster(‘couchbase://XXXXXXXXXXXXXXXXXXX’, {
username: ‘USERNAME’,
password: ‘PASSWORD’
})

Instead, we do:

const couchbase = require(‘couchbase’)
couchbase.connect(‘couchbase://XXXXXXXXXXXXXXXXXXX’, {
username: ‘USERNAME’,
password: ‘PASSWORD’
})
.then(cluster => {
// DO WHATEVER
})
.catch(err => {
// AGAIN, DO WHATEVER
})

Now, to set timeout with cluster connection, we can now pass the timeout parameters alongside username and password. So, we can now write the code like:

const couchbase = require(‘couchbase’)
couchbase.connect(‘couchbase://XXXXXXXXXXXXXXXXXXX’, {
username: ‘USERNAME’,
password: ‘PASSWORD’,
kvTimeout: 3600000,
kvDurableTimeout: 3600000,
viewTimeout: 3600000,
queryTimeout: 3600000,
analyticsTimeout: 3600000,
searchTimeout: 3600000,
managementTimeout: 3600000
})
.then(cluster => {
// DO WHATEVER

cluster.analyticsQuery(QUERY_STRING, { …OPTIONS })
.then((err, rows) => {
if(err)
throw err
else
console.log(rows)
})
.catch(err => console.error(err))
})
.catch(err => {
// AGAIN, DO WHATEVER
})

And THIS WORKS. This surely does. This way, there’s no more timeout error after 75sec. Analytics queries on big datasets can easily be run for hours.
But there’s a little problem. If we want to open a bucket to execute a certain operation and afterwards want to run this analytics, it fails. Timeout no longer works for analytics. For example,

const bucket = cluster.bucket(BUCKET_NAME_STRING)
const collection = bucket.defaultCollection()

collection.get(DOCUMENT_KEY, (err, res) => {
if(err)
throw err
else{
cluster.analyticsQuery(QUERY_STRING, { …OPTIONS })
.then((err, rows) => {
if(err)
throw err
else
console.log(rows)
})
.catch(err => console.error(err))
}
})

In this above piece of code, analytics will face a timeout after 75sec. So basically, all the timeout options we passed with username and password, no longer work. To find the reason, when I read the Couchbase package files from within node_modules directory, I found this:

Whenever we connect to a bucket for a certain operation [i.e., const bucket = cluster.bucket(BUCKET_NAME_STRING)], the SDK redefines the connection without passing the timeout parameters.
The cluster.bucket method inside node_modules/couchbase/lib/cluster.js file returns Bucket constructer, which is defined inside node_modules/couchbase/lib/bucket.js
The Bucket constructor accepts 2 parameters: cluster object and bucketName string. When we call cluster.bucket method, we pass the bucketName only, which is passed down here. The Bucket constructor then calls the cluster._getConn method, with the parameter { bucketName: bucketName } only, which redefines the connection. This is the place, where timeout parameters get missing.

Solution:

I had to update 2 files in specific.

node_modules/couchbase/lib/cluster.js

I defined a class variable (object)
this._user_timeout_opts = {
kvTimeout: options.kvTimeout,
kvDurableTimeout: options.kvDurableTimeout,
viewTimeout: options.viewTimeout,
queryTimeout: options.queryTimeout,
analyticsTimeout: options.analyticsTimeout,
searchTimeout: options.searchTimeout,
managementTimeout: options.managementTimeout
}

just above this line:

this._connStr = connStr;

  1.  node_modules/couchbase/lib/cluster.js

I modified Bucket constructor’s this code-block
this._conn = cluster._getConn({
bucketName: bucketName
});

with this:
this._conn = cluster._getConn({
  bucketName: bucketName,
  ...cluster._user_timeout_opts
});

Now it doesn’t matter how many times we connect to different buckets, every time our timeout parameters (which we passed during connecting the cluster) will be passed within the new connections. Hope, this helps :slight_smile:

1 Like
9

@kaiserindrajit thank you for being a contributor and explaining it very well
@brett19 any comments ?