Failing unit test from slow N1QL query

Node.js SDK
,
1

I built an API that works with couchbase, using node.js and express. I am having an issue when I run my tests. In my test script I start out with defining a test for a POST request, that adds a document to a bucket. Then I have a GET request test that retrieves all the documents in the bucket and another GET request test that retrieves a document by its id. The test for the GET request that retrieves a document by id passes, but the test which retrieves all the documents in the bucket fails.

The difference between these two requests is that the GET request that retrieves all the documents in a bucket uses an N1QL query while the GET request that retrieves documents by id does not.

The issue here is due timing. For example if I add a for loop that iterates over a million numbers, just to waste time, right before the test for getting all the documents in the bucket, the test will pass.

Initially I thought that maybe the document, which is sent by the POST request test, is not being added to the database in time, before the ‘GET all documents’ test is executed. But this is not the problem since the ‘GET document by id’ test is passing.

This makes me think that the issue is being caused by couchbases querying system and that the Query Node in couchbase server is taking too long to recognize that the new document has been added to the database.

I was wondering if anyone else has run into a similar issue before? If yes how did you solve it? Also, I was wondering if their might be a better way for retrieving all the documents from a bucket, without using an N1QL query?

2

Hey @sergeiDev,

You have indeed discovered the real reason for the failure of your getAllDocuments test. Couchbase’s indexing system is eventually consistent, this means that index updates are performed after the KV store itself is done. The solution to this problem is a concept known as scan consistency which allows you to define the consistency requirements of your query. You can get consistency varying from ‘give me whatever you have’ (default), ‘give me everything up to do date as of this query being run’ (request_plus) to ‘just make sure you have X,Y,Z mutations included’ (this is mutation token based consistency).

Cheers, Brett

1 Like
3

Thank you! That solved the issue.