Query index has no pending mutations, yet the query returns no result

[JesusTheHun]

Pretty much the title.
I’m performing a GET http://hostname:9102/api/v1/stats/myBucket._default.myCollection/myIndex and I check num_docs_pending and num_docs_queued.

Both are 0 and yet there is a brief moment during which a query that uses that index returns no results.

If I wait 100ms for example, the query returns the expected documents.

I don’t know what else to check to make sure the docs are indexed. Any ideas ?

[mreiche]

Use QueryScanConsistency = REQUEST_PLUS

I’m performing a GET http://hostname:9102/api/v1/stats/myBucket._default.myCollection/myIndex and I check num_docs_pending and num_docs_queued .

That’s only going to work if hostname:9102 has the updated values.

[JesusTheHun]

I don’t want to use request_plus because it’s within a test. I want to assert the results, but I don’t want request_plus in production.

@mreiche

That’s only going to work if hostname:9102 has the updated values.

Can a single node has stale data about this ?

[mreiche]

@JesusTheHun

I don’t want to use request_plus because it’s within a test. I want to assert the results, but I don’t want request_plus in production.

Use separate query before your test query that uses REQUEST_PLUS to guarantee the indexing is complete. That’s what I do for tests where I don’t want to use REQUEST_PLUS on the actual query.

Can a single node has stale data about this ?

9102 is the indexer node - so if there is only one node then that should have current data (and yet it apparently doesn’t). I wonder if it is possible that when the read of /api/v1/stats/myBucket._default.myCollection/myIndex occurs, that it has not yet added the document(s) to it’s queue. The kv operations don’t wait for that. If you are inserting, try checking “num_docs_indexed” before inserting and then wait until it has increased by one.

[JesusTheHun]

@mreiche

Use separate query before your test query that uses REQUEST_PLUS to guarantee the indexing is complete. That’s what I do for tests where I don’t want to use REQUEST_PLUS on the actual query.

Clever. I’m gonna do that, because like you said the indexer data have some delays (understandable). Testing the num_docs_indexed is one way to workaround it, but I would have liked a method that gives me the remaining mutations in a reliable manner. I’m gonna open a ticket so they add a field last_stats_update in the response, so we can poll the data and check last_stats_update > time of the first poll. This way we make sure the stats take into account mutations done before the call.