Query time in Python sdk 2/3

Python SDK
,
1

Hi Team,

I am trying to evaluate how much time couchbase queries are taking to execute in both Python sdk 2 and 3.
In doing so, I am unable to understand the scenario listed below:

Python sdk 2.5.4

Code snippet:

print ('-----------------')
start = datetime.now()
result = []
query1_5 = "select * from `<bucket-name>` where type='<doc_type>'  \
            and <attr1_name>='<attr1_value>' and <attr2_name>='<attr2_value>' and <attr3_name>='<attr3_value>'"
res = cb.n1ql_query(query1_5)
for i in res:
      result.append(i)

print (f"Elapsed time - {res.metrics['elapsedTime']}")
print (f"Execution time is - {res.metrics['executionTime']}")

end = datetime.now()
print (f'total time is - {(end-start).total_seconds()*1000}ms')
print ('--------------------')

Output in shell:

Elapsed time - 6.498555ms
Execution time is - 6.369113ms
total time is - 53.985ms

Why total time of ~53ms is not getting included in the metrics dict of couchbase result object. Is my understanding of execution time incorrect here?

Kindly clarify!

Thanks,
Devansh

2

Hello,

Did someone get a chance to look into this? Apologies if question was not clear enough.
@ellis.breen @raju

3

The result is streamed from the server in chunks of JSON, so most likely the last chunk was retrieved from the server, along with the metrics, at 6.36ms, early on in the loop iterating through the rows. The remainder of the ‘total time’ you recorded would be, I suspect, due to the time taken for Python to iterate through the remainder.

4

Thank you @ellis.breen
Also, could you clarify - when does couchbase actually executes the query in the above code snippet- when we call n1ql_query() or when we iterate?

  • res = cb.n1ql_query(query1_5), or

  • for i in res

5

Hi, the query command is sent on the first call to the QueryResult’s iterator, so, the latter.
The iterator will not yield a result until the first HTTP response has been received from the server and decoded. There may be numerous chunks in larger queries, and the number of rows decoded per chunk will depend on the size of each row. There will likely be more Python runtime overhead per chunk on smaller rows, as more time is spent marshaling data instead of decoding JSON.

Hope that helps,

Ellis