Couchbase max parallelism

Couchbase Server
1

Hi,

I’m currently testing an application using Couchbase Server community edition, version 4.5.
I think I’m experiencing the max parallelism = 4 limitation set in the community edition, when I run several requests in parallel.
Do I have a way to confirm this (database log file, system query,?), in order to justify an upgrade to Enterprise Edition?

Thanks

2

max-concurrency = 4 limitation set in the community edition. For Enterprise edition it will be number of CPUs.
plus-servicers = 16max-concurrency (number of request-plus requests processed simultaneously )
servicers=4max-concurrency (number of unbounded requests processed simultaneously )

query.log will have similar line at the start.

_msg=cbq-engine started request-size-cap=67108864 timeout=0s loglevel=INFO plus-servicers=128 pipeline-cap=512 pipeline-batch=16 max-index-api=2 max-concurrency=8 servicers=32 request-cap=256

3

Did you set max_parallelism to 4? Why do you think, you are hitting that? How many cores does your system have?

You can check current query engine settings with following curl request.
varakurprasad$ curl -u Administrator:password -X GET http://localhost:8093/admin/settings
{“completed-limit”:4000,“completed-threshold”:1000,“controls”:false,“cpuprofile”:"",“debug”:false,“keep-alive-length”:16384,“loglevel”:“INFO”,“max-index-api”:2,“max-parallelism”:1,“memprofile”:"",“pipeline-batch”:16,“pipeline-cap”:512,“pretty”:true,“profile”:“off”,“request-size-cap”:67108864,“scan-cap”:512,“servicers”:32,“timeout”:0}
varakurprasad$

4

This page lists Enterprise and community edition feature matrix
https://developer.couchbase.com/documentation/server/4.6/introduction/editions.html

5

Thanks for you reply.
We have executed the command and we have max-parallelism set to 1. We will update it to 4 and repeat the tests.

Where can I find configuration recommendations to tune Couchbase?

Thanks

6

Could you please help on how to change max-parallelism to 4?
I’ve checked the options in couchbase-cli but I can’t find anything related with it, in particular with the option cluster-edit.

Thanks

7

max_parallelism makes each query run in parallel and improve the latency. If the CPU are already used max extent increasing the value may not help.

http://docs.couchbase.com/sdk-api/couchbase-java-client-2.2.4/com/couchbase/client/java/query/N1qlParams.html

curl -u Administrator:<password> http://<host>:8093/admin/settings -XPOST -d '{"max-parallelism":2}'

8

Thanks for the feedback!

9

Does changing this setting requires a reboot?

Thanks

10

No reboot is need. Each reboot/restart of query service the value is reset to default. You need to set again. Also you need to set it for each query service.

11

Max_parallelism controls how many threads can run in parallel concurrently for a SINGLE query.
Number of queries running in parallel is separate and is automatically controlled by the system resource and servicers parameter.

You can set max_parallelism as a query parameter and that’s a good practice than setting it at the service level. Not every query needs and will benefit from this.
See this for additional details on query execution: A Deep Dive Into Couchbase N1QL Query Optimization - DZone

/cc @prasad: please verify/improve docs on this.

1 Like
12

Thanks!
How can I change/control the number of queries running in parallel? max-concurrency parameter?

Thanks

13

You can set using following command. Couchbase Server community edition limits query service uses 4 operating system threads simultaneously.

curl -u Administrator:<password>; http://<host>:8093/admin/settings -XPOST -d '{"servicers":32}'

14

Thanks once again.
This limitation could explain the low performance we have when we increase the number of concurrent request (10 requests per second during 10 seconds).
Do I have a way to identify this limitation in the database with numbers or in a log file?

Thanks

15

You can find this in documentation
High Performance Concurrent Query Execution
Couchbase Server query service achieves great query throughput and query latency by taking advantage of large number of queries concurrently and by parallelizing query execution across multiple cores. Only Enterprise Edition comes with full parallelism and concurrency support. Community edition can only be deployed in homogeneous deploy model and comes with limited concurrency and parallelism (max parallelism can be 4) on each node.

https://developer.couchbase.com/documentation/server/4.6/introduction/editions.html

16

If you do have machine with larger number of cores, it’s perfectly fine to download the Enterprise Edition to run the non-production performance tests.
Please reach out if you need help with licensing issues: https://www.couchbase.com/request-pricing

17

Thanks. For example if I have a machine with just 4 cores, the limitation shouldn’t have any impact, right?
Will it use one thread for each core? A query with max_parallelism 1 will use only 1 thread?

Thanks

18

Query with max_parallelism 1 can use all 4 cores