Running the cbq-engine on the same server as couchbase db

SQL++
1

Should the cbq-engine be running locally on the same device as the couchbase server or on a separate device. I know it depends on network/cpu/memory/disk IO/etc… But is there an advantage to running it locally?

Thanks

2

I strongly recommend that if you want to use it from the SDKs. For example the Java SDK has it as a requirement.

If you just want to use the shell or the browser you don’t have to.

3

I’m using php/curl to connect to it. PHP SDK 2.0 in not ready yet or NQ1L as I understand it.

Thanks much.

4

Hi Keith,

The REST API / curl does not require that they be on the same box. Can you try out both configs with your data and queries, and see the perf tradeoffs?

-Gerald

5

So I ran the cbq-engine in two different scenarios.

test 1: curl->localhost->cbq-engine->server2->couchbase
test 2: curl->server2->cbq-engine->couchbase
It seems to be faster running with test2. about a 1/2-1 second average faster.

Crazy.

6

Can you clarify. For each test, where is curl running? where is cbq-engine running? and where is couchbase server running.

Thx.

7

You bet.

Test 1
On server1 I’m running curl to a local cbq-engine which connects to server 2 that couchbase server is running on.
Or server1(curl->cbq-engine) - > server2(couchbase server)

Test 2
On server 1 I’m running curl which connects to a remote cbq-engine on server2 that connects to its local version of couchbase server
Or server1(curl) - > server2(cbq-engine->couchbase server)

8

The tradeoff is shipping documents over the network between Couchbase server and cbq-engine, vs. CPU and memory contention on one box shared by Couchbase server and cbq-engine.

Another consideration is whether your queries are mostly selective, or mostly touch many documents. Anyway, we are working to benchmark and characterize all these tradeoffs.

Thx,
Gerald

9

One of the reasons I was asked to look into N1QL was so we could avoid sending entire documents over the network as we would with other methods from the couchbase client (though I am unsure about using views). We are using single document databases right now, and the documents could get fairly long and we wouldn’t want to transport an entire document when all we needed was a single key-value pair. For this reason, keeping the query engine and server on separate machines would somewhat defeat the point.

Question: When you mention memory contention, Gerald, is that something that would be resolved somewhat when N1QL is fully worked into the server? Also, would there even be a way to separate the query engine and the server once it all comes together? I was under the impression that the query engine would work as a component of the couchbase server once it was all said and done.

10

Hi,

At some point, N1QL will be optimized to avoid shipping entire documents over the network. By going to N1QL now, you will benefit when this optimization is added.

The Couchbase server consists of multiple services; so even when fully integrated, N1QL will be a modular service that can be enabled on a subset of the nodes, to tune your cluster for your workloads.

-Gerald