High Memory Usage in NetClient 2.7.10

[kaiohenrique]

Hello, we are curious about the memory usage in one of our apis. We are currently using Couchbase NetClient 2.7.10.
Seems that exists 90 objects of type System.Byte allocated in memory, we believe is due to Connection buffer. Is that correct? Our cluster has 3 nodes and the ThreadPool is configured to instantiate 30 connections. Will client create 30 connections to each node? Will buffer size increase, besides “BufferSize” configuration? Is connection buffer size limit the server document size limit?

Another question:

couchbase/couchbase-net-client/blob/release27/Src/Couchbase/IO/SharedConnectionPool.cs#L95

As we do not check Pool Size before calling CreateAndAuthConnection method, can the pool size be extrapolated?

[jmorris]

@kaiohenrique

Seems that exists 90 objects of type System.Byte allocated in memory, we believe is due to Connection buffer. Is that correct?

Correct, buffers our allocated for each connection.

@kaiohenrique

Our cluster has 3 nodes and the ThreadPool is configured to instantiate 30 connections. Will client create 30 connections to each node?

For each node and bucket opened a unique connection is opened. This matches the PoolConfiguration.MaxSize setting.

@kaiohenrique

Will buffer size increase, besides “BufferSize” configuration? Is connection buffer size limit the server document size limit?

Yes, I do believe this can happen as the buffer size exceeded.

-Jeff

[btburnett3]

@kaiohenrique

Yes, if you have your connection pool set to 30 connections then it will make 30 connections * the number of data nodes. However, that many connections are rarely needed. The connections are multiplexed, allowing multiple simultaneous requests per connection, so you should probably have a smaller pool.

You could also look at moving to SDK 3.1.1. This version of the SDK introduced some changes to connection buffers, they now use System.IO.Pipelines. This allows it to use a reusable pool of chunked buffers from the ArrayPool. This should reduce the memory pressure you experience, except under high load with large documents.