We experience this error any time we are creating a document greater than ~32K. Please advise as to how we can eliminate issue
Error while encoding Request, cancelling. at com.couchbase.client.core.endpoint.AbstractGenericHandler.encode(AbstractGenericHandler.java:270) at com.couchbase.client.core.endpoint.AbstractGenericHandler.encode(AbstractGenericHandler.java:81) at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec$1.encode(MessageToMessageCodec.java:67) at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:88) at com.couchbase.client.deps.io.netty.handler.codec.MessageToMessageCodec.write(MessageToMessageCodec.java:116) at com.couchbase.client.core.endpoint.AbstractGenericHandler.write(AbstractGenericHandler.java:257) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:732) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:724) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:35) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1061) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1050) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:464) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at com.couchbase.client.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:745) Caused by: com.couchbase.client.core.RequestCancelledException: Error while encoding Request, cancelling. … 16 more Caused by: java.lang.RuntimeException: Could not snappy-compress value. at com.couchbase.client.core.endpoint.kv.KeyValueHandler.handleSnappyCompression(KeyValueHandler.java:307) at com.couchbase.client.core.endpoint.kv.KeyValueHandler.encodeRequest(KeyValueHandler.java:280) at com.couchbase.client.core.endpoint.kv.KeyValueHandler.encodeRequest(KeyValueHandler.java:131) at com.couchbase.client.core.endpoint.AbstractGenericHandler.encode(AbstractGenericHandler.java:268) … 15 more Caused by: java.lang.IndexOutOfBoundsException: index: -32711, length: 4 (expected: range(0, 37624)) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1118) at com.couchbase.client.deps.io.netty.buffer.SlicedByteBuf.getInt(SlicedByteBuf.java:153) at com.couchbase.client.core.endpoint.util.Snappy.encode(Snappy.java:131) at com.couchbase.client.core.endpoint.kv.KeyValueHandler.handleSnappyCompression(KeyValueHandler.java:305) … 18 more
Deferring to @daschl, it looks like a possible defect in our underlying dependency Netty. We happen to be in the process of updating that for a future maintenance release. If you build off the “release27” branch, you can probably try the work in progress, though I know we’re still in testing.
It seems unlikely that it’s just 32k size items alone-- there must be a related issue here. It could be with snappy. Question to you @Alex_Kaniaru: is it just create a 32k document, or are you using a special transcoder? Can you describe your environment-- anything we might want to know to reproduce it?
@ingenthur Thanks for your response. We are just passing-in JSON objects via the OrtusSolutions couchbase extension’s upsert method. It works fine for documents less that 32K but when we try any documents above that we encounter the error consistently post the extension, I did reach out to Ortus and it looks like their extension is fine since there are no hindrances to object size. I tried looking at the jar (cfcouchbase-2.5.5/lib/couchbase-core-io-1.5.5.jar) in the decompiler and it looks like the error is due to the snappy encode() method (com.couchbase.client.core.endpoint.util.Snappy.encode()). I’m not very conversant with the Java language but this as far as I got with my debugging. It may be the error is (or not) directly with the encode method itself or one of it’s passed in ByteBuf dependencies.
I hope this helps.
Thanks,
Alex
Hm, interesting. Any chance you can post a small project to github or the like that demonstrates the problem?
@Alex_Kaniaru which java SDK version are you using? (I assume 2.5.5, but double checking)
If you can try the latest releases there is a good chance this is already fixed (we switched to a different snappy implementation some time ago).
Hi @daschl
We are using the 2.5.5 SDK Version. It takes a while to update all our dev, staging and production servers. Do you know what the fix is and is there a patch we can apply in the meantime?
Thanks,
Alex
@Alex_Kaniaru sorry, 2.5 has been EOL’ed in March 2019 (Support Policy | Couchbase) and the change has been a bit involved, so there is no simple patch i can offer. But since we are not breaking APIs between versions you should be able to upgrade to 2.7.* without much problems.
@daschl Thanks for the information. It may be some time then before I can get back to you about whether it resolves the issue or not.