Key Size Limits with CouchBase/membase (again!)

.NET SDK
1

Hi, is there any plan to re-configure key sizes in excess of 250? I know it was originally based on ASCII limits and standards, but that seems very antiquated as hardware get more powerful, and other products (velocity and built in .net cache?) support beyond this limit, Couchbase could break this limit?

The use case I have problem with is that SQL’s often become the key for caching, and some SQL could get lengthly. File paths for files could also become cache. The only 2 options I’ve seen are listed below, but both seem limited.

  1. Hashing keys – dealing with collision is like hunting down a bug in camouflage. It’s not really an ideal fix.

  2. Re-storing the cache key – others have suggested creating hash and using the hash to store a shortened version. This could work, except in some benchmarking, the (TCP) overhead of going back and fourth just to translate long to short keys is very noticable.

Does anyone have a better solution?

Some others have complained…

2

We actually huffman-encode our keys on the way in to get around the problem. It allows us to extend the plain-text key size up to about 1000 characters or so (our keys have enough patterns to make huffman work quite well). At the obvious cost of having to compress and decompress the keys on the way in and out and less obvious annoyance of having the keys being stored as complete gibberish. I would love for the key size limit to be increased so I could throw those libraries away.

3

There aren’t any current plans to extend the key size. The current implementation as of this writing (2.1) would be very memory-expensive with larger and larger keys, but there are some features coming that should lower the cost. The issue “MB-8596 - support keys longer than 250 bytes” tracks this request for enhancement. You may want to track it there.