Couchbase view index update hanging after adding file over ~1.4mb

.NET SDK
1

I’m using the latest Couchbase .Net SDK and I’m having problems with larger JSON-documents (over 1.4mb) in views.

In the case these documents were there when the design doc was added, the large files are just not shown in the view results (amount of documents given by view is smaller than actual amount).
If I add these larger files (using .Net client) later on, after design doc is created in the bucket, and getting the view with stale=false, the updating of the index never finishes and the method call does not return. I can see from the web UI that the index is being updated, but it never finishes. Eventually the GetView-call in .Net will give a timeout.
Restarting the Couchbase server did not help. Immediately when I remove the large document, GetView works again.

I have the following simple view:

function (doc, meta) {
emit(doc._id, [doc.name, doc.version, doc.type, doc.date]);
}

Adding documents up to 1.2mb works fine, but documents bigger than 1.45mb don’t. I could not find any documentation or actually anything online related views and larger documents. Is there some restriction to how big documents views can handle? I know Couchbase can’t handle documents bigger than 20mb’s, but the ones I use here are significantly smaller than that.

2

Hi the_jov,

The only restriction on document size is the 20MB max size. What version of the server are you using and what is the deployment environment (Win/linux/AWS, etc)?

Thanks,

Jeff

3

Thanks for your reply, jmorris.
I’m using the version 2.2.0 community edition (build-837) of Couchbase server under Windows 7.

4

I asked the same question on Stackoverflow and the answer there was that “There is a limitation in the indexer of 1MB per document; larger documents are skipped (and a message logged).”

5

the_jov -

That indeed is probably the issue you are running into here. Here are the two relevant tickets:

  • http://www.couchbase.com/issues/browse/MB-9467
  • http://www.couchbase.com/issues/browse/MB-11668

It appears that with larger documents (> 1MB) there are some side affects that impact re-balancing and indexing. If you must store larger documents than that, then you will need to avoid views and perhaps use some kind of deterministic key logic to perform iteration over those documents.

-Jeff

6

Ok then, thanks for the clarification! :slight_smile:
Well try splitting our documents into smaller ones to fix this issue.

7

the-jov -

There is a way of configuring the size of the document that CB can index. Read this for more info: http://www.couchbase.com/communities/q-and-a/how-get-list-all-documents-couchbase

This is what you are looking for I believe.

Thanks

Jeff