Automatic Updates to Map-reduce Views

Couchbase Server
1

Hi we are currently testing an application using Map-reduce views and what we are finding is that documents are taking a long time to appear in view results (or even not appearing at all) when we use stale=true in the view query.

Reading this page:

http://www.couchbase.com/docs//couchbase-manual-2.0/couchbase-views-operation-autoupdate.html

it seems that the view should be updated with the new documents according to 1 or more of these parameters:

{
“updateInterval”:5000,
“updateMinChanges”:5000,
“replicaUpdateMinChanges”:5000
}

If I understand correctly then the longest we’d expect to wait with these settings is 5 secs. We are seeing delays of several minutes and are often having to execute a stale=false query to force the refresh. We are seeing this behaviour in multiple environments and multiple versions (2.0.1 and 2.1.0). Can someone clarify what is going on here - i.e. is this to be expected or not. If yes, is there something else we need to do to ensure that auto updating is working?

2

You are correct, the view index should be updated based on one of these conditions.

But, for this the document needs to be on disk, can you check that your disk write queue is empty?

Tug

3

I now have a simple reproducible example on my local development instance (single node, 2.1.0 enterprise edition (build-718)).

  1. Create a bucket with a simple view (default map function, reduce = _count)
  2. Add documents via web interface. I can see the items being persisted to disk in the disk updates counter and the disk write queue is empty.
  3. Query view with stale = ok - e.g. http://localhost:8092/test-bucket/_design/test1/_view/test1?stale=ok&connection_timeout=60000&limit=10&skip=0
  4. View is not updating after several minutes.
  5. If I query with default stale setting (update after) and then re-query, I see the view updated.

Note also that we are not seeing this issue on another (test) environment.

4

UPDATE: having spoken with CB support, it seems that both the updateInterval AND updateMinChanges conditions must be met (i.e. every 5 secs, a check will be made to see if the updateMinChanges limit has been reached). Our confusion was that we assumed that an update would occur when 1 OR the other condition was met.

5

Hello,

So I had the same understanding than you… this is why I commented this way.

I looked closely to the documentation again and it is effectively explained clearly; see: couchbase-manual-2.2

updateInterval : the time interval in milliseconds, default is 5000 milliseconds. At every updateInterval the views engine checks if the number of document mutations on disk is greater than updateMinChanges. If true, it triggers view update. The documents stored on disk potentially lag documents that are in-memory for tens of seconds.

Sorry for the confusion

Regards
Tug
@tgrall