LiveQuery Changed event on *all* changes

Mobile
1

Hi,

I’m hoping I’m doing something really dumb here.
I have a C# program (currently on Mac using Xamarin).
I use a local couchbase mobile DB, connecting to Sync Gateway on ubuntu.

Weird thing is that every time I change any record locally, I get all my LiveQueries ‘changed’ events firing.
For example: I have a singleton user record (‘type’ field == “User”). If I post a change to that using:

document = database.GetDocument(id);
document.PutProperties(properties)

then all of my 3 live queries will fire - yet they are monitoring completely different data.
Here’s an example of one of the live queries:

I have a number of book objects (‘type’ == “Book”) - each of which contains a number of sections ('type == “Section”).
I have a live query watching the book, and a live query watching the current list of sections.

The view I use fetch sections for a given book is as follows. It looks for objects of type Section, and outputs their book_id for the index.

view = database.GetView("SectionsByBook");
view.SetMap((doc, emit) =>
{
    if (doc.ContainsKey("type") && (string)doc["type"] == "Section")
        emit((string)doc["book_id"], doc);
}, "2");

Now I can create a live query:

var query = database.GetView("SectionsByBook").CreateQuery();
query.StartKey = book_id;
query.EndKey = book_id;
var liveQuery = query.ToLiveQuery();

Lastly I hook up the update function.

liveQuery.Changed += UpdateSections;
liveQuery.Start();

In the function “UpdateSections” - I can see the correct list of sections being available.
The problem is that UpdateSections is called to any change to any record in the database,
such as a change to the User record. It should only be called on a change to the current list
of sections for the current book.

Any thoughts on what I’m doing wrong?
Many thanks.
Paul.

2

You aren’t doing anything wrong, that’s just the way that the live query changed event works. It doesn’t check the result set to see if it has actually changed (because the result is not entirely an in memory set, but rather an interator over a table in a file), it only informs you that the database has changed and that the query has been updated to reflect that (thus the underlying iterator has changed).

3

On iOS/Mac the notification is only posted if the results of the query did change. IMHO that’s better because the app’s response to the notification can be very expensive, like reloading and redrawing a table view.

As a workaround you can do what the Objective-c implementation does internally: collect the results into an array, then after the notification check whether the array has changed since the last time.

For 2.0 we should make sure we standardize the behavior across platforms.

4

On iOS/Mac the notification is only posted if the results of the query did change. IMHO that’s better because the app’s response to the notification can be very expensive, like reloading and redrawing a table view.

I’m finding that on the Mac (which is what I’m developing on), the notification is posted if anything in the database changes, not just the results of the query. Is that what is intended?

I thoroughly agree with your comment about not having the Change event fire unless the query results change. Otherwise the live query is of limited use, especially if I have to dig through several queries of data looking for changes when anything in the database changes.

5

Hi Borrrden,
I’m not sure live query is supposed to work the way you describe. From the documentation:

A Couchbase Lite View Query that automatically refreshes every time the Database changes in a way that would affect the results.

I think the intent (per Jens comment) is that you get a changed request when the query result changes.

Further, from the Guide:

A live query stays active and monitors the database and view index for changes. When there’s a change it re-runs itself automatically, and if the query results changed it notifies any observers

Thanks for the feedback on this.
Cheers.
Paul.

6

You’re developing on a Mac, but you’re using the .NET version of Couchbase Lite. (Confusing, I know.)

The iOS and Mac versions of Couchbase lite have an Objective-C API.

7

The iOS and Mac versions of Couchbase lite have an Objective-C API.

Aha! Gotcha. So the .NET version of LiveQuery on Mac, iOS, Windows will fire Changed events on any database change, the Obj-C versions of LiveQuery on Mac, iOS will fire Changed only when the query results change.
hmmmm… as you say, that probably should be aligned.

For 2.0 we should make sure we standardize the behavior across platforms.

Any idea when version 2.0 will be coming?
Thanks for your help in clarifying this.

As a by-the-by… I’ve found the following code in the Guide for checking for changes using isEqual on the QueryEnumerator. It simply keeps a complete copy of the query result table.

// Check whether the query result set has changed:
if (queryResult == null || queryResult.Stale) 
{
    QueryEnumerator newResult = query.Run();
    if (!queryResult.Equals(newResult))
    {
        queryResult = newResult;
        UpdateMyUserInterface();
    }
}

Aside from the bug that queryResult may be null when .Equals is called, this would seem to be the best way to watch for changes.

8

Later this year. We just today released an initial developer preview. It’s quite incomplete —only an Objective-C API, and no replicator yet — but a lot of our customers have asked to take an early look at it.

9

I don’t think the .Equals is working right.

Here’s what I’m using:

			newQueryData = <create a run a query>
			if (bookLiveQueryData == null || !bookLiveQueryData.Equals(newQueryData))
			{
				log.Debug("Definate update to book");
				bookLiveQueryData = newQueryData;
				Reload(bookLiveQueryData);
			}

When I step this in the debugger… I can see that bookLiveQueryData and newQueryData are identical. They each contain a single row with the same _rev value, plus the rest of the fields are the same.

However the .Equals function seems to always return false.

Weird. So I’m writing my own compare.

10

Great! I look forward to seeing it.
In the meantime… I’ll do it the hard way :unamused:

11

You could try LINQ’s SequenceEqual method instead, because Equals might just test if they are the same object in memory.

12

SequenceEqual is a good thought… but I don’t think I can guarantee the order of items from the DB (SequenceEqual tests the two IEnumerables in order). Or perhaps I can - they “should” come out in index order, right?

Secondly, it will use a default equality comparer for QueryRow. I haven’t tested that.

[Pause]

Ok - I tested the default equality comparer for QueryRow, and it too does not work.

13

I think you can override the default equality comparer when you use SequenceEqual, but yes they should come out in the same order.

14

Query results are always sorted by key (in JSON collation order.)

You probably need a custom equality operator for QueryRow objects. The Obj-C implementation has this so its LiveQuery implementation can compare successive query results.

15

You can use the overload that takes an equality comparison mechanism as the last argument as a workaround:

https://msdn.microsoft.com/en-us/library/bb342073(v=vs.110).aspx

16

@borrrden ,
That will work a treat. Thx.

@jens
A small wrinkle in this plan… Right now I don’t return the whole document as the value in my map function. This was after reading this article:

When to emit a whole document as the value? In some places you’ll see code that does something like emit(key, doc) , i.e. emitting the document’s entire body as the value. (Some people seem to do this by reflex whenever they don’t have a specific value in mind.) It’s not necessarily bad, but most of the time you shouldn’t do it. The benefit is that, by having the document’s properties right at hand when you process a query row, it can make querying a little bit faster (saving a trip to the database to load the document.) But the downside is that it makes the view index a lot larger, which can make querying slower. So whether it’s a net gain or loss depends on the specific use case. We recommend that you just set the value to null if you don’t need to emit any specific value.

So currently I use emit(key, null), and then use row.Document.Properties as required.

However, since row.Document.Properties always goes to the database to pull the document properties… the code above wont work because the bookLiveQueryData will get fresh properties every time I read them, rather than the saved/old ones. That is, the equality comparison between saved query data and *newQueryData returns true - they are always equal.

So it seems I have two choices:
a) Make a copy of the query enumerator and all documents + properties
b) Have the document be added to the View as the value in emit. Then the saved query data
will carry a copy of the document in Values.

Are there any recommendations around this? Using emit(key, doc) is fine for most data - but I will have to be careful because some of my data has large text fields (e.g. html docs).

One other option could be to return the _rev as the value - then its easy to detect changes.

I did read somewhere that the LiveQuery only looks for changes in the View key/value - which now makes sense to me. So if you are using LiveQuery… you need to return the doc as a value.

Thanks for your help.
Paul.

17

That sounds exactly like an issue I ran into implementing CBLLiveQuery in Objective-C. IIRC, the workaround I used was that, if the value property in the query row is null, the equality test compares the sequence property instead.

The best thing to do is to emit the specific document properties you’ll need when querying the view. That way you can do your work without having to load any documents from disk. (But I realize it’s not always practical to do that — often the app has code that operates on Document objects, and you end up needing to call that code when handling the query.)

18

Sorry for the spam… but I’m learning a lot here.

Turns out that AllDocumentsQuery uses _rev as the value in the emit statement.
It seems the right thing to do because then it is easy to check for changes
by simply looking at the revs.

BUT what AllDocumentsQuery does is create a NotNullDictionary<string, object> containing
{ ‘rev’: “the rev” }
If you return doc in emit(key, doc), then what you get is { “_rev”: “the rev”, … } (note the underscore).
If instead you try to return a Dictionary<string, object>() {“rev”: “the rev” }
in the emit to make it look the same as the AllDocumentsQuery value, you actually
get a Newtonsoft.JContainer in the Value field of the QueryRow.

So here’s my recommendation:
a) where possible, just return the “_rev” value (not a dictionary)
b) otherwise return the whole doc.
c) In the Equals(a QueryRow, b QueryRow) function,
you need to watch for all cases (here’s some pseudocode):

public static bool Equals< QueryRow >(a QueryRow, b QueryRow) {
if (a.Value != null)
{
    if a.Value is a string:
        a_rev = (string)a.Value
    if a.Value is a dictionary: 
        if a.Value contains "rev": 
            a_rev = (string)a.Value["rev"]
        else
            a_rev = (string)a.Value["_rev"]
} else {
    a_rev = a.Document.Properties["_rev"]
}
    //do the same for b
    // Now compare a's rev with b's rev.
}