DocumentChangeListener getting dropped?

Mobile Couchbase Lite
1

Something weird is happening with a document change listener - I’m wondering if anyone has some guidance.

I have a view that, when it is displayed, has a set of data it watches and reacts to.
There is one core document, and also a set of other document.
I used to use 2 queries with a change listener on each. That was fine.

Since one of the queries just watches a single document (a hangover from v1), I thought it made sense to switch that to a db.AddDocumentChangeListener. What I’m finding is that rarely (during long running automated tests), I am occasionally getting a failure when I close the view and cleanup the listeners.

18:27:31.050618| ERROR: Assertion failed: i != observers.end() (/Users/jenkins/jenkins/workspace/couchbase-lite-core-macosx/couchbase-lite-core/LiteCore/Database/SequenceTracker.cc:296, in removeDocChangeNotifier)
	 0  libLiteCore.dylib         libLiteCore.dylib + 57392
	 1  libLiteCore.dylib         c4docobs_free + 74
	 2  ???                       0x0 + 377316445
	 3  ???                       0x0 + 234612243
	 4  ???                       0x0 + 327917324
	 5  ???                       0x0 + 327916707
	 6  ???                       0x0 + 218871532
...

at <unknown> <0xffffffff>
  at (wrapper managed-to-native) LiteCore.Interop.Native.c4docobs_free (LiteCore.Interop.C4DocumentObserver*) [0x00009] in <78dfe9efed564df89e9ff941d2f5e0b3>:0
  at LiteCore.Interop.LiteCoreImpl.c4docobs_free (LiteCore.Interop.C4DocumentObserver*) [0x00000] in C:\Jenkins\workspace\couchbase-lite-net-edition-build\couchbase-lite-net-ee\couchbase-lite-net\src\LiteCore\src\LiteCore.Shared\Interop\LiteCore_impl.cs:150
  at Couchbase.Lite.Interop.Native.c4docobs_free (LiteCore.Interop.C4DocumentObserver*) [0x00000] in C:\Jenkins\workspace\couchbase-lite-net-edition-build\couchbase-lite-net-ee\couchbase-lite-net\src\LiteCore\src\LiteCore.Shared\Interop\LiteCore_shell.cs:152
  at Couchbase.Lite.Database/<>c__DisplayClass72_0.<RemoveChangeListener>b__0 () [0x000be] in C:\Jenkins\workspace\couchbase-lite-net-edition-build\couchbase-lite-net-ee\couchbase-lite-net\src\Couchbase.Lite.Shared\API\Database\Database.cs:776
  at Couchbase.Lite.Support.ThreadSafety.DoLocked (System.Action) [0x00011] in C:\Jenkins\workspace\couchbase-lite-net-edition-build\couchbase-lite-net-ee\couchbase-lite-net\src\Couchbase.Lite.Shared\Support\ThreadSafety.cs:39
  at Couchbase.Lite.Database.RemoveChangeListener (Couchbase.Lite.ListenerToken) [0x00014] in C:\Jenkins\workspace\couchbase-lite-net-edition-build\couchbase-lite-net-ee\couchbase-lite-net\src\Couchbase.Lite.Shared\API\Database\Database.cs:760
  at AppleLibrary.SharedNoteController.StopWatchers () [0x00067] in /Users/Paul/Dev/AppleLibrary/SharedNoteController.cs:165
...

The lines in question are below.
Setting up the listener:

    docHandlerToken = database.AddDocumentChangeListener(doc_id, UpdateDocument);
    docHandlerActive = true;

Removing the listener

    if (docHandlerActive)
        try
        {
            docHandlerActive = false;
            database.RemoveChangeListener(docHandlerToken);
        }
        catch (Exception e)
        {
            log.Error(e, "Exception removing doc change listener");
        }

It seems that the document handler token is getting invalidated.
What could cause invalidation of a handler token? How can I test for an invalidated change listener? I can’t see an API for that.

Thanks for any advice you can give.
Paul.

2

I suppose this might happen if you happened to call remove with the same token twice. Basically this is saying that the change listener from the token you passed is not registered with the database and therefore cannot be found for removal. Off the top of my head I can think of one way this might happen. The logic you have presented for removal is not thread safe and so if multiple threads call it, it could lead to an intermittent situation like you describe. But without a reproduction case I can’t be sure.

3

Also it may already be fixed at this point. A similar bug was already reported.

EDIT Three developer builds for 2.5 are out which should contain this fix so if you try those things should improve.

4

@borrrden
Re: not thread safe…
Yeah - I was worried about that too. So I have the “StopWatchers” routine wrapped in a Mutex:

public virtual void StopWatchers()
{
    WatcherMutex.WaitOne();
    try
    {
        // listeners and timers all shut down in here...
    } 
    finally
    {
        WatcherMutex.ReleaseMutex();
    }
5

That trace looks very like what I am seeing, thanks for pointing it out.
It doesn’t explain why my listener is getting invalidated or lost, but getting that code to ignore invalid tokens will help.
Being able to check the state of a listener would be useful too (e.g. access to the observers list?)

6

This is a known bug that will be fixed in an upcoming patch release (2.1.5.) Until then, you’ll need to avoid removing document-change listeners. Sorry!

7

The reasons for that particular bug are not related to your code. It was an internal problem with the way that they were being tracked.

8

@jens, @borrrden
Thank you for clarifying this.
For the moment… its probably better to switch back to a query listener - otherwise if I leave the doc change listener about, I’ll get errors relating to observers when I attempt to delete the database from within the app.