Sync task seem to "block" UI thread on Android

I’m using CBLite 2.6.0, Community Edition 6.0.0 build 1693 and Couchbase Sync Gateway/2.6.0(127;b4c828d) CE.
Developing on Mac using Visual Studio 2019 - and Xamarin.Forms 4.4.0.xxxx (and same issue was on 4.3.0.yyyy)

I have developed an app that syncs with the server :slight_smile: - and when the sync. process is running on Android everything is very sluggish or non-responding. Once sync goes idle - everything starts working again. On iOS it just works smoothly…

I guess I must be doing something wrong in the way I start the replication task. But I have built it based on inputs from various examples and questions so I don’t really know what to do to fix this. In my service I have this method that I call on start:

        public async Task InitAsync()
    {
        var options = new DatabaseConfiguration();

        // Borrow this functionality from Couchbase Lite
        var defaultDirectory = Service.GetInstance<IDefaultDirectoryResolver>().DefaultDirectory();
        var dbFolder = Path.Combine(defaultDirectory, "data");
        if (!Directory.Exists(dbFolder))
        {
            Directory.CreateDirectory(dbFolder);
        }

        options.Directory = dbFolder;

        Logger.Debug($"Will open/create DB at path {dbFolder}");
        if (!Database.Exists(AppConstants.DB_NAME, dbFolder))
        {
            // Load prebuilt database to path
            await seedService.CopyDatabaseAsync(dbFolder);
            Db = new Database(AppConstants.DB_NAME, options);
            CreateDatabaseIndexes();
            Analytics.TrackEvent("Installed prebuilt database");

            //Db = new Database(AppConstants.DB_NAME, options);       // TODO: Reinstate prebuilt db once new version attached!
        }
        else
        {
            Db = new Database(AppConstants.DB_NAME, options);
        }

        // Database.SetLogLevel(Couchbase.Lite.Logging.LogDomain.Replicator, Couchbase.Lite.Logging.LogLevel.Debug);
        // Listen to events from app about going to sleep or wake up
        eventAggregator.GetEvent<OnBackgroundEvent>().Subscribe(HandleReplicator);
        eventAggregator.GetEvent<OnConnectivityEvent>().Subscribe(HandleReplicator);
        eventAggregator.GetEvent<OnLoginEvent>().Subscribe(RestartReplicator);
        eventAggregator.GetEvent<OnImagePushedEvent>().Subscribe(DisableReplication);
        AppUtils.LastReplication = null;

        StartReplicator();
        return;
    }

And this is the StartReplicator method:

void StartReplicator()
{
    AppUtils.ConnectionStatus = ReplicatorActivityLevel.Offline;
    if (null == Db)
    {
        Crashes.TrackError(null, new Dictionary<string, string> { { "StartReplicator", "Database not set!" } });
        throw new InvalidOperationException("Database not set!");
    }
    // Anonymous settings
    var username = AppUtils.AnonymousUserId;
    var password = AppUtils.AnonymousPassword;
    //var replType = ReplicatorType.Pull;           TODO: Should we revert to PULL?? - Pull-push avoids the crash error right now...
    var replType = ReplicatorType.PushAndPull;
    var channels = new[] { "!" };
    if (AppUtils.IsLoggedIn || AppUtils.ActivationPending)
    {
        username = AppUtils.UserId;
        password = AppUtils.Password;
        channels = new[] { "!", $"channel.{username}" };
        //replType = ReplicatorType.PushAndPull;    TODO: Reinstate if pull only above is reverted to
    }

    var dbUrl = new Uri(syncUrl, AppConstants.DB_NAME);
    Logger.Debug($"Start {replType} Replicator: Will connect to: {syncUrl} - user: '{username}'" + (AppUtils.ActivationPending ? " (activation pending!)" : ""));

    var config = new ReplicatorConfiguration(Db, new URLEndpoint(syncUrl))
    {
        ReplicatorType = replType,
        Continuous = true,
        Authenticator = new BasicAuthenticator(username, password),
        Channels = AppUtils.IsLoggedIn ? new[] { "!", $"channel.{username}" } : new[] { "!" }
    };

    config.PushFilter = (document, flags) =>
        {
            if (document.GetBoolean(nameof(BaseDoc.IsSyncDisabled).ToLower()))
            {
                Logger.Debug($"Pushfilter: Replication diabled for {document.Id}. Skip...");
                return false;
            }
            return true;
        };
    replicator = new Replicator(config);
    replListener = replicator.AddChangeListener((sender, args) =>
    {
        if (args is null) return;
        var s = args.Status;
        AppUtils.ConnectionStatus = s.Activity;
        Logger.Debug($"{replType} Replicator: {s.Progress.Completed}/{s.Progress.Total}{(string.IsNullOrEmpty(s.Error?.Message) ? "" : $", error {s.Error?.Message}")}, activity = {s.Activity}");
        if (!(s.Error is null))
        {
            if (s.Error is CouchbaseWebsocketException)
            {
                var cbex = s.Error as CouchbaseWebsocketException;
                if (CouchbaseLiteErrorType.CouchbaseLite.Equals(cbex?.Domain) && CouchbaseLiteError.HTTPAuthRequired.Equals(cbex?.Error))
                {
                    // If this is the anonymous user then we have a serious problem....
                    if (AppUtils.AnonymousUserId.Equals(username))
                    {
                        Crashes.TrackError(null, new Dictionary<string, string> { { "StartReplicator", $"User '{username}' with password: '{password}' cannot authenticate with server!!!" } });
                        Logger.Debug($"User '{username}' with password: '{password}' cannot authenticate with server!!! - Replication will not run");
                        StopReplicator();
                        return;
                    }
                    else
                    {
                        Logger.Debug($"Wrong credentials for user: {username}. Disable auto login...");
                        AppUtils.LoggedInTime = DateTime.MinValue;
                        AppUtils.Password = null;
                        // Retry start of replication without a logged in user - carefull here....!
                        StartReplicator();
                        return;
                    }
                }
            }
            Logger.Debug($"Error :: {s.Error}");
        }
        else if (s.Activity == ReplicatorActivityLevel.Idle)
        {
            AppUtils.LastReplication = DateTime.Now;
            if (s.Progress.Completed > 0)
            {
                eventAggregator.GetEvent<OnDataUpdateEvent>().Publish(true);
            }
        }
        eventAggregator.GetEvent<OnSyncStatusEvent>().Publish(s.Activity);
    });
    docListener = replicator.AddDocumentReplicationListener((sender, args) =>
    {
        // TODO: Consider checking docs (e.g. ispublic and userkey) to send more specific update events!!

        // Remove Image docs after they have been sent to the server. Images should be read from url's
        if (args.IsPush)
        {
            Logger.Debug($"Push replication finished sending {args.Documents.Count} documents");
            foreach (var document in args.Documents)
            {
                if (document.Error == null)
                {
                    Logger.Debug($"  Doc ID: {document.Id}" + (document.Flags.HasFlag(DocumentFlags.Deleted) ? " (deletion)" : "") + (document.Flags.HasFlag(DocumentFlags.AccessRemoved) ? " (access removed)" : ""));
                    if (!document.Flags.HasFlag(DocumentFlags.Deleted))
                    {
                        if (typeof(Image).Name.Equals(BaseDoc.GetTypeFromID(document.Id)))
                        {
                            eventAggregator.GetEvent<OnImagePushedEvent>().Publish(document.Id);
                        }
                    }
                }
                else
                {
                    Logger.Error($"  {(string.IsNullOrEmpty(document.Id) ? "" : $"Doc.id: {document.Id}, ")}Error: {document.Error}");
                }
            }
        }
        else
        {
            // TODO: Probably needs to be "silenced"/removed...
            Logger.Debug($"Pull replication finished receiving {args.Documents.Count} documents");
            foreach (var document in args.Documents)
            {
                if (document.Error == null)
                {
                    // Update User's full name in preferences if there is an update from the server.
                    if (AppUtils.IsLoggedIn && string.Equals(document.Id, BaseDoc.GetID(typeof(User).Name, AppUtils.UserId)))
                    {
                        var doc = Db.GetDocument(document.Id);
                        if (!(doc is null))
                        {
                            var user = doc.ToObject<User>();
                            AppUtils.UserFullName = user.AbbreviatedName;
                            Logger.Debug($"  Updated full name: {user.AbbreviatedName} in preferences for logged in user id: {AppUtils.UserId}");
                            if (user.Deleted)
                            {
                                AppUtils.StayLoggedIn = false;
                                AppUtils.Password = null;
                                AppUtils.LoggedInTime = DateTime.MinValue;
                                eventAggregator.GetEvent<OnLoginEvent>().Publish();
                            }
                        }
                    }
                    Logger.Debug($"  Doc ID: {document.Id}" + (document.Flags.HasFlag(DocumentFlags.Deleted) ? " (deletion)" : "") + (document.Flags.HasFlag(DocumentFlags.AccessRemoved) ? " (access removed)" : ""));
                }
                else
                {
                    Logger.Error($"  {(string.IsNullOrEmpty(document.Id) ? "" : $"Doc.id: {document.Id}, ")}Error: {document.Error}");
                }
            }
        }
    });
    replicator.Start();
}

Any suggestions or input is very much appreciated as the way it is running now is going to make Android people throw their phones to the ground…

I don’t know much about the Xamarin development. Is it possible to do some profiling where the time get spent that may cause the UI sluggish.

@Sandy_Chuang Can you help reviewing the code and see if you have any opinion?

I suspect that your problem is in your DocumentChangeListener. I’m the Android guy, not the Xamarin guy, but I believe that your DocumentChangeListener is running on the UI thread. It is doing a lot of work…

As my colleagues have pointed out, it would be useful to do some profiling to find the hot spots in your code. Otherwise that’s a rather big block of code to just put out and ask “is anything wrong here?”

@blake.meike thanks for the pointer to the Document Change listener. I’m not sure that I agree on it doing a lot of work. I have tried to limit the work - and it is mostly logging (and then a little more work for very few documents). I would not consider that “a lot”??? How else would I react to changes on replicated docs? I don’t do the real work - just publish an event if there is anything to do… I’m surprised that the code in the listener runs on the UI thread… I would have expected it to be running on the background thread as I expect the sync. to do…?

@borrrden I haven’t tried profiling in Xamarin - but it is a good point and I will look into that. I didn’t mean to throw in a lot of code and say “something is wrong here”… I really tried to code this without putting very much work into the listeners - and tried to set up the replicator correctly. I tried to describe the problem and just “document” it with the code - and if I didn’t copy the code in (or left out most of it) then that would probably just lead to more questions… So I’m sorry you see it the way you do - I really tried not to do that but be very specific about the problem and add the code for reference documentation.

Ok, I tried to disable the replication and document listeners - just to see if it made any significant change… First, I thought it did - and then I re-added the same listeners… - and I thought that there wasn’t really any notable difference.

So I guess this just points me in the direction of finding out how to profile the application - instead of just having a “feeling”…

If you have any input on the background/UI thread I would appreciate to learn more about how to best do this in replication and attached listeners :slight_smile:

I’ve investigated further (just using print statements in the code) and found a couple of places where access to the data service querying the database was not “awaited” (ie. it was running on the UI thread). And this happens in some tabs that all load at the same time (not when the tab is selected). A combination of these seem to be spoiling the user experience - the sync. task just seemed to add to the total degrade of the response time - but it may actually just be the “coincidence” that the sync. task is kicked off at the same time…

But I’m still interested in hearing more about background vs. UI task for the sync - if that is in any way running on the UI thread… this was actually the comment that lead me to test various parts of the app (simply by looking through code - and commenting chuncks of code out - so thanks for that!