Couchbase lite crashing the app with the following error after bumping to version 3.1.1

Couchbase lite crashing the android app with the following error after bumping to version 3.1.1. I am not getting any hint from the crash about the place and reason of crash. I am inserting and pulling data from the database using Coroutines. If parent coroutine’s dispatcher i use is Dispatcher.Main then there is no crash. But i want insertion and access to happen on background thread so i use Dispatcher.Default or Dispatcher.IO and when i do it app crashes after few seconds of app start. Within this parent coroutine i start multiple child coroutines that work concurrently to retreive as well as save data to database. To further ensure the safety on insertion and to avoid any database lock situation i have also created a seperate queue post the crash in which all coroutines insert the doc to the queue and then one single seperate coroutine pick a doc at a time and insert it to database. But even this has not helped in avaoiding the crash.

pid: 22245, tid: 22591, name: Timer (CBL) >>> {package_name}.debug <<<
2023-09-22 12:23:44.945 22745-22745 DEBUG crash_dump64 A #01 pc 00000000003b8110 /data/app/~~dnXLyBC2Ts-9nH1pRr6Phg==/{package_name}.debug-UIKwYQvQa3YDtLLKNYs_3A==/base.apk!libLiteCore.so (std::__ndk1::recursive_mutex::lock()+12) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-22 12:23:44.945 22745-22745 DEBUG crash_dump64 A #02 pc 000000000018f064 /data/app/~~dnXLyBC2Ts-9nH1pRr6Phg==/{package_name}.debug-UIKwYQvQa3YDtLLKNYs_3A==/base.apk!libLiteCore.so (litecore::BackgroundDB::useInTransaction(fleece::slice, fleece::function_ref<bool (litecore::KeyStore&, litecore::SequenceTracker*)>)+56) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)

Can you set the log level to debug/all domains and get us adb logcat logs from around the time of the crash?

Hope this helps

E UPLOAD INIT FROM CONNECTIVITY MANAGER
2023-09-26 17:52:15.484 1447-1447 IVMS - V3.0 com.aidash.ivms_pro.debug E PERMISSION RESULT 0
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A Build fingerprint: ‘google/sdk_gphone64_arm64/emulator64_arm64:12/SE1A.220630.001/8789670:userdebug/dev-keys’
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A Revision: ‘0’
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A ABI: ‘arm64’
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A Timestamp: 2023-09-26 17:52:14.842363737+0530
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A Process uptime: 0s
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A Cmdline: com.aidash.ivms_pro.debug
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A pid: 1447, tid: 1666, name: Timer (CBL) >>> com.aidash.ivms_pro.debug <<<
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A uid: 10149
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A tagged_addr_ctrl: 0000000000000001
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x48
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A Cause: null pointer dereference
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x0 0000000000000048 x1 b4000074bb741b54 x2 0000000000000007 x3 000000741fef1f40
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x4 00000073eba7dbb8 x5 00000000017a99c4 x6 26ff307cff284b41 x7 7f7f7f7f7f7f7f7f
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x8 0000000000000003 x9 00000073eba7daf0 x10 00000073eba7db20 x11 00000073eba7dba0
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x12 000000000000000a x13 000000007fffffff x14 00000000017a99c4 x15 000005ba83bb358c
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x16 000000742016c0c0 x17 0000007752083688 x18 00000073eb208000 x19 0000000000000048
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x20 0000000000000000 x21 00000073eba7dbb8 x22 000000741fef1f40 x23 0000000000000007
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x24 b4000074bb741b54 x25 00000073eba7dcb0 x26 00000073eba7dff8 x27 00000000000fc000
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A x28 00000073eb985000 x29 00000073eba7da70
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A lr 0000007420112114 sp 00000073eba7da70 pc 0000007752083690 pst 0000000020001000
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A backtrace:
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #00 pc 00000000000b2690 /apex/com.android.runtime/lib64/bionic/libc.so (pthread_mutex_lock+8) (BuildId: ba489d4985c0cf173209da67405662f9)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #01 pc 00000000003b8110 /data/app/~~dcNanA_HVX9PdbHn2MsByA==/com.aidash.ivms_pro.debug-mDnB–TjLLDuCScfFRrVSQ==/base.apk!libLiteCore.so (std::__ndk1::recursive_mutex::lock()+12) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #02 pc 000000000018f064 /data/app/~~dcNanA_HVX9PdbHn2MsByA==/com.aidash.ivms_pro.debug-mDnB–TjLLDuCScfFRrVSQ==/base.apk!libLiteCore.so (litecore::BackgroundDB::useInTransaction(fleece::slice, fleece::function_ref<bool (litecore::KeyStore&, litecore::SequenceTracker*)>)+56) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #03 pc 0000000000197378 /data/app/~~dcNanA_HVX9PdbHn2MsByA==/com.aidash.ivms_pro.debug-mDnB–TjLLDuCScfFRrVSQ==/base.apk!libLiteCore.so (litecore::Housekeeper::_doExpiration()+52) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #04 pc 0000000000258d68 /data/app/~~dcNanA_HVX9PdbHn2MsByA==/com.aidash.ivms_pro.debug-mDnB–TjLLDuCScfFRrVSQ==/base.apk!libLiteCore.so (litecore::actor::timer::Manager::run()+184) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #05 pc 000000000025921c /data/app/~~dcNanA_HVX9PdbHn2MsByA==/com.aidash.ivms_pro.debug-mDnB–TjLLDuCScfFRrVSQ==/base.apk!libLiteCore.so (litecore::actor::timer::Manager::Manager()::$_0::operator()() const+12) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #06 pc 000000000025920c /data/app/~~dcNanA_HVX9PdbHn2MsByA==/com.aidash.ivms_pro.debug-mDnB–TjLLDuCScfFRrVSQ==/base.apk!libLiteCore.so (decltype(std::__ndk1::forwardlitecore::actor::Timer::Manager::Manager()::$_0(fp)()) std::__ndk1::__invokelitecore::actor::Timer::Manager::Manager()::$_0(litecore::actor::timer::Manager::Manager()::$_0&&)+8) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #07 pc 00000000002591ac /data/app/~~dcNanA_HVX9PdbHn2MsByA==/com.aidash.ivms_pro.debug-mDnB–TjLLDuCScfFRrVSQ==/base.apk!libLiteCore.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_deletestd::__ndk1::__thread_struct >, litecore::actor::timer::Manager::Manager()::$_0> >(void*)+44) (BuildId: ee0c202701aa67ffcdc739dea1e8911592e589ba)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #08 pc 00000000000b1910 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: ba489d4985c0cf173209da67405662f9)
2023-09-26 17:52:15.484 1672-1672 DEBUG crash_dump64 A #09 pc 00000000000513f0 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: ba489d4985c0cf173209da67405662f9)

@chauhanabhi321 That is somewhat helpful.

In order to figure out what is happening, though, I will need the logs. What you are providing is a crash dump. By the time the app crashes, all that is left are ashes. The fire happened seconds ago.

I need the log. Please use adb logcat to collect the log from several seconds before the crash, until the crash takes place and attach the output to this ticket.

CRASH_logs.zip (3.8 KB)
PFA the logs 30 seconds before the crash

Yep. Here’s the problem:

09-26 21:58:17.666 24698 24709 E System  : Uncaught exception thrown by finalizer
09-26 21:58:17.669 24698 24709 E System  : java.lang.NullPointerException
09-26 21:58:17.669 24698 24709 E System  :      at java.util.Objects.requireNonNull(Objects.java:220)
09-26 21:58:17.669 24698 24709 E System  :      at com.couchbase.lite.internal.core.C4Database$ManagedC4Database.closePeer(C4Database.java:174)
09-26 21:58:17.669 24698 24709 E System  :      at com.couchbase.lite.internal.core.C4Database$ManagedC4Database.finalize(C4Database.java:170)
09-26 21:58:17.669 24698 24709 E System  :      at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:291)
09-26 21:58:17.669 24698 24709 E System  :      at java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:278)
09-26 21:58:17.669 24698 24709 E System  :      at java.lang.Daemons$Daemon.run(Daemons.java:139)
09-26 21:58:17.669 24698 24709 E System  :      at java.lang.Thread.run(Thread.java:920)

Here is the code that is causing the problem:

private void closePeer(@Nullable LogDomain domain) { releasePeer(domain, impl::nFree); }

Apparently impl is null.

But here is the definition of impl

private final NativeImpl impl;

So: a final field has been set null by finalization, before the finalization of an object that references it.

I have seen this before. It happens on some versions of ART. It is, absolutely, not behavior that is in accordance with the JVM spec. … and, yes, I know, this isn’t the JVM. Still… it is, uhh… surprising.

I thought I had nailed most of these cases. I have opened ticket CBL-4950 to track this.

1 Like

Thanks for closing the related ticket CBL-4950. Could you please also tell when this fix will be included in the release. As i could see from this release page (Release Notes | Couchbase Docs), this fix is not yet part of the public release

It will be included in the next platform release.
Please note that it is usually possible to avoid this problem by explicitly closing AutoClosables. If they are closed explicitly, the finalizer will never have anything to do.

Thanks Blake. Will look into AutoClosables. Btw, any tentative date of the release?

Dates are way, way above my pay grade. I can speculate that the release seems likely to be mid 2024

I did try closing resultSet but found no relief. It’s still crashing. Looking deeper into the code i found it’s crashing at database.setDocumentExpiration(id). To be more specific, we are running a loop over 500 items and doing database.setDocumentExpiration(id, Date(System.currentTimeMillis() - 100000)) on each item to delete it. It’s crashing instantly when we do it. Also, as explained before, it’s running fine on 3.0 but from 3.1 it’s crashing. Also, we did try delete() method for each document but it was too slow. That’s why we shifted to setDocumentExpiration instead of delete.

It is very, very (very!!!) unlikely that you are experiencing the same crash that you cite in your original post, after calling close() on the Database.

That original crash was in a finalizer that was closing a C4Document. I can’t think of any reason that closing a ResultSet would have anything to do with that issue.

In fact, you say that the new crash is in “setDocumentExpiration”. This seems to be an entirely unrelated issue. I suggest you open a new post for it and provide a stack trace and some logs, so that I can analyze it.

Also, FWIW, nobody suggested trying to use “delete” on every document. That wouldn’t make sense.

Okay, may be i misunderstood. With AutoClosables as you suggested, i assumed i need to close ResultSet. Sorry, i am not sure what you meant when you said “avoid this problem by explicitly closing AutoClosables”. Anyway, as i will open a fresh issue as i can see crash in relation to document expiration.