Android query native crash

Mobile Couchbase Lite
1

cb version is 3.1.9
This only occurs on one database, the others are normal

11-04 10:45:30.771 F/DEBUG (11431): uid: 10295
11-04 10:45:30.771 F/DEBUG (11431): tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
11-04 10:45:30.771 F/DEBUG (11431): signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xb400006ccf5d8000
11-04 10:45:30.771 F/DEBUG (11431): x0 b400006ccf5d7ffe x1 0000000000000004 x2 0000000000000002 x3 0000000000000000
11-04 10:45:30.771 F/DEBUG (11431): x4 b400006e6fdabddc x5 b400006ccf5d7ffe x6 000000006a630080 x7 0000567b00007117
11-04 10:45:30.771 F/DEBUG (11431): x8 0000000000000080 x9 b400006ccf5d8000 x10 0000000000000000 x11 0000000000473244
11-04 10:45:30.771 F/DEBUG (11431): x12 0000000000000000 x13 0000000030cff6be x14 0000000000000000 x15 000000000000001f
11-04 10:45:30.771 F/DEBUG (11431): x16 0000006bd8107110 x17 0000006fa4abaac0 x18 0000006b881d2000 x19 0000006cd9e55780
11-04 10:45:30.771 F/DEBUG (11431): x20 b400006e6fdabdd0 x21 0000006cd9e55780 x22 b400006eefa930e0 x23 00000000000018da
11-04 10:45:30.771 F/DEBUG (11431): x24 0000000000005deb x25 0000000000000030 x26 b400006d2f9f40f0 x27 0000000000000000
11-04 10:45:30.771 F/DEBUG (11431): x28 0000006cd9e59000 x29 0000006cd9e556e0
11-04 10:45:30.771 F/DEBUG (11431): lr 0000006bd7f1b4c8 sp 0000006cd9e556e0 pc 0000006bd7f28758 pst 0000000080001000
11-04 10:45:30.771 F/DEBUG (11431): 94 total frames
11-04 10:45:30.771 F/DEBUG (11431): backtrace:
11-04 10:45:30.771 F/DEBUG (11431): #00 pc 00000000002ab758 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #01 pc 000000000029e4c4 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #02 pc 000000000029e7e0 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #03 pc 000000000029e840 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #04 pc 00000000001f0c74 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #05 pc 00000000001f06b4 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #06 pc 00000000001af398 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #07 pc 00000000001af4e8 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #08 pc 0000000000199688 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCore.so (offset 0x2000) (c4query_run+16) (BuildId: efa2bc6c4979d864858e88f26473421585c8e1d0)
11-04 10:45:30.771 F/DEBUG (11431): #09 pc 000000000001b5b0 /data/app/~~pIirYPx3YC7IZpsiuQF6kg==/com.onyx.galaxy.note-W69653G0zooxMShEt1pypA==/base.apk!libLiteCoreJNI.so (offset 0x491000) (Java_com_couchbase_lite_internal_core_impl_NativeC4Query_run+76) (BuildId: efd6be4bf71ef11f5fcee3aee05964d618d03e5c)
11-04 10:45:30.771 F/DEBUG (11431): #10 pc 0000000000355630 /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+144) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #11 pc 00000000005bb68c /apex/com.android.art/lib64/libart.so (nterp_helper+3852) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #12 pc 000000000035e054 [anon:dalvik-DEX data] (com.couchbase.lite.internal.core.impl.NativeC4Query.nRun+0)
11-04 10:45:30.771 F/DEBUG (11431): #13 pc 00000000005bd304 /apex/com.android.art/lib64/libart.so (nterp_helper+11140) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #14 pc 000000000035a980 [anon:dalvik-DEX data] (com.couchbase.lite.internal.core.C4Query.lambda$run$0+36)
11-04 10:45:30.771 F/DEBUG (11431): #15 pc 00000000005bb6d4 /apex/com.android.art/lib64/libart.so (nterp_helper+3924) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #16 pc 000000000035a918 [anon:dalvik-DEX data] (com.couchbase.lite.internal.core.C4Query.j+0)
11-04 10:45:30.771 F/DEBUG (11431): #17 pc 00000000005ba7b4 /apex/com.android.art/lib64/libart.so (nterp_helper+52) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #18 pc 000000000035d4e4 [anon:dalvik-DEX data] (com.couchbase.lite.internal.core.g1.apply+16)
11-04 10:45:30.771 F/DEBUG (11431): #19 pc 00000000005bc4f4 /apex/com.android.art/lib64/libart.so (nterp_helper+7540) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #20 pc 000000000035a466 [anon:dalvik-DEX data] (com.couchbase.lite.internal.core.C4NativePeer.withPeerOrNull+30)
11-04 10:45:30.771 F/DEBUG (11431): #21 pc 00000000005bb6d4 /apex/com.android.art/lib64/libart.so (nterp_helper+3924) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #22 pc 000000000035aaa2 [anon:dalvik-DEX data] (com.couchbase.lite.internal.core.C4Query.run+10)
11-04 10:45:30.771 F/DEBUG (11431): #23 pc 00000000005bb6d4 /apex/com.android.art/lib64/libart.so (nterp_helper+3924) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #24 pc 000000000033af54 [anon:dalvik-DEX data] (com.couchbase.lite.AbstractQuery.execute+68)
11-04 10:45:30.771 F/DEBUG (11431): #25 pc 00000000005bb6d4 /apex/com.android.art/lib64/libart.so (nterp_helper+3924) (BuildId: 02bec5940be704b863f6514fc7d81c41)
11-04 10:45:30.771 F/DEBUG (11431): #26 pc 0000000000349954 [anon:dalvik-DEX data] (com.couchbase.lite.Limit.execute+0)

2

Please give more information on what operation(s) were happening at the time and as much information as you can that may be relevant.

3

If you can get me a tombstone for this event, I’ll have a much better chance of figuring out what happened.

On modern Androids, you can get the tombstones with the command:

adb -s <device serial #> bugreport

The tombstones are then in the directory: <unzip>/FS/data/tombstones/

If you can just zip up that tombstones directory and get it to me, I will be able to see what’s going on

4

@lxm: You say that this happens with only one database. Can we get a copy of that database?

… and don’t worry about the tombstones. I believe I found a way to get symbols into that stacktrace. What would be handy is any logs you have around the time of the crash, and the query you are running.

I’ve created * CBL-6437 to track the issue.

5

We were able to decode the backtrace. It’s down in some very low-level C++ code that’s packing the query results into a binary data structure, and it looks like its write-stream somehow ran off the end of a heap block into unmapped address space. Very weird, since this is code that’s used all the time by CBL and which hasn’t changed in about five years.

I’m not an Android developer; are there any heap-debugging tools you can use or modes you can enable, that would provide more diagnostics in the event of heap corruption?

Also, if there’s any way we could reproduce this in-house then we could run a debug build of CBL that uses the C++ address sanitizer, which would help a lot.