Reproducable bug: document with attachment not pulled down

Mobile
1

Reproducable: yes
CBL 2.7 Android
SG 2.7
CB 6 CE

One-shot pulling down all documents of my user account never downloads 1 specific document. The logs show:

...

2020-02-26 22:33:37.131 I/chatty: uid=12395(com.myapp) Thread-164 identical 21 lines
2020-02-26 22:33:37.131 D/NETWORK: C4Socket.completedReceive @516289027392: com.couchbase.lite.internal.replicator.CBLWebSocket@6a598b5
2020-02-26 22:33:37.132 V/NETWORK: WebSocketListener received data of 73 bytes
2020-02-26 22:33:37.132 D/NETWORK: C4Socket.received @516289027392: 73
2020-02-26 22:33:37.132 D/NETWORK: C4Socket.completedReceive @516289027392: com.couchbase.lite.internal.replicator.CBLWebSocket@6a598b5
2020-02-26 22:33:37.134 I/chatty: uid=12395(com.myapp) Thread-164 identical 12 lines
2020-02-26 22:33:37.134 D/NETWORK: C4Socket.completedReceive @516289027392: com.couchbase.lite.internal.replicator.CBLWebSocket@6a598b5
2020-02-26 22:33:37.134 D/NETWORK: C4Socket.completedReceive @516289027392: com.couchbase.lite.internal.replicator.CBLWebSocket@6a598b5
2020-02-26 22:33:37.139 I/chatty: uid=12395(com.myapp) Thread-186 identical 20 lines
2020-02-26 22:33:37.140 D/NETWORK: C4Socket.completedReceive @516289027392: com.couchbase.lite.internal.replicator.CBLWebSocket@6a598b5

First error below:

2020-02-26 22:33:37.140 E/CouchbaseLite/REPLICATOR: {N8litecore4repl12IncomingBlobE#3}==> N8litecore4repl12IncomingBlobE for doc '7JFlD2OEV7U2ZBYiBXCC20aEaIL2::group::7JFlD2OEV7U2ZBYiBXCC20aEaIL2'._attachments["40394571ed67e486efeb24442358a4dd::small"] [sha1-mTIaPpVTmY3YamtJVG07wQZ9bn0=] @0x781beb89c8
2020-02-26 22:33:37.140 E/REPLICATOR: {N8litecore4repl12IncomingBlobE#3}==> N8litecore4repl12IncomingBlobE for doc '7JFlD2OEV7U2ZBYiBXCC20aEaIL2::group::7JFlD2OEV7U2ZBYiBXCC20aEaIL2'._attachments["40394571ed67e486efeb24442358a4dd::small"] [sha1-mTIaPpVTmY3YamtJVG07wQZ9bn0=] @0x781beb89c8
2020-02-26 22:33:37.140 D/NETWORK: C4Socket.write @516289027392: com.couchbase.lite.internal.replicator.CBLWebSocket@6a598b5
2020-02-26 22:33:37.140 D/NETWORK: C4Socket.completedWrite @516289027392: 70
2020-02-26 22:33:37.140 E/CouchbaseLite/REPLICATOR: {N8litecore4repl12IncomingBlobE#3} Got error response: HTTP 403 'Attachment's doc not being synced'
2020-02-26 22:33:37.140 E/REPLICATOR: {N8litecore4repl12IncomingBlobE#3} Got error response: HTTP 403 'Attachment's doc not being synced'
2020-02-26 22:33:37.140 D/NETWORK: C4Socket.write @516289027392: com.couchbase.lite.internal.replicator.CBLWebSocket@6a598b5
2020-02-26 22:33:37.140 D/NETWORK: C4Socket.completedWrite @516289027392: 70
2020-02-26 22:33:37.141 V/NETWORK: {N8litecore4blip6BLIPIOE#4}==> N8litecore4blip6BLIPIOE ->wss://domain.com/database/_blipsync @0x7828b63cc8
2020-02-26 22:33:37.141 V/NETWORK: {N8litecore4blip6BLIPIOE#4} Finished sending REQ #5 
2020-02-26 22:33:37.141 V/NETWORK: {N8litecore4blip6BLIPIOE#4} ...Wrote 70 bytes to WebSocket (writeable=1)
2020-02-26 22:33:37.143 V/REPLICATOR: {N8litecore4repl8InserterE#5}==> N8litecore4repl8InserterE ->wss://domain.com/database/_blipsync @0x784f0ea348
2020-02-26 22:33:37.143 V/REPLICATOR: {N8litecore4repl8InserterE#5} Inserting 42 revs:
2020-02-26 22:33:37.143 V/DATABASE: {DB#6}==> N8litecore14SQLiteDataFileE /data/user/0/com.myapp/files/couchbase_database.cblite2/db.sqlite3 @0x783500f8c0
2020-02-26 22:33:37.143 V/DATABASE: {DB#6} begin transaction
2020-02-26 22:33:37.144 V/DATABASE: {DB#6} Saved '7JFlD2OEV7U2ZBYiBXCC20aEaIL2::direction::cjrfu2kuu881::qf5t1v7bcxeh' rev #1-8d7c05040930a3d69bb5d68a64f01a77 as seq 118

...

Subsequently my Android app would crash as it assumes that the document is there. I’m able to catch the exception and then download the missing document which works just fine. Once downloaded, the attachment - an image - is viewable. Here’s how I start the one-shot pull recplication with just the missing document:

URI syncGatewayURI = new URI(SyncgatewayHelper.getUrl());
URLEndpoint endpoint = new URLEndpoint(syncGatewayURI);
ReplicatorConfiguration configuration = new ReplicatorConfiguration(CouchbaseUtil.getInstance(context).getDatabase(), endpoint);
configuration.setAuthenticator(new SessionAuthenticator(UtilPreference.getInstance(context).getSessionId()));
configuration.setReplicatorType(ReplicatorConfiguration.ReplicatorType.PULL);
configuration.setContinuous(false);
configuration.setDocumentIDs(Collections .singletonList("missing-doc-id"));
Replicator replicator = new Replicator(configuration);
replicator.start();

I saw reports in my bug tracker but never knew what went wrong and luckily I ran into the same issue and can reproduce it everytime now! This only happens with one of my user accounts and it’s the only document which is not pulled down. There are other documents which also have attachments. Old attachments from CBL 1 and blobs from CBL2.

The missing document is:

{
  "_attachments": {
    "40394571ed67e486efeb24442358a4dd::small": {
      "content_type": "image/jpeg",
      "digest": "sha1-mTIaPpVTmY3YamtJVG07wQZ9bn0=",
      "length": 60011,
      "revpos": 2,
      "stub": true
    }
  },
  "channels": "7JFlD2OEV7U2ZBYiBXCC20aEaIL2",
  "groupMembers": [],
  "id": "7JFlD2OEV7U2ZBYiBXCC20aEaIL2",
  "name": "",
  "type": "group"
}

---
meta data
---

{
  "meta": {
    "id": "7JFlD2OEV7U2ZBYiBXCC20aEaIL2::group::7JFlD2OEV7U2ZBYiBXCC20aEaIL2",
    "rev": "36620-15979b80d97d00000000000000000000",
    "expiration": 0,
    "flags": 0,
    "type": "json"
  },
  "xattrs": {
    "_sync": {
      "rev": "3-63ebe1c7e77b2e5b5808788678eeac3c",
      "sequence": 82468967,
      "recent_sequences": [
        82158911,
        82468934,
        82468967
      ],
      "history": {
        "revs": [
          "2-6cebe022093547235f48a5b89fba94f6",
          "3-63ebe1c7e77b2e5b5808788678eeac3c",
          "1-7a14cfdc02b0c94b042900e8fd2d3774"
        ],
        "parents": [
          2,
          0,
          -1
        ],
        "channels": [
          [
            "7JFlD2OEV7U2ZBYiBXCC20aEaIL2"
          ],
          [
            "7JFlD2OEV7U2ZBYiBXCC20aEaIL2"
          ],
          [
            "7JFlD2OEV7U2ZBYiBXCC20aEaIL2"
          ]
        ]
      },
      "channels": {
        "7JFlD2OEV7U2ZBYiBXCC20aEaIL2": null
      },
      "cas": "0x00007dd9809b9715",
      "value_crc32c": "0xdaa6d9ed",
      "time_saved": "2018-11-20T10:16:26.629403765+01:00"
    }
  }
}
2

The error appears to be "Got error response: HTTP 403 'Attachment's doc not being synced'" which means it came from Sync Gateway. Any idea, @adamf or @bbrks?

3

That error generally means that Sync Gateway doesn’t think the requested attachment is associated with the document being replicated, and so the proveAttachment is failing. There was an error manifesting the same way in 2.5.0 (https://issues.couchbase.com/browse/CBG-245), but that’s fixed in 2.7.0 and doesn’t appear to be exactly what’s going on here, based on the metadata.

Filed https://issues.couchbase.com/browse/CBG-741 for followup.

4

Following up on how I “fixed” the issue. Just to reiterate the issue

  • user signs in with one-shot pull down sync
  • one document with attachment is not downloaded
  • downloading the missing document with one-shot pull down replicator, resetting the checkpoint and setting the doc ID
  • the issue reappears if user reinstalls app, signs in again, …

The “fix”:

  • Replacing the attachment with a blob once it was downloaded and saving the change to disk and replicate it
  • Reinstalling app, signing in again, document was pulled down on first try. No issues!

I’m not sure if the issue disappeared because it’s now a blob (CBL 2.7).

If I may add one more question about resetCheckpoint().

ReplicatorConfiguration configuration = new ReplicatorConfiguration(database, endpoint);
// ...
configuration.setReplicatorType(ReplicatorConfiguration.ReplicatorType.PULL);
configuration.setContinuous(false);
configuration.setDocumentIDs(Collections.singletonList("my-doc-id"));
Replicator replicator = new Replicator(configuration);
replicator.resetCheckpoint();
replicator.start();

Are the following statements all true:

  • checkpoint is only reset for this single replicator
  • only the specified document will be downloaded
  • once downloaded this single replicator stops (i.e. does not replicate anything anymore)

I’m asking as I purged documents in the past and I want to be sure that they are not downloaded (if their document IDs are not specified). Thanks!

5

Now fixed! Thanks to @adamf and @bbrks!

1 Like