Older document revision getting purged from couchbase

Mobile
1

I am using 4.1.1 and syncgateway 1.3.1.
Older revisions of document is getting purged from couchbase .
i have assigned a channel1 in 1st revision and in 2nd revision assigned channel2.
i could see that channel removed entry for channel1 in _changes feed. but whenever i do GET operation from syncgateway the first revision, i could see that “not found” reason:“missing”.

Due to this documents replication is stopping between clusters.

i would like to know in what scenarios documents purged from couchbase, is it syncgateway setting any expiration value for older revisions.

2

If the user you’re using to do a get doesn’t have access to the active revision, they won’t be able to retrieve the document (even older revisions of the document).

This is the expected behaviour, and shouldn’t prevent replication. What type of replication are you attempting, and what kind of issue are you hitting?

3

@adamf, i have accessed admin port to access the data. looks like revision got purged from couchbase itself. we have set rev_limit to 5 in syncgateway. how many revision can couchbase stores and when actually older revisions get deleted from couchbase

i am using the sg-replication with channel filtering.

setup:
Both clsuter have below setup
3 CBS nodes
2 sync gateway(1st syncgw assignes–> channel1 and 2nd syncgw → channel2)
2 sg replicate(1st → replicates channel1 and 2nd → replicates channel2)
Scenario: 1. 1st revision is assgined to channel1
2. 2nd revision is assigned to channel2
3. 3rd revision is channel1

Behavior : 1st sg-replicate _changes with channel1 filter feed has entry for 1st revision as removed[“channel1”] . but could see that document with 1st revision is not found error in bulk get response. hence bulk post is failing with 400 Bad Revisions error. but response status as 201.

when will older revisions get purged from couchbase. do we have any configuration to set for purging from couchbase.

4

Can you share a few more specifics of your architecture?

  1. Are your two Sync Gateway instances backed by different buckets?
  2. What are the source and target endpoints for each of your sg-replicate replications?
  3. When you say ‘1st Sync Gateway assigns channel1 and 2nd assigns channel2’ - is that just in this particular test flow (i.e. you’re writing rev1 to SG1 and rev2 to SG2), or does each SG also have a different sync function?

Thanks.

5

Thanks for you time @adamf
SETUP Details

Cluster1
2syncgateway(1 : assigns channel1 and 2: assigns channel2)
2syncgateway replicator(1 : replicates channel1 and 2: assigns channel2)
3couchbase nodes
Cluster2
2syncgateway(1 : assigns channel3 and 2: assigns channel4)
2syncgateway replicator(1 : replicates channel3 and 2: assigns channel4)
3couchbase nodes

syncgateway and sg-replicate configs below
https://gist.github.com/ArihantRk/40ad192427f9558451c858ccd2497b4c

Each of syncgateway differs by assigning channels, in above config syncgateway assigns channel “channel1” and sg-replicate replicates “channel1”

  1. syncgateways backed by single bucket
  2. sg-replicate have both sg-replication config and sync function without webhook
    sg-replicates does only replicates between clusters. No requests hit to sg-replicates from clients. source : local
    target : fqdn(resolves to one of cluster2 sg-replicates)
  3. Each cluster has fqdn for each of components. hence revisions of same documents can hit any of syncgateway. In above scenario 1st revision hit syncgateway1 and 2nd revision hit 2nd revision.

Each of syncgateway differs in assigning channels. This we have implemented to achieve scalability for sg-replicates.

Please let us know if any config is missing or incorrectly used.

Same issue is filed in Channel filtered sg-replicate stops due to channel removals · Issue #2212 · couchbase/sync_gateway · GitHub