Couchbase6.x join to cluster and rebalance slow take a long time and get failed

[zepeng_chen]

question describtion:
our customer rebalane one node cost 3hours and failed
my workmates saids that the old version like version 5.x and version4.x seems not low like this
so please help me to confirm, why the couchbase is so slow when rebalancing

we created 21 buckets using restful api like below

curl -X POST \
-u couchbase:couchbase_pass \
-d authType=sasl \
-d bucketType=couchbase \
-d flushEnabled=0 \
-d name=bucketName\
-d parallelDBAndViewCompaction=false \
-d ramQuotaMB=100 \
-d replicaIndex=1 \
-d replicaNumber=1 \
-d saslPassword=bucketNamePass \
-d threadsNumber=8 \
http://${COUCH_HOST}/pools/default/buckets

curl -X PUT \
-u couchbase:couchbase_pass \
-d name=geofence-msgmeta \
-d roles=bucket_full_access[bucketName] \
-d password=bucketNamePass \
http://${COUCH_HOST}/settings/rbac/users/local/bucketName

couchbase server info:
hardware info: 8 core cpu 32 g memory 200g disk
version 6.5 enterprise and version6.6 commiunity

test: two nodes cluster scenarios
the cluster server just create 21 buckets and no data save, just join the new node like above 3 different scenarios

1. version6.5en as cluster and join version6.6ce
2. version6.5en and version6.5en
3. version6.6ce and version6.6ce

the memeory is enough

get cpu info:

get disk info:

get rebalance info take long time:

the rebalance faild info

[Nebasuke]

Hey @zepeng_chen,

Sorry to hear you’re experiencing slowness while rebalancing.
Could you give a few more details on the data in the buckets? Particularly given the relatively low ramQuota.

Have you seen Rebalance | Couchbase Docs and the corresponding settings here: General | Couchbase Docs?

It might be worth trying to limit some of the rebalancing options and adding retries. You can then access Rebalance | Couchbase Docs to inspect further details on why some of this slowness might have occurred.