About the time cost of swap rebalance

Couchbase Server
1 
  I have a cluster with 3 nodes.

node1 : 10.130.24.43
node2 : 10.130.24.77
node3 : 10.130.24.90
3 Buckets were created.
I replaced the machine 10.130.24.90 with the other machine(10.130.24.91) in two scenes, and the swap rebalance taked place:
Scene 1:
All 3 buckets have no items.
When the swap rebalance process were completed, it cost about 10 minuts. Is this result normal?
Scene 2:
2 buckets have no items, 1 with 1027524 items(just key-value format with string type, no views; 200M per Node RAM Quota, data size less than 70M per node).
And also the swap rebalance takes about 10 minuts .

So, How to explain the phenomenon above? Does it mean that the time cost of swap rebalance just involves the number of vBuckets with a small amount of items?

Environment:
Linux 2.6.39 x86_64
Read and Write speed are above 600MB/s (it’s very good, isn’t it?)
CPU 16 cores
Couchbase 2.2.0

2

You would think "Hey the bucket is empty why does it take 10 minutes to re balance no data?"
Answer. When you talk buckets in CB what you are really talking about vBuckets.

1 Bucket = 1024 vbuckets AND if you have replicas you have another 1024 vbuckets.
You can count them in the data folder /opt/couchbase/couchbase/data//{file-here}

Example - In a CB cluster if you have 3 buckets with 1 replica each your are talking about moving
3 X 1024 + 3 x 1024 (replica vbuckets) = 6144 files.

"So what 6144 files , but they are empty so it should take a less then a seconds."
Well also during re-balance by default every 6 vbuckets you move you will compact the content of those buckets.
“But compacting a empty bucket should take less then a second?” … Yes.

So 6144 vbucket * 0.1 seconds(moving and compact per vbucket) = 10min 24 seconds.

In reality you don’t move all vbuckets around , network speed 10 Mbs , 100Mbs , 1Gbs or 10Gbs, and Number of CPU, SSD or HD or shared HD(Amazon’s EBS). 10-17 min. is what you should be getting.

3

Thanks a lot for your answer. It does make sense to me.