Couchbase cloud backup: request limit configuration

Couchbase Server
1

Hello,

We are currently testing Linode Object Storage as the cloud storage solution for our Couchbase database backups. As per the Couchbase documentation on managing backup and restore (Couchbase Backup and Restore Documentation), we have configured the Couchbase backup repository to work with Linode Object Storage.

While the integration works as expected at times, we frequently encounter issues due to rate limits. The following error is observed:

"failed to start uploader: failed to create uploader: failed to create multipart upload: operation error S3: CreateMultipartUpload, failed to get rate limit token, retry quota exceeded, 4 available, 5 requested "

Based on the Linode Object Storage documentation, it appears there is a rate limit of 750 requests per second.

How to configure rate limit for couchbase backup as per our S3 Cloud storage.

2

Hi @hiren.rojasara, which version of cbbackupmgr are you using? We recently disabled client side rate limiting (see this bug). If you upgrade to 7.6.4 that should resolve your issue.

The only other way to control the throughput of the backup, and therefore not hit the rate limit, is to decrease the number of threads (using -t)