CBRESTORE Issues

Couchbase Server
1

I’m testing the backup capabilities of couchbase. I’ve managed to get that part working, telling it to backup EVERYTHING in the Cluster, since it’s only one node.

My problem is that cbrestore seems to be almost deliberately making me restore on my hands and knees. While cbbackup has a “backup everything” option, cbrestore does not have a “restore everything” option. As far as I can tell from multiple documents I’ve read, I have to MANUALLY create the buckets I want to restore first, and then, ONE BY ONE, restore the contents.

This might be just tedious but simple while I only have a couple test databases, but if I’m in production with 148 buckets or whatever, this restore process is going to be insanely time-consuming and prone to random errors.

I think, not certain, that the docs say that I have to first restore the design documents, and then the data, which doubles the already crazy effort needed.

Can someone comment on this, and/or provide a solution or work-around?

As a side note, I see at the top of this page a note about security best practices. I’m somewhat alarmed that the username and password have to be passed in the clear on the commandline. MySQL has a simple way to specify a credentials file which avoids this. Is there any chance of Couchbase implementing something like this?

2

A couple of things to note:

  1. You do need to create the bucket separately and the reason is that you need to specify the memory quota and bucket password of the bucket you are creating for each bucket. Depending on the cluster you are restoring to you the memory quota may be different than the memory quota of the bucket you originally backed up. Also, in all current releases of Couchbase (latest being 4.6) there is a bucket password and we don’t want to store this as part of the backups since this can be a security issue. You also may want to change the password depending on the where you are restoring to. Imagine for example backing up a production cluster and restoring to a test cluster. You wouldn’t want to restore the bucket password to the test cluster and potentially have to give the password to someone else who should not know it.

I would recommend writing a short script to create the buckets based on how you want your restore to take place for a specific target cluster. You can use the couchbase-cli command in this script to create buckets with the appropriate parameters and then restore the data.

  1. cbrestore is limited to restoring one bucket at a time right now. Our enterprise cbbackupmgr tool will restore all buckets that are backed up automatically with one command, but still does not create the buckets for the reasons I mentioned above.

  2. We limit a cluster to 10 buckets so if you are planning on having 148 in a single cluster you should rethink your application design since this would not work well with Couchbase.

  3. You’ve misread the docs. You can restore indexes, design documents and data in the same command.

  4. You can specify the password through the CB_REST_PASSWORD environment variable in cbrestore. This will hide the password on the command line. I’ll also note that cbbackupmgr, our enterprise tool has the same ability, but also allows for non-echoed stdin on the command line.