When operating a Couchbase Server cluster, or any other kind of service, it is critical to keep backups, whether it be daily, weekly, monthly or something else. However, after you’ve taken that backup, where does it end up? Saving your cluster data to the server hosting Couchbase probably isn’t a good idea because if it fails you’ll lose your backup.
There are plenty of remote storage solutions available. For example you could rsync the backups to a dedicated backup server or you could upload them to Amazon S3 or Amazon Glacier. The possibilities don’t end with those three solutions.
I’ve been playing around with an open source object storage solution called Minio which is based on the same APIs as Amazon S3, but it is something you can deploy in-house.
Minio, like other object storage solutions, protects against hardware failure and silent data corruption using erasure code making it a solid solution for backup storage.
Let’s take a look at how you might transfer your backups to Minio if you’re operating your own distributed object storage. Starting by taking a backup might be a good place to start.
If you’re using Couchbase Server Enterprise Edition you’ll want execute the following:
|
1 2 |
cbbackupmgr config --archive </path/to/archive/directory> --repo cbbackupmgr backup --cluster http://:8091 --username --password --archive </path/to/archive/directory> --repo |
The above commands will configure an archive and backup any particular cluster to that archive. Remember, backups using cbbackupmgr will be incremental, but it shouldn’t stop us from backing them up.
If you’re not already running Minio server, download the appropriate binary for your platform. You shouldn’t be running Minio server on one of your Couchbase Server nodes. From the Terminal, execute the following:
|
1 |
./minio server /path/to/storage/location |
Make note of the access key and secret key values because they will be necessary when configuring the Minio client application.
Now you can download the Minio client application to the Couchbase Server node that you wish to run the backups on.
Before you can push your backups, you’ll want to configure the client to connect to your Minio server. Execute the following from the Terminal to make this possible:
|
1 |
./mc config host add backups http://:9000 |
A fresh server install of Minio will have no object buckets. Before the data can be pushed, execute the following to create a new bucket:
|
1 |
./mc mb backups/couchbase |
Now there are several ways to push data to this couchbase storage bucket. We can mirror changes to the bucket by doing the following:
|
1 |
./mc mirror /path/to/couchbase/archive backups/couchbase |
Some sources online suggest that object storage should be a write once, read many solution. If following this strategy, you probably don’t want to mirror your backups since they are incremental and files will change rather than being created. Instead you can do something like the following:
|
1 |
tar -czf - /path/to/couchbase/archive | ./mc pipe backups/couchbase/archive-`date +%Y-%m-%d`.tar.gz |
The above line will TAR the backup archive and pipe it to the Minio server. Each pushed backup will have a date stamp so you can keep a collection of backups. This process can be done on a schedule by creating a cron task on your client operating system.
More information on creating cluster backups of Couchbase can be found in the Couchbase Developer Portal.
