The enterprise grade backup and restore tool, cbbackupmgr, provides a high performance, consistent, reliable and secure solution for backing up and restoring business critical data for enterprise customers.

Backing up Exponential Data Growth at Speed

Exponential data growth, the need for more frequent data protection, and a variety of other challenges requires the backup tool to be highly efficient and performant.

One of the core requirements for the new tool is the ability to backup large datasets (in terabytes) and handle an on-going explosive data growth.

With the introduction of Couchbase Server 4.5, backup capabilities have expanded beyond just taking full backups or incremental backups. The new tool enables the creation of a full backup valid to the point of the incremental backup, simply by merging the incremental backups into single full backup. This approach achieves an incremental forever strategy by saving incremental updates versus taking a full backup.

Let’s take an example of a large deployment with

  • 64 TB of data

  • spread across 60 nodes in a cluster

  • more than 100 billion keys

  • 1 TB change in data per day

In this scenario, a periodic full backup of the cluster every day is not a viable option. The reason is even if the backup tool runs with a speed of 800MB/sec it will still need 24hours to complete the backup operation. Instead, the recommended backup strategy for this scenario is the Periodic Merge strategy. Here, cbbackupmgr takes incremental backups on the 1TB changed data and periodically merges all those incremental backups offline to create a full backup. The backup operation with the same speed of 800MB/sec will now complete in less than 21minutes.

Let’s go through an example to demonstrate the periodic merge strategy.

In this example we have a four nodes cluster and “UserProfileStore” bucket which has 2 million documents each of 1 KB document size.

Screen Shot 2016-05-30 at 4.21.52 PM.png

Screen Shot 2016-05-30 at 5.01.43 PM.png

The below screenshot shows the first incremental backup file for UserProfileStore under backup archive repository.

Screen Shot 2016-05-30 at 5.07.11 PM.png

As data gets changed or more data gets added the cbbackupmgr tool by default performs incremental backups to back up only the new data or changed data.

Screen Shot 2016-05-30 at 5.42.12 PM.png

Now you can merge these two incremental backups into single full backup offline you can do that using cbbackupmgr merge command.

Screen Shot 2016-05-30 at 7.02.38 PM.png

As you can see from the above screenshot cbbackupmgr merges incremental backups into a single full backup offline. This is a great backup strategy which will have the lowest cluster overhead and require the least amount of time. The periodic merge strategy takes full advantage of the fact that cbbackupmgr only takes incremental backups as well as its ability to merge incremental backup. You can read more about merge here http://developer.couchbase.com/documentation/server/4.5/backup-restore/cbbackupmgr-merge.html

Learn more about new Enterprise Backup and Restore tool using the following links –

Author

Posted by The Couchbase Team

Jennifer Garcia is a Senior Web Manager at Couchbase Inc. As the website manager, Jennifer has overall responsibility for the website properties including design, implementation, content, and performance.

All Posts Website

Leave a reply