If you’re running a Couchbase Server deployment, did you know you can easily migrate to Couchbase Capella on AWS and instantly gain the scale and efficiency of a managed cloud service model while offloading the work of installations, upgrades and general database infrastructure maintenance? Couchbase Capella is a fully managed NoSQL Database-as-a-Service (DBaaS) for Couchbase Server. It provides massive scale, allows you to create multi-node clusters in minutes, and supports all the Couchbase features you use today, such as SQL++ (SQL for JSON), Full Text Search, Eventing and Analytics. Couchbase Capella gives you all the power of Couchbase while freeing you up to focus on your application, not your database deployment. There has never been a better time for migrating from Couchbase Server to Couchbase Capella on AWS!
Your guide to the cloud
If you’re ready to make the move to Couchbase Capella, we have a great resource for you! Working with AWS, we recently launched this AWS Prescriptive Guidance pattern, which describes a repeatable process for migrating data and indexes from a self-managed Couchbase Server Enterprise Edition environment to Couchbase Capella on the Amazon Web Services (AWS) Cloud.
The pattern should be used by any Couchbase customer who is new to Couchbase Capella and who needs to understand the order of activities for migration.
The basic steps are:
Prepare the migration
- Review configurations for the nodes and buckets on your self-managed cluster on Couchbase Server.
- Record IP addresses for your self-managed cluster on Couchbase Server.
Deploy and configure resources on Couchbase Capella
- Configure and deploy the destination cluster on Couchbase Capella.
- Use Self-Managed cluster and bucket configurations gathered in step 1 as a general guide for Couchbase Cloud cluster configurations.
- Configure data access on Couchbase Capella.
- Create a Database Credentials user for the destination destination cluster.
- Record the Couchbase Capella cluster endpoint.
- Save the Couchbase Capella cluster Root Certificate.
- Save the self-managed Couchbase Server Root Certificate.
- Add “allowed IP” addresses to your Couchbase Capella cluster for connecting from your self-managed Couchbase Server cluster.
- Create target buckets, scopes, and collections on Couchbase Capella.
Migrate the data and indexes
- Connect self-managed Couchbase Server cluster to Couchbase Capella cluster.
- From the self-managed Couchbase Server cluster, set up XDCR Replication between the source and destination buckets on Capella, which migrates the data.
- Migrate indexes using Couchbase Shell from the self-managed cluster to the Couchbase Capella cluster.
- Verify and validate that the data and indexes are successfully migrated.
Since the pattern already covers the individual granular steps for migration, in this post I’ll discuss my experience going through them myself, and I’ll point out a few tips along the way. For the exercise it’s assumed I already have a Couchbase Capella account, as well as a self-managed Couchbase Server cluster, as such I’ll focus mainly on the actual process of migrating from one environment to the other.
Review The Source and Destination For Migration
A first step for any migration is careful planning, in short, you want to make sure that you create a destination environment that matches the source as closely as possible, which makes for smoother migration. The pattern describes general steps for reviewing your source environment, then using the gathered information to create a similar environment on the destination. Remember that Couchbase Cloud uses multi-dimensional scaling best practices. Services and nodes can only be chosen according to deployment best practices. This could mean that you can’t exactly match your self-managed Couchbase Server cluster’s configurations.
It’s important to note that the pattern provides general guidance for assessing the size of your self-managed Couchbase Server cluster and using it to approximate the required configurations for your Couchbase Capella destination cluster. For assistance with a more detailed Couchbase Cloud sizing exercise please contact Couchbase Capella or your Couchbase customer account team.
Once your destination cluster is created, you create your destination buckets. It’s a best practice to create your destination buckets with the same names, memory settings, and conflict resolution methods of the buckets in your self-managed Couchbase Server cluster.
My Environments
To run through the steps, I first established my source – a three node cluster running Couchbase Server Enterprise Edition 7.0.3 deployed on Microsoft Azure Virtual Machines. Next I established my destination cluster on Couchbase Capella – using the Control Plane I created a cloud on AWS, then for my cluster I used the “Evaluation Optimized” configuration template, giving me 3 nodes with all services running on each node. With source and destination environments established, I was ready to begin the process. For the exercise I chose to migrate the “travel-sample” sample database that comes bundled with Couchbase Server, since it has a fair amount of data and a lot of indexes, what better way to test the steps?
On Couchbase Server I simply installed the “travel-sample” sample bucket. Then on Couchbase Capella, I manually created a bucket called “travel-sample” to serve as the destination for migrated data. Note that I did not import the “travel-sample” sample database that comes with Couchbase Capella, I simply created an empty bucket with the same name as my source bucket on Couchbase Server to serve as the destination for migration.
Connect your Self-Managed Clusters to Couchbase Capella
To make the connection between my self-managed Couchbase Server cluster and Couchbase Capella, I leveraged the XDCR (Cross Data Center Replication) feature.
The pattern walks you through the process of connecting your self-managed Couchbase Server cluster to your Capella cluster, which is quick and easy.
Once I had the settings in place, I clicked submit and in just a few seconds I had connected my self-managed cluster to Couchbase Capella. Now I was ready to migrate data!
XDCR Rocks!
After my self-managed cluster was connected with my Capella cluster, I was able to set up an XDCR replication between the source and target buckets. In my self-managed Couchbase Server console, I set up the replication by simply selecting the “travel-sample” bucket as the source, then selecting the newly created empty “travel-sample” bucket that I created on Couchbase Capella as the destination. Once I saved the replication settings, data began replicating from my self-managed cluster to my Couchbase Capella cluster almost immediately, it could not have been any easier! And the replication is secure and resilient, allowing for efficient migration of even the largest buckets of data.
Index Migration With cbshell? Shell Yeah!
Next I set out to migrate indexes. Of course it’s very easy to manually migrate indexes by copying the index definition statements from your self-managed cluster and running them in the Query Workbench on Couchbase Capella, but I had enough indexes that I wanted a more streamlined approach. Couchbase Shell provided the perfect method!
Couchbase Shell, also known as cbshell, is a modern command line shell for Couchbase Server and Couchbase Capella. The cool thing about cbshell is that you can install it anywhere to perform the migration, all it needs is internet access to both the source and destination clusters, and you can use simple commands in a terminal to migrate the indexes.
On the system where you install cbshell, make sure that Couchbase SDK and OpenSSL are also installed, with both mapped in the system user profile PATH.
Once cbshell was installed, I was able to create a config dotfile in order to connect to my source and destination clusters at the same time, and easily switch context between them. The config file contains connection information for both clusters, as well as root certificates to ensure a secure index migration. The pattern contains a sample config file that you can use by simply copying the sample, editing the properties in the file to add your own cluster endpoints, user/password and root certificates, then saving it to the .cbsh folder.
Once the config file was saved, I launched cbshell in a terminal and set the context to my self-managed cluster.
Then I ran a simple command to migrate the index definitions from my self-managed cluster to my Couchbase Cloud cluster.
And the final step was to use a simple cbshell command to build the index definitions on my Couchbase Capella bucket.
And that was it, my indexes were all migrated! Couchbase Shell was so easy, and it was fun to watch the indexes appearing magically in Couchbase Capella after running the cbshell commands.
To test the migration I ran a few sample SQL++ queries on both the source and destination clusters and verified the same results on both environments, SUCCESS!
The experience was straightforward and in the end went very fast after setting up the environments – of course the speed of migration can vary depending on data volume. The pattern is designed as a repeatable process, and is so easy I now use it regularly when moving demo databases from my self-managed Couchbase Server clusters to Couchbase Capella.
Take advantage of the AWS Workload Migration Program for Couchbase and $ave!
If you’re ready to migrate to Couchbase Capella on AWS, you may qualify for the AWS Workload Migration Program for Couchbase, which offers significant savings for eligible customers on the first year AWS infrastructure spend in the form of AWS credits for Couchbase Capella migrations.
The program supports your migration journey by leveraging best practices and a repeatable process and methodology for migrating to Couchbase on AWS infrastructure. Eligible workloads receive an AWS Credit to offset the initial cost of using the AWS Infrastructure-as-a-Service. It is intended to help customers migrate their database workloads to Couchbase on AWS across a variety of use cases, including from self-managed Couchbase Server deployments to Couchbase Capella.
Click HERE to learn more and find out if you are eligible for the program!