Set Up AWS PrivateLinks with Couchbase Capella

What are PrivateLinks?

PrivateLinks are a networking service that allows for the private connection between a cloud service and your virtual network. This connection is made without exposing your data to the public internet, enhancing security by keeping network traffic within the cloud provider’s network. By utilizing PrivateLinks, you can ensure more secure and reliable access to cloud resources, minimizing the risk of external threats and reducing latency by keeping the traffic internal to the cloud provider’s network.

PrivateLinks facilitate uni-directional communication, enhancing the security of network connections by ensuring that traffic can only flow in a single direction. This design helps in preventing unauthorized access and data exfiltration, further solidifying the security benefits of using PrivateLinks for cloud networking.

 Note: XDCR is not compatible with Private Links at the moment.

1. Sign into the AWS Management Console

• • Log in to your AWS Management Console
    
  • You should land on the following page:
    

2. Create a VPC in AWS

• • In the search bar, enter VPC:
    
  • You should land on the following page:
    
    
  • This should open the VPC Dashboard:
     
    
  • Choose your region in the black App Bar. In this example, we are using: US East (N. Virginia) – us-east-1
    
    
  • Click on Create VPC. This opens a page with your VPC Settings.
    • Resources to create: Select VPC and more.
      
    • Name tag auto-generation: Name your VPC.
      
      Note: Recommendation is to use a meaningful name so you can easily identify which VPC is yours later on in the lab. In this example, we use privatelinkaws but we recommend to use a more personalized name for your testing.
    • IPV4 CIDR Block: Accept default.
      
      
    • IPV6 CIDR Block: None.
      
    • Tenancy: Default
      
      
    • Number of AZs: 1
      
      
    • Number of public subnets: 1
      
      
    • Number of private subnets: 0
      
      
    • NAT gateways: 0
      
      
    • VPC Endpoints: S3 Gateway
      
      
    • DNS Options
      • Enable DNS hostnames – Check
      • Enable DNS resolution – Check
        
        
  • Check the Preview in the right side of the page. Once you’re satisfied, click Create VPC.
    
    

• • The VPC is now being created. You see the progress in the following page.
    
    
  • Note down the VPC ID and Subnet ID. You will need it later on. In this case:
    • VPC ID =  vpc-076a949ba49ce9ab6
    • Subnet ID = subnet-08245a74b801954ca

3. Create AWS EC2 instance

• • In the search bar, enter EC2.
    
    
  • You should land into the EC2 Dashboard.
    
    
  • Make sure that you are in the correct region, like the one where you created your VPC.
    
    
  • Click Launch Instance.
    
  • This will open a page with your EC2 settings.
    • Name: Choose name (eg. Privatelink-tester). Same as before, please choose a more meaningful name.
      
      
      • Application and OS Image: Amazon Linux
        
      • Instance Type: t2.micro (free)
        
        
      • Key pair login: click on Create a new key pair.
        • Enter a name for your Key Pair. (e.g. <Yourname>.<Region>)
        • Click Create Key Pair
          
          
        • This will download a .pem file that you must store in a secure location.
          
          
        • Your Key Pair is now selected and available to reuse in the same region.
          
          
      • Network settings
        • Click Edit
          
          
        • VPC: choose the same VPC as the one you’ve created previously. You can filter using your VPC name or using the VPC ID.
          
          
        • Subnet: choose the subnet you created previously. It should be selected automatically as you created only 1 subnet for this VPC.
          
          
        • Auto-assign public IP: Enable
          
          
        • Firewall security group: Create Security Group should be selected
          
          
        • Inbound security group rules:  in the Source type, select MY IP. This will automatically add your IP.
          
          
      • Configure Storage: default.
        
        
  • Check the summary and click Launch instance.
    
  • After a few seconds, you should see that the instance has been successfully launched. Note the EC2 instance ID. In this example: i-0d78dab1f00d85746. Click on the EC2 instance ID.

• • You should land on the Instances page of the region with your instance filtered out. Click on your instance ID. The instance page should be as follows:
    

4. Install AWS CLI Tool

• • To use the AWS CLI Tool, please follow the documentation.
  • You should get temporary credentials.

• • To configure the AWS CLI Tool, simply copy the Option 1, Short-Term Credentials command to set the AWS environment variables and paste it in your terminal. (The below is an example – please paste in your own credentials.)

5. Create a Capella Database

1. • Login into Capella with your corporate email and credentials.
   • Within your project, create a database with the following configuration:
     • Cloud: AWS
     • Region: same as the region where you created your VPC and EC2 instance
     • Name of your choice. Here Privatelinkdev
       
       

• • • Services groups: default is data, index, query and search services. Add the analytics service as well, used later in the lab and keep the of the topology as is.
      
      
    • All other settings can stay as default.
  • Once the database is Healthy, deploy travel-sample bucket:
    • Open the database
    • Navigate to the Data Tools tab
    • On the Import page, click Import under the travel-sample tile.
      
      

B. Stage 1

1. Enable Private Endpoint

• • In your Capella database, open the Settings page, navigate to Private Endpoint and click Enable Private Endpoint.
    
    
  • This action will usually take around 10 minutes to enable Private Endpoint.
    
    
  • When ready, you should see that the Private Endpoint DNS is now available in the Capella UI. Also notice that Private Endpoints are billed hourly for AWS Private Endpoints for this database until you disable this option.
    
    

2.  Add Private Endpoint

• • Click Add Private Endpoint.
    
  • Enter the VPC ID and Subnet ID from Step 2 of the Prerequisites section. Click Next.
    
    

3. Create VPC Endpoint on AWS from CLI

• • Copy the Run Command.
    
    
  • Open a terminal and run the command:
    

• • Note down VPCEndpointId (highlighted) and accept the Endpoint ID in Capella.
    
    

4. Complete Connection

• • Click Finish. Your Endpoint is now being created and in Pending status.
    
    
  • After a minute or 2, the status should be Linked.
    
    

C. Stage 2

1. Enable Private DNS on AWS

• • Navigate to the VPC Dashboard, click Endpoints in the left menu and filter on your VPC Name. You will see at least 2 endpoints:
    • The S3 endpoint created by AWS
    • An unnamed Endpoint, created by the CLI command
      
      
    • Open the unnamed Endpoint (in this case vpce-01dfcbabe1bef175e) and from the Actions drop-down, with the same ID as the one provided in the Capella Private Endpoint Interface Endpoint list. Select Modify private DNS name.
      
      
    • Check Enable for this endpoint and click Save Changes.
      
      

2. Edit Ingress Rules for Private Endpoint on AWS

• • Let’s first get the CIDR Block of your VPC.
    • Click Your VPCs in the VPC Dashboard
    • Filter on your VPC Name.
      
      
    • In the bottom main panel, note the IPV4 CIDR. It should be 10.0.0.0/16
      
      
    • Let’s now get the Security Group of Endpoint
      • Click on Endpoints in the VPC Dashboard
      • Filter on your VPC Name. You will see at least 2:
        • The S3 endpoint created by AWS
        • An unnamed Endpoint, created by the CLI command
          
          
      • Choose the unnamed Endpoint
      • At panel bottom navigate to Security Groups Tab
        
      • Click the Group ID Link and locate Inbound Rule. There will be one Inbound Rule which will be chosen by default. Click Edit Inbound rules.
        
      • Click Add Rule
        • Type: All traffic
        • Source: Custom
        • Source Value: VPC CIDR from Item 1)  – e.g. 10.0.0.0/16
        • Click Save rules.
          

3. Edit Network ACL for Private Endpoint on AWS

• • Let’s now access VPC Dashboard Network ACLs
    • Click Network ACLs in VPC Dashboard
    • Filter on your VPC Name.
    • There will be one unnamed Network ACL chosen by default
    • From the Actions drop-down menu, select Edit Inbound rules
      

• • Let’s now add an Inbound rule.
    • Click Add new rule.
    • Rule number: 101
    • Type:  All traffic
    • Source: the IPv4 CIDR which was obtained from the previous steps (e.g. 10.0.0.0/16).
    • Click Save changes.
      
    • Let’s now add an Outbound rule.
      • From the Actions drop-down menu, select Edit Outbound rules
        
        
      • Click Add new rule.
      • Rule number: 101
      • Type:  Custom TCP
      • Port range: 1024-65535
      • Source: the IPv4 CIDR which was obtained from the previous steps (e.g. 10.0.0.0/16).
      • Click Save changes.
        

D. Test Private Endpoint with an Application

1. Add Capella Security Prerequisites

• • In your Capella database, create Database Credentials. Example: privatelinkcapella/Couchbase123$
    • Open the Settings page of your database
    • In the Database Access page, click Create Database Access
    • This access must have Read + Write privileges on travel-sample bucket
      
      
    • You should see that your database credentials have been created.
      
      
• In Capella, copy the private Endpoint. In this example, this is o3bak2eyqhmw2tq.pl.cloud.couchbase.com
  
  
• In the Capella Settings of your database, open the Security Certificate page and download the certificate.
  
  
• You should get an privatelinkdev-root-certificate.txt file or similar depending on your database name.
  
  

2. Install Python in your AWS EC2 instance

• • Let’s now SSH into EC2 Instance.
    • Navigate to EC2 Dashboard
      
      
    •  Open the Instances page
    • Filter on your EC2 Instance name and select your instance.
      
    • Click Connect
      
      
    • This should open a Connect to Instance page.
      
      
  • Let’s now connect to the instance from your laptop.
    • Click SSH client Tab and follow the instructions to make sure your private key is accessible from your terminal and not publicly viewable.
      

• • • Locate the ssh Example command at the bottom and run it in your terminal.

• • Let’s now install the Python SDK in the instance. Note: the following installation is composed of 3 commands. Make sure you run each of those commands separately in your terminal. Each command will also interactively ask for confirmation of package installation.

• • For the lib, you can also execute the following command.

3. Test a python app connecting to the Capella Private Endpoint

• • Exit ssh to get back to your local laptop or you can open a new shell window

• • Rename the Capella Security .txt certificate you downloaded before as privatelink-cert.pem. Replace the name of the root certificate with your own.

• • mv Downloads/privatelinkdev-root-certificate.txt privatelink-cert.pem

    1	mv Downloads/privatelinkdev-root-certificate.txt privatelink-cert.pem

  • Let’s copy privatelink-cert.pem into your EC2 instance. In the command below:
    • Replace the name of your certificate with your own.
    • Replace the Key Pair file with the one you used to create your VPC.
    • Replace the ec2 name with your own (reuse the same as the one provided in the SSH Client example).
    • Execute the command.
      Note that the following scp command is a single command.
      

• • Copy the following simple python code in your preferred IDE.
    • Replace the private endpoint with the one you created before
    • Replace the credentials with the Capella database credentials you created before
    • Replace the privatelink-cert.pem with your certificate name.
    • Save the file as example.py.

• • Copy the example.py file into your ec2 instance. Same as previously, replace the certificate with your own, as well as the ec2 name with your own.

• • SSH into your EC2 instance again.
• ssh -i "NishantNVirginia.pem" ec2-user@ec2-18-212-126-71.compute-1.amazonaws.com

  1	ssh -i "NishantNVirginia.pem" ec2-user@ec2-18-212-126-71.compute-1.amazonaws.com

  • Check that both your acme-cert.pem and your example.py file are there.

• • Run your Python script. You should get the JSON document airline_10.

4. Test curl commands connecting to the Capella Private Endpoint

• • Still in the EC2 instance, try a SQL++ query using Server REST APIs. Replace the private endpoint with yours.

• • Still in the EC2 instance, try an Analytics query using Server REST APIs.

You have now established a private connection between your AWS VPC and Couchbase Capella using a Private Link! 

Conclusion

Implementing AWS Private Link for Couchbase Capella is crucial for enhancing the security and reliability of your data communication. By establishing a private connection between your VPC and Couchbase Capella, you effectively isolate traffic from the public internet, reducing the risk of exposure to potential threats. This approach ensures that data remains within your secure network boundaries, utilizing AWS’s robust infrastructure to maintain low latency and high throughput. AWS Private Link provides only uni-directional communication, ensuring that data flows securely from your resources to Couchbase Capella without any reverse path. This makes AWS Private Link a highly secure and efficient solution for enterprise-grade database connectivity.

References

• • Docs: Add an AWS PrivateLink Connection
  • Docs: Connect your VPC to services using AWS PrivateLink
  • Get started with a free trial of Couchbase Capella