Rafael Ugolini is a full stack software developer currently based in Brussels, Belgium. He has been working with software development for more than 10 years and is lately focused on designing web solutions and developing using Python and JavaScript. Rafael Ugolini is Senior Software Developer at Famoco.
Introduction
Docker is a great project that is helping developers around the world run applications in containers. This not only helps shipping software faster, but it also results with the famous “it works in my machine” phrase. In this article I will explain how to create a modular Couchbase image that doesn’t require any Web UI interaction to have a ready-to-go database for you.
All the code is available online here.
Dockerfile
The first step is to create the Dockerfile.
Couchbase Version
|
1 |
FROM couchbase/server:enterprise-4.6.1 |
This example is based on Couchbase Server Enterprise 4.6.1, but you can feel free to change to the specific version you are running in your environment.
Memory Configuration
|
1 2 3 |
ENV MEMORY_QUOTA 256 ENV INDEX_MEMORY_QUOTA 256 ENV FTS_MEMORY_QUOTA 256 |
All the values here are in MB:
– MEMORY_QUOTA: per node data service ram quota
– INDEX_MEMORY_QUOTA: per node index service ram quota
– FTS_MEMORY_QUOTA: per node index service ram quota
Services
|
1 |
ENV SERVICES "kv,n1ql,index,fts" |
These are the services that will be available for the node created:
– kv: Data
– n1ql: Query
– index: Index
– fts: Full-Text Search
Credentials
|
1 2 |
ENV USERNAME "Administrator" ENV PASSWORD "password" |
Username and password to be used in Couchbase Server.
Cluster Options
|
1 2 |
ENV CLUSTER_HOST "" ENV CLUSTER_REBALANCE "" |
These options are only used if you want to add more than one node in the cluster.
– CLUSTER_HOST: hostname of the cluster for this node to join
– CLUSTER_REBALANCE: set “true” if you want the cluster to rebalance after the node is joined
Entrypoint
|
1 2 3 |
COPY entrypoint.sh /config-entrypoint.sh ENTRYPOINT ["/config-entrypoint.sh"] |
The Couchbase Server image already ships with an entrypoint.sh script and we don’t want to override it. The trick here is to copy our version of entrypoint.sh to /config-entrypoint.sh, run Couchbase Server entrypoint.sh in the background, and after configuring the node attach the script back to the original ENTRYPOINT.
Entrypoint
The ENTRYPOINT is used in combination with the original script from the Couchbase Server image. Let’s go line by line to understand how it works.
Initialize Couchbase Server
|
1 2 3 4 |
# Monitor mode (used to attach into couchbase entrypoint) set -m # Send it to background /entrypoint.sh couchbase-server & |
First we use set -m to enable job control, process running in background (like the original ENTRYPOINT) run in a separate process group. This option is turned off by default in non-interactive mode, like scripts.
Util Functions
|
1 2 3 4 5 |
# Check if couchbase server is up check_db() { curl --silent http://127.0.0.1:8091/pools > /dev/null echo $? } |
This function is used to check when Couchbase Server starts answering HTTP calls.
|
1 2 3 4 5 6 7 |
# Variable used in echo i=1 # Echo with numbered_echo() { echo "[$i] $@" i=`expr $i + 1` } |
This is just a util function, add a number before any echo in the script to count the steps taken automatically.
|
1 2 3 4 5 6 7 8 |
# Parse JSON and get nodes from the cluster read_nodes() { cmd="import sys,json;" cmd="${cmd} print(','.join([node['otpNode']" cmd="${cmd} for node in json.load(sys.stdin)['nodes']" cmd="${cmd} ]))" python -c "${cmd}" } |
In order to parse the output of the nodes in Couchbase Server API, I’m using a function which runs ython to read STDIN, transform it to JSON and the Couchbase nodes. This is used for rebalancing.
Configure the Node
|
1 2 3 4 5 6 7 8 |
# Wait until it's ready until [[ $(check_db) = 0 ]]; do >&2 numbered_echo "Waiting for Couchbase Server to be available" sleep 1 done echo "# Couchbase Server Online" echo "# Starting setup process" |
The first step is to wait until the server is ready, then using the function numbered_echo you can see how long it took for Couchbase Server to have the API calls available.
|
1 2 3 4 |
HOSTNAME=`hostname -f` # Reset steps i=1 |
Then we set a variable HOSTNAME to be used in all the API calls we do and we also reset the counter from numbered_echo by setting it to 1.
|
1 2 3 4 5 6 7 8 |
numbered_echo "Initialize the node" curl --silent "http://${HOSTNAME}:8091/nodes/self/controller/settings" \ -d path="/opt/couchbase/var/lib/couchbase/data" \ -d index_path="/opt/couchbase/var/lib/couchbase/data" numbered_echo "Setting hostname" curl --silent "http://${HOSTNAME}:8091/node/controller/rename" \ -d hostname=${HOSTNAME} |
First thing to do is to set up disk storage configuration and then we set the hostname.
Joining a Cluster
|
1 2 3 4 5 6 7 8 |
if [[ ${CLUSTER_HOST} ]];then numbered_echo "Joining cluster ${CLUSTER_HOST}" curl --silent -u ${USERNAME}:${PASSWORD} \ "http://${CLUSTER_HOST}:8091/controller/addNode" \ -d hostname="${HOSTNAME}" \ -d user="${USERNAME}" \ -d password="${PASSWORD}" \ -d services="${SERVICES}" > /dev/null |
If CLUSTER_HOST is set, the script will try to add the current container to the cluster.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
if [[ ${CLUSTER_REBALANCE} ]]; then # "Unexpected server error without the sleep 2 sleep 2 numbered_echo "Retrieving nodes" known_nodes=$( curl --silent -u ${USERNAME}:${PASSWORD} http://${CLUSTER_HOST}:8091/pools/default | read_nodes ) numbered_echo "Rebalancing cluster" curl -u ${USERNAME}:${PASSWORD} \ "http://${CLUSTER_HOST}:8091/controller/rebalance" \ -d knownNodes="${known_nodes}" fi else |
After adding the node into the cluster, the script can also check for the variable CLUSTER_REBALANCE to see if it needs to rebalance the cluster automatically. This is where we use the Python function to read the nodes from /pools/default endpoint.
Not joining a cluster
|
1 2 3 4 5 |
numbered_echo "Setting up memory" curl --silent "http://${HOSTNAME}:8091/pools/default" \ -d memoryQuota=${MEMORY_QUOTA} \ -d indexMemoryQuota=${INDEX_MEMORY_QUOTA} \ -d ftsMemoryQuota=${FTS_MEMORY_QUOTA} |
Memory settings for the services.
|
1 2 3 |
numbered_echo "Setting up services" curl --silent "http://${HOSTNAME}:8091/node/controller/setupServices" \ -d services="${SERVICES}" |
Services to be used by the node.
|
1 2 3 4 5 6 7 |
numbered_echo "Setting up user credentials" curl --silent "http://${HOSTNAME}:8091/settings/web" \ -d port=8091 \ -d username=${USERNAME} \ -d password=${PASSWORD} > /dev/null fi |
Set up the credentials for the node.
Finalize
|
1 2 3 |
# Attach to couchbase entrypoint numbered_echo "Attaching to couchbase-server entrypoint" fg 1 |
To end the script, we attach it to the original ENTRYPOINT.
Example
To demonstrate how to use it, I will be using the image registered in Docker Hub with the code here.
Single node
|
1 2 3 4 5 6 7 |
docker run -ti --name couchbase-server-nosetup \ -h node1.cluster \ -p 8091-8093:8091-8093 \ -p 11210:11210 \ -p 4369:4369 \ -p 21100-21299:21100-21299 \ rugolini/couchbase-server-nosetup |
This runs a single node using the minimum required memory and the default credentials (Administrator/password) registered in the image. All the network ports Couchbase Server uses are exposed as well.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
docker run -ti --name couchbase-server-nosetup \ -h node1.cluster \ -p 8091-8093:8091-8093 \ -p 11210:11210 \ -p 4369:4369 \ -e MEMORY_QUOTA=512 \ -e INDEX_MEMORY_QUOTA=512 \ -e FTS_MEMORY_QUOTA=512 \ -e USERNAME=admin \ -e PASSWORD=adminadmin \ -p 21100-21299:21100-21299 \ rugolini/couchbase-server-nosetup |
The command above plays a little with the environment variables available in the Dockerfile.
Cluster
In this example, we will connect 3 nodes in the cluster.
|
1 |
docker network create couchbase |
We must first create a network Couchbase where we will connect all nodes.
|
1 2 3 4 5 6 7 8 |
docker run -ti --name node1.cluster \ -p 8091-8093:8091-8093 \ -p 11210:11210 \ -p 4369:4369 \ -p 21100-21299:21100-21299 \ -h node1.cluster \ --network=couchbase \ rugolini/couchbase-server-nosetup |
Then we create the first node.
|
1 2 3 4 5 6 |
docker run -ti --name node2.cluster \ --network=couchbase \ -h node2.cluster \ -e CLUSTER_HOST=node1.cluster \ -e CLUSTER_REBALANCE=true \ rugolini/couchbase-server-nosetup |
Since all the network ports are exposed in the first node, it’s not necessary to expose them here.
Attention to the detail that CLUSTER_HOST is set to node1.cluster which is the hostname of the first node and CLUSTER_REBALANCE is also set to true. Once the node is added to the cluster, it will rebalance automatically.
|
1 2 3 4 5 |
docker run -ti --name node3.cluster \ --network=couchbase \ -h node3.cluster \ -e CLUSTER_HOST=node1.cluster \ rugolini/couchbase-server-nosetup |
The node3 is also added to the cluster, but since CLUSTER_REBALANCE wasn’t set, it will require manual rebalance of the cluster for it to become available.
This post is part of the Couchbase Community Writing Program
