Couchbase Meets .Net Core and Docker

Brant Burnett is a Couchbase Expert, systems architect, and .Net developer experienced in desktop and web full stack development.  For the last 12 years, he has been a working with CenterEdge Software, a family entertainment software company based in Roxboro, NC.  Brant is experienced in developing applications for all segments of their software suite.  Over the last 4 years, he has worked to transition the company’s cloud infrastructure from a Microsoft SQL platform to a pure Couchbase NoSQL platform.  Through his work at CenterEdge, Brant has been able to focus on creating serious software solutions for fun businesses.

With the release of the Couchbase .NET SDK 2.4.0, Couchbase now has official support for .NET Core. This opens up a wide new world for .NET Couchbase developers. In particular, we can now use Docker to easily manage our applications and improve our deployment process, something previously reserved for the likes of Java and Node.js.

At CenterEdge Software, we’re quickly moving to break our ASP.NET monolithic applications into Docker-based ASP.NET Core microservices. We’re very excited about the new possibilities that it provides, and the improvements to our application’s robustness and ease of deployments.  Hopefully, this overview of the approaches we’re using to make this transition will help others follow suit.

Configuration and Environments

In most ASP.NET Core applications, configuration is based on settings read from the appsettings.json file in the root of your project. These settings are then overridden by environment-specific settings (such as appsettings.Development.json). These settings can then be overridden in turn by environment variables present when the application is started.

At CenterEdge, we’ve defined the .NET Core environments to mean specific things relative to our real-world environments. Note that you can also add your own environment names, you don’t need to use the defaults, but the defaults worked for us.

• Development – Local machine development using Visual Studio. The configuration points to Couchbase Server on the local machine, etc.
• Staging – In-cloud testing environments
• Production – Both the pre-production environment (for final tests before deployment) and the final production environment. These environments are generally the same as Staging but with lighter logging by default.

So our base appsettings.json usually looks something like this:

{
“Logging”: {
“IncludeScopes”: false,
“LogLevel”: {
“Default”: “Debug”,
“System”: “Information”,
“Microsoft”: “Information”,
“Couchbase”: “Debug”
}
},
“Couchbase”: {
“Buckets”: [
{
“Name”: “my-bucket”
}
]
}
}

The above configuration uses localhost for Couchbase Server by default, since we don’t have any server URLs specified. Next we’ll create appsettings.Staging.json and/or appsettings.Production.json like this:

{
“Logging”: {
“LogLevel”: {
“Default”: “Information”,
“Couchbase”: “Information”
}
}
“CouchbaseServiceDiscovery”: “_couchbase._tcp.services.local”
}

This reduces our log levels to something more reasonable, and also has a setting for service discovery (discussed later).

Dependency Injection

ASP.NET Core uses a lot of techniques that are different from the traditional ASP.NET model, which means integrating Couchbase into .NET Core applications is a bit different. In particular, ASP.NET Core is built from the ground up to work with dependency injection.

To support this, we use the Couchbase.Extensions.DependencyInjection package to bridge the gap between Couchbase SDK bucket objects and the dependency injection system. Couchbase is registered during ConfigureServices in the Startup class, passing the configuration section from above. We also add some shutdown code to close connections when the web application is exiting.

public void ConfigureServices(IServiceCollection services)
{
// Register Couchbase with configuration section
services

.AddCouchbase(Configuration.GetSection(“Couchbase”))

.AddCouchbaseBucket<IMyBucketProvider>(“my-bucket”);

if (!Environment.IsDevelopment())
{
services.AddCouchbaseDnsDiscovery(Configuration[“CouchbaseServiceDiscovery”]);
}

services.AddMvc();
// Register other services here
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env,

ILoggerFactory loggerFactory,
IApplicationLifetime applicationLifetime)
{
// …

// Not showing standard application startup here

// …

// When application is stopped gracefully shutdown Couchbase connections
applicationLifetime.ApplicationStopped.Register(() =>
{
app.ApplicationServices.GetRequiredService<ICouchbaseLifetimeService>().Close();
});
}

You can access any bucket in any controller by injecting IBucketProvider via the constructor. However, you may note that the above example also makes a call to AddCouchbaseBucket<IMyBucketProvider>(“my-bucket”).

This method allows you to register an empty interface inherited from INamedBucketProvider:

public interface IMyBucketProvider : INamedBucketProvider
{
}

And then inject it into a controller or business logic service. It will always provide the same bucket, based on the configuration you provided during ConfigureServices.

public class HomeController : Controller
{
private readonly IMyBucketProvider _bucketProvider;

public HomeController(IMyBucketProvider bucketProvider)
{
_bucketProvider = bucketProvider;
}

public IActionResult Index()
{
var bucket = _bucketProvider.GetBucket();

var result =
await bucket.QueryAsync<Model>(
“SELECT Extent.* FROM my-bucket AS Extent”);

if (!result.Success)
{
throw new Exception(“Couchbase Error”, result.Exception);
}

return View(result.Rows);
}
}

Service Discovery

When working with microservices, service discovery is a common problem. Each environment that you run will tend to have different services at different endpoints. Couchbase is one such service, which may exist at a different address in each environment. There are many solutions for service discovery, but at CenterEdge we decided to stick with a simple solution for now, DNS SRV records.

To support this, we use the Couchbase.Extensions.DnsDiscovery package. This package will find DNS SRV records which list the nodes in the cluster. To support this, we create a private DNS domain in AWS Route 53 named “services.local”, and create a SRV recordset named “_couchbase._tcp.services.local” that has the list of Couchbase nodes. The Route 53 recordset looks something like this:

10 10 8091 couchbasedata1.int.dev.centeredgeonline.com
10 10 8091 couchbasedata2.int.dev.centeredgeonline.com
10 10 8091 couchbasedata3.int.dev.centeredgeonline.com

In the above example for ConfigureServices in startup, you may have noticed the following section:

if (!Environment.IsDevelopment())
{
services.AddCouchbaseDnsDiscovery(Configuration[“CouchbaseServiceDiscovery”]);
}

This will replace any servers passed via configuration with the servers found by looking up the DNS SRV record. We also provide the DNS name via configuration, making it easy to override if necessary. We specifically don’t use this extension in our Development environment, where we’re using localhost to access the Couchbase cluster.

That’s Cool, What About Docker?

So far, everything we’ve done is applicable to ASP.NET Core in general, and is not necessarily specific to Docker. So how do we move from a general application to one that runs in a Docker container?

First, there a few preparatory steps you’ll need to complete on your development machine:

1. Ensure that you have Hyper-V enabled in Windows
2. Install Docker for Windows
3. Configure a Shared Drive in Docker for the drive where your application lives
4. Install Visual Studio Tools for Docker
5. Ensure that Docker is started (you can configure Docker to autostart on login)

Now, you’re ready to go. Just right click on your project in Visual Studio, and go to Add > Docker Support. This adds the necessary files to your project.

Add Docker Support

While several files are added, there are some files that are particularly important. The first file I’d like to point out is Dockerfile:

FROM microsoft/aspnetcore:1.0.1
ENTRYPOINT [“dotnet”, “TestApp.dll”]
ARG source=.
WORKDIR /app
EXPOSE 80
COPY $source .

There are two key lines in this file that you might need to modify:

FROM microsoft/aspnetcore:1.0.1

You must change this line if you’re using a different version of .NET Core, such as 1.0.3 or 1.1.0. The version tag on this line should match the version of .NET Core used in your project.json file.

ENTRYPOINT [“dotnet”, “TestApp.dll”]

If you rename your project, it will output a different DLL filename. Change this line to reference the correct DLL filename.

The next file is docker-compose.yml. This file, along with some related files, controls the nature of the Docker containers started when you click Run. We’ll need to make a change in docker-compose.yml to get the Couchbase Server connection working.

Our configuration for the Development environment is trying to access “localhost” to access Couchbase Server. This approach works fine if the application is running in IIS Express. However, inside a Docker container “localhost” no longer points to your development computer.  Instead it refers to the isolated Docker container, much like it would within a virtual machine.

To fix this, we need to add an environment section to docker-compose.yml to use your computer’s name instead of “localhost”:

version: ‘2’

services:
testapp:
image: user/testapp${TAG}
build:
context: .
dockerfile: Dockerfile
ports:
– “80”
environment:
– Couchbase:Servers:0=http://$COMPUTERNAME:8091/

Just add the last two lines above to your file. Docker Compose will automatically substitute $COMPUTERNAME with the name of your computer, which is helpful when sharing the application with your team via source control.

Now you’re ready to test in Docker. Just change the Run drop down in your Visual Studio toolbar to Docker instead of IIS Express before you start your app. It even supports debugging and shows logs in the Debug window.

If you want to get really fancy, you can also tweak docker-compose.yml to do things like launch additional required containers, override other settings via environment variables, and more. For example, at CenterEdge we use this approach to launch additional microservices that are dependencies of the application being developed.

Deployment

Your exact deployment approach will vary depending on your Docker platform. For example, CenterEdge uses Amazon AWS, so we’ll deploy using EC2 Container Service. Regardless of your platform of choice, you’ll need to make a Docker image from your application and publish it to a Docker container registry.

At CenterEdge we’ve added this to our continuous integration process, but here’s a summary of the steps involved:

1. Run “dotnet publish path/to/your/app -c Release” to publish your application. This will publish to “bin/Release/netcoreapp1.0/publish” by default, but this can be controlled with the “-o some/path” parameter. For .NET Core 1.1, it will be netcoreapp1.1 instead of netcoreapp1.0 by default.
2. Run “docker build -t myappname path/to/your/app/bin/Release/netcoreapp1.0/publish” to build a Docker image. It will be tagged as “myappname”.
3. Run “docker tag myappname yourdockerregistry/myappname:sometag” to tag the Docker image for your Docker registry. Substitute “yourdockerregistry” with the path to your Docker registry. For Docker Hub, this is just your username. Substitute “sometag” with tag you want to use, such as “latest” or “1.0.5”.
4. Run “docker push yourdockerregistry/myappname:sometag” to push the image to your Docker container registry. This assumes that you’ve already used “docker login” to authenticate with your registry.

Regarding versioning, at CenterEdge we use NuGet-style version numbering for our microservices. For example, “1.1.0” or “2.0.5-beta002”. This version number is the tag we use in our Docker container registry. We also follow SemVer, meaning that increments to different parts of the number have specific meanings. If we increment the first digit, it means the API has breaking changes and is not fully backwards compatible. Incrementing the second digit indicates significant new features. The third digit is incremented for bug fixes.

Conclusion

Hopefully, you now have the basic tools you’ll need to transition your .NET applications using Couchbase to .NET Core and Docker. We’ve found the transition to be fun and exciting.  While ASP.NET Core has changed some approaches and other things have been deprecated, the overall platform feels much cleaner and easier to use. And I’m sure even more great things are coming in the future.