Tayeb Chlyah is a Java lead architect with solid experience on performance large-scale applications, microservices, and NoSQL databases. He has developed a couple of open source Java libraries for Couchbase.
Introduction
Every successful developer needs to keep up to date with cutting-edge new technologies and emerging trends, trying them, playing with them, seeing if they can fit into a project, and how they can improve upon them.
But how can one achieve that in a fast-changing world where new technologies and frameworks are born daily?
If you are a Java full-stack developer, you are very lucky: JHipster is the solution to quickly apprehend new technologies in best way. It is a Yeoman generator, a scaffolding system, that helps you generate complete applications combining one of the best frameworks in web development: Spring Boot backend and Angular or React frontend.
Of course, you can use Spring Initialzr, or implement everything by yourself, but to successfully implement those combinations is not trivial at all. You need to spend a lot of time studying each concept, and then try to glue everything together, without forgetting tests, which can be very expensive and error-prone. And most of all, it can discourage you or, at best, oblige you to focus on one of the frameworks.
JHipster builds fully tested applications quickly and easily, using best practices, methodologies, and strategies. It proposes a lot of options (Architecture – monolith or microservices; Database – SQL or NoSQL; Security – session, JWT, or OAuth2 authentication; Other – WebSocket, caches, Docker, Kubernetes …), which can help you try the best technologies out there, and choose the most suitable combinations for your use case.
Today we are going to focus on some of the most important new architectural concepts: microservices and NoSQL with Couchbase.
Why Couchbase?
Couchbase is a NoSQL document-oriented database. With document databases, you have a schemaless design which allows you to change your data freely and easily. You can also store the whole structure in a single document, avoiding lots of unnecessary joins, which means naturally faster read and write operations.
Couchbase exposes a fast key-value store and a powerful query engine along with built-in indexers to query data with N1QL, a SQL-like language for JSON documents. With its masterless distributed architecture, it is very easy to scale. It can deliver millions of operations per second without the need of a third-party cache to perform. Couchbase also comes with built-in full-text search, an analytics engine, and a complete mobile solution.
Why microservices?
Microservice architecture is a pattern of developing software systems that focus on building applications as a set of small, modular, and loosely coupled services, enabling easier continuous delivery, better testability, scalability, and improved fault isolation. Each service can be written in different languages and may use different data storage techniques, which enables the ability to organize development around multiple feature teams.
But microservices comes with some challenges: decomposing the application into services can be very complicated, and is very much an art. Developers also have to deal with the additional complexity of a distributed system.
JHipster handles most of the complexities of microservices: Service discovery and configuration with Consul or JHipster Registry (Netflix Eureka, Spring Cloud Config Server, and monitoring dashboard), load balancing with Netflix Ribbon, fault tolerance with Netflix Hystrix, and centralized logging and monitoring with jhipster-console (Elasticsearch, Logstash and Kibana personalized stack), and more.
Creating a brewery microservice
Prerequisites:
- Install Docker Engine
- Install JDK 8
- Install LTS 64-bit version on Node.js
- Install Yarn
- Install JHipster:
yarn global add generator-jhipster
Generate an API Gateway
In order to access our different services in a microservice architecture, we’re going to need an API Gateway. It is the entrance to your microservices. It provides HTTP routing (Netflix Zuul) and load balancing (Netflix Ribbon), quality of services (Netflix Hystrix), security (Spring Security), and API documentation (Swagger) for all microsevices.
In a terminal window:
|
1 2 3 4 5 |
mkdir couchbase-jhipster-microservices-example cd couchbase-jhipster-microservices-example mkdir gateway cd gateway jhipster |
JHipster asks about the type of application you want to create and what features you want to include. You can find all the details about available options on JHipster Website. Use the following answers in order to generate a gateway with Couchbase support.
Generate a brewery microservice application
In couchbase-jhipster-microservices-example, create a brewery directory, then run jhipster to generate a microservice with Couchbase database with the following answers:
Generate brewery entities
Create a file brewery.jh in couchbase-jhipster-microservices-example directory with the following JDL (JHipster Domain Language):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
entity Beer { name String required, category String required, description String, style String, brewery String, abv Float, ibu Integer, srm Integer, upc Integer, updated LocalDate } entity Brewery { name String required, description String, address String required maxlength(200), city String, code String, country String, phone String pattern(/[0-9- .]+/), state String, website String, updated LocalDate } |
paginate Beer with pager
paginate Brewery with infinite-scroll
In a terminal window, run the following commands:
|
1 2 3 4 5 |
cd brewery jhipster import-jdl ../brewery.jh cd ../gateway jhipster entity brewery jhipster entity beer |
When asked to overwrite files, always answer a.
Run the microservice architecture
In order to run our architecture, we need to start the following:
- JHipster Registry
- Keycloack: an open source identity and access management solution
- Couchbase Server
- Brewery microservice
- Gateway
Fortunately, JHipster has a docker-compose sub-generator that will start all necessary services without headache. But first, we need to build our applications. In couchbase-jhipster-microservices-example:
|
1 2 3 4 5 6 |
cd gateway ./mvnw package -Pprod dockerfile:build -DskipTests cd ../brewery ./mvnw package -Pprod dockerfile:build -DskipTests cd .. mkdir docker-compose && cd docker-compose |
Now, we can generate docker-compose.yml using the following command and answers:
|
1 |
jhipster docker-compose |
In order to make our app work with a local keycloak, we need to add to your hosts file (Windows: C:\Windows\System32\drivers\etc\hosts, Mac/Linux: /etc/hosts) the following line:
|
1 |
127.0.0.1 keycloak |
Before starting everything, make sure you have configured Docker with enough memory and CPUs, then run:
|
1 |
docker-compose up |
Once everything finishes starting, open a browser to Gateway (http://localhost:8080/), click on account, then sign in.
You should be redirected to keycloak, log in using admin for both user and password.
You’ll return to the gateway, where you can see administration interfaces, change language, view and edit your entities…
JHipster-registry
Now open your browser to http://localhost:8761/. You’ll be automatically signed in since you have already done that in keycloak, which is the magic of oauth2 authentication.
Apart from being the backbone of your microservice application, as it is a discovery server with Eureka that handles routing, load balancing and scalability, and a configuration server with Spring Cloud Config Server that provides runtime configuration of your applications, JHipster registry is also an administration server, where you can visualize your application instances, health, metrics, and logs.
How does it work ?
Accessing the database
First, you need access to brewery-couchbase instance. In order to do so, update docker-compose/docker-compose.yml to publish its administration interface port with the following:
|
1 2 3 4 5 6 7 8 |
brewery-couchbase: build: context: ../brewery/src/main/docker dockerfile: couchbase/Couchbase.Dockerfile environment: - BUCKET=brewery ports: - 8091:8091 |
Let’s apply our changes, on docker-compose directory:
|
1 |
docker-compose up -d |
When everything is up, open Couchbase administration interface on http://localhost:8091/, then log in using Administrator as user and password as password. Open brewery bucket Documents.
Couchmove
As you can see, your bucket is already populated with some documents: JHipster populates bootstrap documents using Couchmove, a Java data migration library for Couchbase, widely inspired from Flyway, as it strongly favors simplicity and convention over configuration.
You can take a look on changelog files in
|
1 |
brewery/src/main/resources/config/couchmove/changelog directory: |
- V0.1__initial_setup directory: contains user and authority documents, used by Spring Security to authenticate users.
- V0__create_indexes.n1ql: creates needed indexes.
As Flyway or liquibase for SQL databases, Couchmove maintains changelog documents, keeping track of executed changelogs.
Spring Boot and Spring Data Couchbase
JHipster generates a Spring Boot 2 application, and uses Spring Data to access relational and non-relational databases. In our case, it is using Spring Data Couchbase to provide integration with the Couchbase Server database. But it goes much further by personalizing how it works:
By default, Spring Data Couchbase uses N1qlCouchbaseRepository.java which leverages N1QL only for paging or sorting findAll operations. All other operations are using views as you can see in the implementation of SimpleCouchbaseRepository.java. Since view indexes are always accessed from disk, they are not quite as performant.
Let’s see how JHipster improves this behavior:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
@Configuration @Profile("!" + JHipsterConstants.SPRING_PROFILE_CLOUD) @EnableCouchbaseRepositories(repositoryBaseClass = CustomN1qlCouchbaseRepository.class, basePackages = "com.couchbase.example.brewery.repository") @Import(value = CouchbaseAutoConfiguration.class) @EnableCouchbaseAuditing(auditorAwareRef = "springSecurityAuditorAware") public class DatabaseConfiguration { ... @Bean public Couchmove couchmove(Bucket couchbaseBucket) { log.debug("Configuring Couchmove"); Couchmove couchMove = new Couchmove(couchbaseBucket, "config/couchmove/changelog"); couchMove.migrate(); return couchMove; } ... } |
DatabaseConfiguration class configures and starts Couchmove migrations, and also enables Couchbase Repositories but with a custom base class: CustomN1qlCouchbaseRepository
|
1 2 3 4 5 6 7 8 |
public class CustomN1qlCouchbaseRepository<T, ID extends Serializable> extends N1qlCouchbaseRepository<T, ID> { ... @Override public <S extends T> S save(S entity) { return super.save(populateIdIfNecessary(entity)); } ... } |
This class extends the default repository to auto-generate document ID before saving, and enables usage of N1QL for all operations by implementing CustomN1qlRespository interface:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
@NoRepositoryBean public interface N1qlCouchbaseRepository<T, ID extends Serializable> extends CouchbasePagingAndSortingRepository<T, ID> { @Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter}") List<T> findAll(); @Query("SELECT count(*) FROM #{#n1ql.bucket} WHERE #{#n1ql.filter}") long count(); @Query("DELETE FROM #{#n1ql.bucket} WHERE #{#n1ql.filter} returning #{#n1ql.fields}") T removeAll(); default void deleteAll() { removeAll(); } } |
Ids are auto generated using GeneratedValue annotation with a prefix using IdPrefix annotation, which by default is the name of the class, separated by __ delimiter.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
@Document public class Brewery implements Serializable { private static final long serialVersionUID = 1L; public static final String PREFIX = "brewery"; @SuppressWarnings("unused") @IdPrefix private String prefix = PREFIX; @Id @GeneratedValue(strategy = UNIQUE, delimiter = ID_DELIMITER) private String id; ... } |
Tests
We said earlier that JHipster-generated applications are fully tested with unit and integration tests, but how does it work for integration ones?
For Couchbase, it uses Couchbase testContainers, which is a module extending testcontainers, a Java library that makes it easy to launch any Docker containers for the duration of JUnit tests, in order to automatically start an instance of Couchbase Server, and configure it with the necessary services, users, buckets, and indexes.
In order to run the tests, run the following command in a terminal window:
|
1 |
./mvnw clean test |
Populate some data
In order to populate some data, we will take advantage of Couchbase Sample buckets. Open Settings tab, Sample Buckets, then select beer-sample, and click on Load Sample Data.
You need to shape data in Spring Data Couchbase serialization format. To do so, open Query tab, and execute the following queries:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
INSERT INTO brewery (KEY k, VALUE v) SELECT type || "__" || meta().id as k, { "name": name, "category": category, "description": description, "style": style, "brewery": brewery_id, "abv": abv, "ibu": ibu, "srm": srm, "upc": upc, "updated": STR_TO_MILLIS(updated), "_class" : "com.couchbase.example.brewery.domain.Beer" } as v FROM `beer-sample` WHERE type = 'beer'; INSERT INTO brewery (KEY k, VALUE v) SELECT type || "__" || meta().id as k, { "name": name, "description": description, "address": address[0], "country": country, "website": website, "code": code, "city": city, "phone": phone, "state": state, "updated": STR_TO_MILLIS(updated), "_class" : "com.couchbase.example.brewery.domain.Brewery" } as v FROM `beer-sample` WHERE type = 'brewery'; |
Here is an example of an inserted beer document:
Now let’s see how JHipster handles newly inserted documents. Navigate to Gateway, Entities, then open Brewery. JHipster first loads 20 breweries, and if you scroll down, it loads more! It proposes page navigation for Beer entity, because we choose this behavior when generating entities with brewery.jh JDL.
|
1 2 |
paginate Beer with pager paginate Brewery with infinite-scroll |
Source code
The source code for generated applications is available at tchlyah/couchbase-jhipster-microservices-example.
What’s next?
JHipster, with the help of Elasticsearch, proposes an option that adds search capabilities on top of the database. But Couchbase has an out-of-the-box Full-Text Search feature that provides extensive capabilities for natural-language querying. Why not implement that support into JHipster? If you can help, don’t hesitate to contribute!
If you have any questions, tweet me at @tchlyah or leave a comment below.
English