SpringBoot CI/CD with Azure Container Registry and Jenkins (Part 1)

Create Git Clone, ACR Push/Pull, Docker Run Pipeline Stages

Jenkins is the mostly and widely used pipelining tool for continuous deployments. Here I am planing to explain you that how to push an SpringBoot application build image to Azure Container registry and reused it inside your docker-compose file while you run your deployment. There are some basic steps to complete before the Jenkins pipeline part.

  1. From your Azure Console, create container registry and get your login password from Settings -> Access keys (We are going to create Jenkins credential id using this)
  2. You have a SpringBoot app with Dockerfile, docker-compose file and Jenkins file located in same directory structure
SpringBoot Application Directory Structure
Azure Container Registry

Here is the sample content of above files

Dockerfile
docker-compose file
Jenkinsfile

In your Jenkins, if you are using Jenkins docker image you have to have below configurations

First execute below in your Jenkins host machine

sudo adduser jenkins
sudo usermod -a -G docker jenkins

Here I map docker and docker-compose executable files from host machine to Jenkins docker container, so Jenkins can execute docker commands.

Jenkins docker-compose file

Jenkins – Create pipeline using existing Jenkinsfile

Log into your Jenkins and Select “New Item” -> Pipeline , put a suitable name for your build

You can add triggers later, for now , go to “Pipeline Section” and select “pipeline script from SCM”

Put your Git repo, if your repo is a private one add credentials by clicking Add button

Mention your branch, if you are using different branch than master, put that name

Script path is “Jenkinsfile”, its relative url to your project

Adding Azure Container registry credentials to Jenkins

Go to Credentials -> System -> Global credentials (unrestricted) -> Add Credentials

Username/password Entry

In your Pipeline script , put the same credential ID as above

Now you can execute “Build Now” command

Build Now

Summery of the flow (pipeline)

  1. Compile your java code using maven
  2. Package it to jar
  3. Optimize and jar (break fat jar)
  4. Build docker image
  5. Push the image to ACR with tag
  6. Pull back the image
  7. Remove all previous docker containers
  8. Spin up all the containers (including dependencies) using docker-compose

You can check the containers after the deployment

Running Containers

Source-code

https://github.com/dumindarw/reactive-eventservice

To be continued ….

Command Query Responsibility Segregation Pattern Implementation – Spring Boot

CQRS is an Architectural pattern which solves complexity issues in larger projects. CQRS simplifies your service layer by dividing into main two blocks. Most people confusing about this pattern but it is simple.

  1. Commands
  2. Queries
  1. What is a Command – Command is basically a CRUD without R , that means CUD, simple right? CUD – CREATE, UPDATE, DELETE. Commands are database transaction operations, for example you are going to change the state of your database by editing, inserting or deleting a record or set of records.
  2. What is a Query – Query is R in CRUD, means RETRIEVE. Queries are fetching database records and they are not deal with transaction stuff.

Is this pattern something you should try?

Yes but “Use Responsibly”, This may increase the complexity of small projects but ideal if you need clean separation between commands and queries.

What technologies we should use for this

In Spring Ecosystem, ideally an ORM aka any JPA technology works for command implementation since they optimized to persist any complex objects. but I recommend Spring Data JPA Crud repository over plain Hibernate since Spring Data CRUD is well optimized and can do almost all the CRUD operations. If you have really complex DB architecture use Hibernate with Spring Data.

For Queries we can use MyBatis. Hibernate would attempt to load the entire object graph and you’d need to start tuning queries with LazyLoading tricks to keep it working on a large domain. This is important when running complex analytic queries that don’t even return entity objects. Hibernate only offers SqlQuery and bean Transformers in this case with huge default types like BigDecimal, while myBatis could easily map to a simple POJO non-entity (https://softwareengineering.stackexchange.com/questions/158109/what-are-the-advantages-of-mybatis-over-hibernate). If you want to just fetch information, use myBatis.

Technically, that’s how you gonna separate the two components, you can find simple implementation from below repository.

https://github.com/dumindarw/CQRS

In above example there are two end-points for Query and Command

Command – http://localhost:8081/doc/upload/Salary [POST]

Query – http://localhost:8081/doc/download/Salary/1 [GET]

First end-point is multipart file upload end-point and it will persist the CSV content to the table and it is a CREATE operation. I have used JPA for this operation.

Second end-point is for retrieve data from the DB by documentID. I have used MyBatis for this operation.

Spring Boot Microservices with Keycloak

Micro-service architecture is the new revolution of the software industry and it has been popular topic since later 2015 and now it became the most suited architecture for bigger and complex systems. It is a nice and maintainable replacement to the traditional monolithic (aka all in one) system. As the name says it all, you have to develop and deploy services as small business components, not only in your application layer but also your db layer need to be componentized. Functionality and behaviour should unique to the individual service and dependencies need to be injected via service calls, not using tightly bound db relationships.

Here I am going to explain the basic components of the micro-service system and how to develop a basic functioning micro-service with proper security. In my sample application I have separated my system to few layers

  1. Simple API (Micro-service; you can have multiple MSs)
  2. Service Registry
  3. API Gateway
  4. Identity and Access Management

Later you can introduce more layers to the system, such as config service (Spring Cloud Config), log service (Sleuth), latency and fault tolerance (Hystrix), client side load balancing (Ribbon) etc,  but as a first step I am going to prepare the ground layer for the basic start up system.

You can refer below GIT repository for code setup.

https://github.com/dumindarw/springboot-microservices-starter.git

Step 1 : Configure your KeyCloak Server

Keycloak is an open source Identity and Access Management solution developed by JBoss (on top of WildFly).Download and extract zip/tar file from here “https://www.keycloak.org/” and start up the server using

C:\keycloak-4.5.0.Final\bin
λ .\standalone.bat -Djboss.socket.binding.port-offset=100

Server will startup and on port 8180 . Visit http://localhost:8180/auth/ and create new admin login first.

Visit http://localhost:8180/auth/admin/master/console/#/create/realm and import the JSON file found in above GIT repo (zuul-server/config). It will create realm, clients, roles and users for you (Demo purpose). You can create them manually as you wish.

Step 2 : Start your Eureka Server

Netflix Eureka is a service registry and service discovery component. It will ease our work by registering each micro-service to centralized location and it provides easier way for inter process communication.

Build and run using maven, execute below command withing the project location.

C:\Users\Duminda\spring-workspace\microservices-project\eureka-server
λ mvn install && mvn spring-boot:run

This will start your Eureka server on port 8761

Step 3 : Start your Micro-service

C:\Users\Duminda\spring-workspace\microservices-project\api-service
λ mvn install && mvn spring-boot:run

This will start your API service on port 8080

Step 4 : Start your Zuul API Gateway

Netflix Zuul is a API gateway. It will provide easy routing towards your microservices.

C:\Users\Duminda\spring-workspace\microservices-project\zuul-server
λ mvn install && mvn spring-boot:run

This will start your ZUUL server on port 8762

Step 5 : Get your access token first

We have created a user duminda inside keycloak (Admin console) with password 12345(or you can create your own). We are now able to get the token using below POST request. You can use Postman, SoapUI or CUrl for that.

curl -k -u app-authz-rest-springboot:secret -d "grant_type=password&username=duminda&password=12345" -H "Content-Type: application/x-www-form-urlencoded" http://localhost:8180/auth/realms/spring-boot-quickstart/protocol/openid-connect/token

Above request will return JWT token with some additional data like token expiry time, refresh token etc.

{
  "access_token": "eyJhbGci......",
  "expires_in": 1800,
  "refresh_expires_in": 1800,
  "refresh_token": "eyJhbG .....",
  "token_type": "bearer",
  "not-before-policy": 0,
  "session_state": "1947e319-0793-4cb4-99a6-624d4961a209",
  "scope": "profile email"
}

6. Use above JSON Web Token to call your web service through API Gateway

curl -H "Authorization: Bearer access_token" http://localhost:8762/api/v1/resourceb

Replace access_token with the JWT then you can access  the API resource. Without Authorization token, ZUUL proxy will reject your request and it will not reach to the destination (Microservice1 or Microservice2) and return unauthorized response.

This architecture diagram shows how the system behaves top to bottom.

Architecture Diagram

Note that this is a basic architecture. You can add more components to the architecture and add refresh token capability to the system as well.