Jenkins Dynamic Provisioning

Objectives

In this article , We will see how we can create dynamic slave on the fly when job come and attach to the jenkins master . So that all the job run on the slave . These slave are docker container . So it will be super fast...

Why we need dynamic slave ?

• Initially we don't know how many slave nodes are required . So if we use static provisioning , then it will increase the cost . Because static nodes are continuously consuming RAM/CPU and other resources.
• So we need dynamic provisioning . By this, we get slave node on the fly when demand comes and automatically terminated when it's work completed.

Now let's understand about jenkins

What is Jenkins ?

• Jenkins is a free and open-source automation server. It helps automate the parts of software development related to building, testing, and deploying, facilitating continuous integration and continuous delivery. It is a server-based system that runs in servlet containers such as Apache Tomcat.
• Jenkins is written in Java with plugins built for Continuous Integration purposes. Jenkins is used to building and testing your software projects continuously making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build.

Let' Started

We need two system :-

1. Jenkins Master

2.Linux (Rhel -8) System

In Rhel-8 system , we installed docker . Now when any job comes in Jenkins , then it contact to docker and launch the container (or slave). And this container terminated automatically after finished the job.

For better understanding , I divide this task in steps.

Step-1 Install Plugins in Jenkins

As we know , jenkins by default don't know about Docker. To gives this ability , we need to install docker plugin in jenkins master system

Note-

1. Make sure to disable firewalld in both the systems

2. Enable agent port in Jenkins master

Go to Manage jenkins -> Configure Global security -> Enable Agent port

Step-2 Enable TCP socket in Docker system

• By default , docker engine don't have ability to control externally. But we want jenkins connect to docker . So we have to enable TCP socket in docker system(Rhel-8).
• For this , open docker service file /usr/lib/systemd/system/docker.service and add -H tcp://0.0.0.0:2376 in the file .

This is socket binding. Now restart systemd daemon and docker service.

systemctl daemon-reload

systemctl restart docker        

Now docker can be manage locally and externally. Here I use 0.0.0.0:2376 . So if IP of jenkins master change , then we don't need to edit this file again and again.

Step-3 Add docker information in Jenkins Master

• Now configure the slave in jenkins master . For this Manage Jenkins -> Manage Nodes and Cloud -> configure clouds -> Add a new cloud -> Docker
• We get two section :- a. docker cloud details b. Docker agent templates

Click a. docker cloud details -> In this we add docker system ip

# Add this 
Docker Host URI = tcp://192.168.43.31:2376
        

b. Docker agent template

# Add this
Label = mydockerarth

Name = myslave

Image = vimal13/jenkins-slave-maven

Usage = Only build job with label expression matching this node  
        

We need an image that contains java and jenkins agent . So that jenkins master can contact to the container(slave). So i use an image vimal13/jenkins-slave-maven . This is also contain maven pre-installed . So we can able to run any maven related job.

Step-4 Run Jenkins Job

In this video , I show you the demo how slave node (container) is launched on the fly when demand comes and automatically terminate the slave(container) when job is finished.