Jenkins Dynamic Cluster

What is Jenkins ?

Jenkins is a free and open source automation server. It helps automate the parts of software development related to building, testing, and deploying, facilitating continuous integration and continuous delivery. It is a server-based system that runs in servlet containers such as Apache Tomcat.

Jenkins is an open-source automation tool written in Java with plugins built for Continuous Integration purposes. Jenkins is used to build and test your software projects continuously making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. It also allows you to continuously deliver your software by integrating with a large number of testing and deployment technologies.

With Jenkins, organizations can accelerate the software development process through automation. Jenkins integrates development life-cycle processes of all kinds, including build, document, test, package, stage, deploy, static analysis, and much more.

Jenkins achieves Continuous Integration with the help of plugins. Plugins allows the integration of Various DevOps stages. If you want to integrate a particular tool, you need to install the plugins for that tool. For example: Git, Maven 2 project, Amazon EC2, HTML publisher etc.

What is Kubernetes ?

Kubernetes is an open-source container-orchestration system for automating computer application deployment, scaling, and management. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation.

Kubernetes is open source software that allows you to deploy and manage containerized applications at scale. Kubernetes manages clusters of Amazon EC2 compute instances and runs containers on those instances with processes for deployment, maintenance, and scaling. Using Kubernetes, you can run any type of containerized applications using the same toolset on-premises and in the cloud.

What is Pipeline ?

The pipeline is a set of instructions given in the form of code for continuous delivery and consists of instructions needed for the entire build process. With pipeline, you can build, test, and deliver the application. Node. The machine on which Jenkins runs is called a node.

Jenkins Pipeline (or simply "Pipeline" with a capital "P") is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.

A continuous delivery (CD) pipeline is an automated expression of your process for getting software from version control right through to your users and customers. Every change to your software (committed in source control) goes through a complex process on its way to being released. This process involves building the software in a reliable and repeatable manner, as well as progressing the built software (called a "build") through multiple stages of testing and deployment.

Problem Statement :

1. Create a container image that has Linux and other basic configuration required to run Slave for Jenkins. ( eg. here we require kubectl to be configured )

2. When we launch the job it should automatically starts job on slave based on the label provided for dynamic approach.

3. Create a job chain of job1 & job2 using build pipeline plugin in Jenkins 

4. Job1 : Pull the Github repo automatically when some developers push repo to Github and perform the following operations as:

4.1 Create the new image dynamically for the application and copy the application code into that corresponding docker image

4.2 Push that image to the docker hub (Public repository) 

 ( Github code contain the application code and Dockerfile to create a new image )

5. Job2 : ( Should be run on the dynamic slave of Jenkins configured with Kubernetes kubectl command):

Launch the application on the top of Kubernetes cluster performing following operations:

5.1 If launching first time then create a deployment of the pod using the image created in the previous job. Else if deployment already exists then do rollout of the existing pod making zero downtime for the user.

5.2  If Application created first time, then Expose the application. Else don’t expose it.

Solution :

First we have to create the custom image in which Linux and other basic configuration required to run Slave for Jenkins ( example here we require kubectl to be configured ) as well as ssh already configured.

Dockerfile for the custom image :

Build custom docker image and push this image to the Docker Hub.

$ docker build -t dynamicslave:latest 
$ docker tag dynamicslave:latest nikhil/dynamicslave:latest
$ docker push nikhil/dynamicslave:latest

Now we setup the environment for dynamic jenkins cluster.Here we use two vm one is used for docker server and another is used for docker client.We have to change something in the service of docker in vm2.

$ systemctl status docker

$ vim /usr/lib/systemd/system/docker.service

After adding this we have to restart the docker services.

$ systemctl daemon-reload
$ systemctl restart docker

Now we have to stop the docker services of another vm1 using systemctl stop docker. After this you need to export the DOCKER_HOST, so that you can use this VM(vm1) as a docker client.

$ export DOCKER_HOST=IP of VM2:port

Here we give the port number same as the port number which we change in docker service of vm2.

Now we have to configure the cloud environment

First you have to install the Docker plugin from manage plugins. Now go to

Manage jenkins–> Manage Node and Clouds –> configure cloud.

JOB1: Pull the Github repo automatically when some developers push repo to Github and perform the following operations as:

1. Create the new image dynamically for the application and copy the application code into that corresponding docker image

2. Push that image to the docker hub (Public repository).

Dockerfile for the web code:

FROM centos:latest

RUN yum install httpd -y

COPY web.html /var/www/html

CMD /usr/sbin/httpd -DFOREGROUND

EXPOSE 80

JOB2: ( Should be run on the dynamic slave of Jenkins configured with Kubernetes kubectl command):

Launch the application on the top of Kubernetes cluster performing following operations:

1. If launching first time then create a deployment of the pod using the image created in the previous job. Else if deployment already exists then do rollout of the existing pod making zero downtime for the user.

2. If Application created first time, then Expose the application. Else don’t expose it.