Implementing a CI/CD Jenkins pipeline for a Dockerised application with different environments

29/04/2019 - DOCKER, GIT, JENKINS

In this example we are going to create a full CI/CD (Continuous Integration/Continuous Delivery) pipeline with Jenkins to work with a Dockerised application. Our Jenkinsfile will depend on Makefile commands. See below the pipeline stages.


Pipeline


PR builder


This runs pipeline when a new feature/* branch gets created in GitHub or a new feature/* push event occurs.


> Build > Test > Destroy

  1. Build: Builds the docker containers.

  2. Test: Runs the application tests.

  3. Destroy: Destroys the docker artefacts.

Develop branch merger


This runs pipeline when any branch gets merged into origin/develop branch in GitHub. This pipeline involves deployment stage but only for Staging environment. If it was origin/master branch then the deployment stage would involve Production environment.


> Build > Test > Push > Destroy > Deploy

  1. Build: Builds the docker images.

  2. Test: Tests the docker containers.

  3. Push: Pushes the docker images to DockerHub.

  4. Destroy: Destroys the docker artefacts.

  5. Deploy: Deploys the application docker images. Note: We won't implement this stage because it is on its own a blog post which I will touch on some day in the future.

Note: When we build the docker image of the application, the application code is being baked into the image so all we want to include in the image is the necessary files and folders. This is to keep the image size as small as possible. All the other files and folders listed in .dockerignore file are ignored.


Application structure


.
├── cicd
│   ├── merge
│   │   └── develop
│   │       └── Jenkinsfile
│   └── push
│       └── feature
│           └── Jenkinsfile
├── docker
│   ├── dev
│   │   ├── docker-compose.yml
│   │   ├── Makefile
│   │   └── php
│   │       └── Dockerfile
│   └── stag
│       ├── docker-compose.yml
│       ├── Makefile
│       └── php
│           └── Dockerfile
├── .dockerignore
├── .gitignore
├── readme.md
└── src
    └── test.php

Files


cicd/merge/develop/Jenkinsfile


pipeline {
    agent any

    options {
        skipDefaultCheckout(true)
    }

    stages {
        stage('Git') {
            steps {
                echo '> Checking out the Git version control ...'
                checkout scm
            }
        }
        stage('Build') {
            steps {
                echo '> Building the docker images ...'
                sh 'make -sC docker/stag build'
            }
        }
        stage('Test') {
            steps {
                echo '> Testing the docker containers ...'
                sh 'make -sC docker/stag test'
            }
        }
        stage('Push') {
            steps {
                echo '> Pushing the docker images ...'
                sh 'make -sC docker/stag push'
            }
        }
        stage('Destroy') {
            steps {
                echo '> Destroying the docker artifacts ...'
                sh 'make -sC docker/stag destroy'
            }
        }
        stage('Deploy') {
            steps {
                echo '> Deploying the application images ...'
            }
        }
    }
}

cicd/push/feature/Jenkinsfile


pipeline {
    agent any

    options {
        skipDefaultCheckout(true)
    }

    stages {
        stage('Git') {
            steps {
                echo '> Checking out the Git version control ...'
                checkout scm
            }
        }
        stage('Build') {
            steps {
                echo '> Building the docker containers ...'
                sh 'make -sC docker/dev build'
            }
        }
        stage('Test') {
            steps {
                echo '> Running the application tests ...'
                sh 'make -sC docker/dev test'
            }
        }
        stage('Destroy') {
            steps {
                echo '> Destroying the docker artifacts ...'
                sh 'make -sC docker/dev destroy'
            }
        }
    }
}

docker/dev/php/Dockerfile


FROM alpine:3.9

RUN apk add --no-cache php7

WORKDIR /app

RUN rm -rf /var/cache/apk/*

CMD tail -f /dev/null

docker/dev/docker-compose.yml


version: "3"

services:
  mini_php:
    build:
      context: "php"
    hostname: "mini-php"
    volumes:
      - "../../:/app:consistent"

docker/dev/Makefile


PHP_SERVICE := mini_php

build:
	@docker-compose up -d --build

test:
	@docker-compose exec -T $(PHP_SERVICE) php src/test.php

destroy:
	@docker-compose down --rmi=all
	@docker system prune --force

all:
	@make -s build test destroy

config:
	@docker-compose config

docker/stag/php/Dockerfile


FROM alpine:3.9

RUN apk add --no-cache php7

COPY . /app

WORKDIR /app

RUN rm -rf /var/cache/apk/*

CMD tail -f /dev/null

docker/stag/docker-compose.yml


version: "3"

services:
  mini_php:
    build:
      context: "../.."
      dockerfile: "docker/stag/php/Dockerfile"
    image: "inanzzz/mini_php:latest"

docker/stag/Makefile


PHP_SERVICE := mini_php

build:
	@docker-compose build --no-cache

test:
	@docker-compose run -T --rm $(PHP_SERVICE) php src/test.php

push:
	@docker-compose push

destroy:
	@docker-compose down --rmi=all
	@docker system prune --force

all:
	@make -s build test push destroy

config:
	@docker-compose config

src/test.php


<?php
echo 'success';

.dockerignore


.git/
.idea/
.DS_Store/
.gitignore
.dockerignore
readme.md
composer.json
composer.lock
docker/

.gitignore


.idea/
.DS_Store/

Setup


Any command below is run on Jenkins server.


Docker installation


vagrant$ sudo apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common
vagrant$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
vagrant$ sudo apt-key fingerprint 0EBFCD88
vagrant$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
vagrant$ sudo apt-get update
vagrant$ sudo apt-get install docker-ce docker-ce-cli containerd.io

Docker compose installation


vagrant$ sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
vagrant$ sudo chmod +x /usr/local/bin/docker-compose

Jenkins user permissions


# Allow "jenkins" user to run docker commands.
vagrant$ sudo usermod -aG docker jenkins
vagrant$ sudo su -l jenkins
jenkins$ docker ps # This command should work now

GitHub SSH connection


jenkins$ ssh-keygen -t rsa -b 4096 -C "your@email.com"
jenkins$ eval "$(ssh-agent -s)"
jenkins$ ssh-add ~/.ssh/id_rsa
jenkins$ cat ~/.ssh/id_rsa.pub # Add this to GitHub
jenkins$ ssh -T git@github.com # Run this after step above

Jenkins GitHub credentials


Apply steps defined under "Jenkins > Credentials" section here.


Prepare GitHub repository for PR builder


Follow the steps defined here - starting from "Prepare Github repository" section.


Prepare GitHub repository for develop branch merger


Follow the steps defined here - starting from "Prepare Github repository" section.


Docker login


jenkins$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: inanzzz
Password: 
WARNING! Your password will be stored unencrypted in /var/lib/jenkins/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

Tests


First of all manually run pipeline for each items in Jenkins so that you know it works.