Hello everyone!

We have been investing plenty of personal time and energy for many years to share our knowledge with you all. However, we now need your help to keep this blog running. All you have to do is just click one of the adverts on the site, otherwise it will sadly be taken down due to hosting etc. costs. Thank you.

In this example we are going to use GitHub actions to deploy our Golang application to a Kubernetes cluster. We have qa, stage and prod Kubernetes environments. We have three different workflows to work with these environments. I am going to use a local Minikube cluster so there will be a "setup" section below to expose it to public with Ngrok.


Structure


Although we are going to focus on the Kubernetes here, you can run go run -ldflags "-s -w -X main.commit=`git rev-parse --short HEAD`" -race main.go command to run the application in your local environment and access it with http://0.0.0.0:3000.


├── .env
├── .gitignore
├── .dockerignore
├── .github
│   └── workflows
│   ├── continuous_deployment.yaml
│   ├── manual_deployment.yaml
│   └── pull_request_builder.yaml
├── deploy
│   └── k8s
│   ├── prod.yaml
│   ├── qa.yaml
│   └── stage.yaml
├── docker
│   └── ci
│   └── Dockerfile
├── go.mod
├── go.sum
├── main.go
└── main_test.go

Files


.env


# All these values are used for the local environment, not K8S!
ENV=dev
ADR=:3000
KEY=dev-secret

.dockerignore


bin/

.dockerignore
.gitignore
*.md
.env

.git/
.idea/
.DS_Store/

Dockerfile


FROM golang:1.15-alpine3.12 as build

WORKDIR /source
COPY . .

ARG COMMIT
RUN CGO_ENABLED=0 go build -ldflags "-s -w -X main.commit=${COMMIT}" -o bin/pipeline main.go

FROM alpine:3.12

COPY --from=build /source/bin/pipeline /bin/pipeline

EXPOSE 8080

ENTRYPOINT ["./bin/pipeline"]

main.go


package main

import (
"fmt"
"log"
"net/http"
"os"

"github.com/joho/godotenv"
)

var commit string

func main() {
// Load environment variables
if err := godotenv.Load(".env"); err != nil {
log.Fatal(err)
}
env, _ := os.LookupEnv("ENV")
adr, _ := os.LookupEnv("ADR")
key, _ := os.LookupEnv("KEY")

// Bootstrap HTTP handler
hom := home{commit, env, adr, key}

// Bootstrap HTTP router.
rtr := http.DefaultServeMux
rtr.HandleFunc("/", hom.handler)

// Start HTTP server.
log.Printf("running pipeline: %+v\n", hom)
log.Fatalln(http.ListenAndServe(adr, rtr))
}

type home struct {
com string
env string
adr string
key string
}

func (h home) handler(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(fmt.Sprintf("%+v", h)))
}

main_test.go


package main

import (
"net/http"
"net/http/httptest"
"testing"
)

func Test_home_handler(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/", nil)
res := httptest.NewRecorder()

home{}.handler(res, req)

if res.Code != http.StatusOK {
t.Error("expected 200 but got", res.Code)
}
}

GitHub workflows


I have kept the jobs as minimal as possible otherwise this post would be very long. As a result certain steps are ignored such as linter so on.


pull_request_builder.yaml


# Trigger the workflow only when:
# - a new pull request with any name/type is opened against the master, develop, hotfix/* or release/* branch
# - a commit is directly pushed to the pull request

name: Pull request builder

on:
pull_request:
branches:
- master
- develop
- hotfix/*
- release/*

jobs:

setup:
runs-on: ubuntu-latest
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum

test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...

continuous_deployment.yaml


# Trigger the workflow to deploy to qa environment only when:
# - an existing pull request with any name/type is merged to the master or develop branch
# - a commit is directly pushed to the master or develop branch

name: Continuous deployment

on:
push:
branches:
- master
- develop

jobs:

setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum

test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...

docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:qa
build-args: COMMIT=${{ needs.setup.outputs.commit }}

deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/qa.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-qa \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"

manual_deployment.yaml


# Trigger the workflow to deploy a specific git reference to a specific environment only when:
# - the `workflow_dispatch` event is used in the UI.
# This is ideal for environment such as production, staging or sandbox if you wish to make the
# deployment manual.

name: Manual deployment

on:
workflow_dispatch:
inputs:
env:
description: "Environment to deploy - options: qa|stage|prod"
required: true
ref:
description: "Git reference to deploy - example: branch/tag/sha"
required: true

jobs:

setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Deployment info
run: echo "Deploying '${{ github.event.inputs.ref }}' to '${{ github.event.inputs.env }}' environment"
- name: Verifying environment
run: |
envs=("qa stage prod")
[[ ${envs[*]} =~ ${{ github.event.inputs.env }} ]] || { echo "Invalid environment"; exit 1; }
- name: Use repository
uses: actions/checkout@v2
with:
ref: ${{ github.event.inputs.ref }}
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum

test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...

docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:${{ github.event.inputs.env }}
build-args: COMMIT=${{ needs.setup.outputs.commit }}

deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
case ${{ github.event.inputs.env }} in
qa) echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config ;;
stage) echo "${{ secrets.KUBE_STAGE_CONFIG }}" > $HOME/.kube/config ;;
prod) echo "${{ secrets.KUBE_PROD_CONFIG }}" > $HOME/.kube/config ;;
*) echo "Invalid environment"; exit 1;;
esac
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/${{ github.event.inputs.env }}.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-${{ github.event.inputs.env }} \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"

Kubernetes configuration


As shown below, there are three environments and each of which is isolated within their dedicated namespaces. The rollout and imagePullPolicy properties make sure that the new image is pulled at every deployment.


qa.yaml


apiVersion: v1
kind: Namespace
metadata:
name: pipeline-qa

---

apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-qa
data:
ENV: qa
ADR: :8080

---

apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-qa
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-qa
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:qa
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env

stage.yaml


apiVersion: v1
kind: Namespace
metadata:
name: pipeline-stage

---

apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-stage
data:
ENV: stage
ADR: :8080

---

apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-stage
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-stage
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:stage
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env

prod.yaml


apiVersion: v1
kind: Namespace
metadata:
name: pipeline-prod

---

apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-prod
data:
ENV: prod
ADR: :8080

---

apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-prod
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-prod
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 2
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:prod
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env

Logic


Pull Request


- First time PR creation runs pull_request_builder.yaml
- Pushing a commit to an existing PR runs pull_request_builder.yaml
- Merging an existing PR to develop/master runs continuous_deployment.yaml
- Direct commit push to develop/master runs continuous_deployment.yaml

Release/Hotfix


- First time release/hotfix creation against develop/master runs pull_request_builder.yaml
- Pushing a commit to an existing release/hotfix runs pull_request_builder.yaml
- Creating a new PR against the release/hotfix runs pull_request_builder.yaml
- Pushing a commit to an existing PR that tracks a release/hotfix runs pull_request_builder.yaml
- Merging an existing PR to a release/hotfix runs pull_request_builder.yaml
- Direct commit push to release/hotfix runs pull_request_builder.yaml
- Merging an existing release/hotfix to develop/master runs continuous_deployment.yaml
- Merging an existing release/hotfix to develop/master in CLI and pushing develop/master runs continuous_deployment.yaml

Tag


- Manually creating a tag does nt trigger any pipeline
- Pushing a tag from local does not trigger any pipeline

Manual Deployment


- Manually deploying a specific git ref (branch/tag/sha) to a specific environment

Others


- Closed PRs will not trigger any pipeline
- Rollback is same as manual deployment so deploy a previous tag

Setup


Add Docker secrets and Kube config to GitHub


This will let your workflow to push images to DockerHub registry.


DOCKERHUB_USERNAME
DOCKERHUB_TOKEN

This will let your workflows to deploy the application to your local K8S cluster.


// Enable routing public requests to K8S API.

$ kubectl proxy --disable-filter=true
Starting to serve on 127.0.0.1:8001
(Optionally $ kubectl proxy --port=8001 --accept-hosts='.*\.ngrok.io')

// Expose kubectl proxy to the Internet with ngrok.

$ ./ngrok http 127.0.0.1:8001
Forwarding http://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001
Forwarding https://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001

// Create a modified copy of your local kube config.

$ kubectl config view --flatten > ~/Desktop/kube_config
Remove `certificate-authority-data` line
Add `insecure-skip-tls-verify: true` line
Replace `server` value to `https://c309b9b1359f.ngrok.io`

Finally it should look like something like below.
```
apiVersion: v1
clusters:
- cluster:
insecure-skip-tls-verify: true
server: https://c309b9b1359f.ngrok.io
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0.......
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0.......
```

Copy this content into `KUBE_QA_CONFIG` GitHub secret. Also additional ones for the other environments.

This is what you should have.



Prepare local cluster


You have to repeat these steps for each three environments.


// Create namespace

kubectl create namespace pipeline-qa

// Create application secrets for the deployment beforehand

# ./Desktop/.env
KEY=qa-secret

$ kubectl --namespace=pipeline-qa create secret generic pipeline-secret --save-config --from-file=./Desktop/.env
secret/pipeline-secret created

$ kubectl --namespace=pipeline-qa get secret/pipeline-secret -o jsonpath='{.data}'
{".env":"S0VZPXFhLXNlY3JldAo="}
$ echo 'S0VZPXFhLXNlY3JldAo=' | base64 --decode
KEY=qa-secret

Test


Assuming that you have completed all these steps then pull request builder and deployment succeeded. Lets test the API. Very first deployment will have two replicas however the consequent ones will have only one as we patch "rollout" annotation.


Cluster info


$ kubectl --namespace=pipeline-qa get all
NAME READY STATUS RESTARTS AGE
pod/pipeline-deployment-5767699c9c-dzp2m 1/1 Running 0 26s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/pipeline-service ClusterIP 10.111.146.68 80/TCP 28s

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/pipeline-deployment 1/1 1 1 27s

NAME DESIRED CURRENT READY AGE
replicaset.apps/pipeline-deployment-5767699c9c 1 1 1 26s
replicaset.apps/pipeline-deployment-5d5c66fcf6 0 0 0 27s

Logs


$ kubectl --namespace=pipeline-qa logs pod/pipeline-deployment-5767699c9c-dzp2m
2021/08/03 19:45:37 running pipeline: {com:3f07885 env:qa adr::8080 key:qa-secret}

Accessing the application


$ kubectl --namespace=pipeline-qa port-forward service/pipeline-service 3000:80
Forwarding from 127.0.0.1:3000 -> 8080
Forwarding from [::1]:3000 -> 8080
Handling connection for 3000

This is what you should get as a result.


$ curl http://0.0.0.0:3000
{com:3f07885 env:qa adr::8080 key:qa-secret}

Pull request builder



Continuous deployment



Manual deployment




DockerHub tags