Deploying a Golang application to Kubernetes cluster using GitHub actions
01/08/2021 - DOCKER, GIT, GO, KUBERNETES
In this example we are going to use GitHub actions to deploy our Golang application to a Kubernetes cluster. We have qa, stage and prod Kubernetes environments. We have three different workflows to work with these environments. I am going to use a local Minikube cluster so there will be a "setup" section below to expose it to public with Ngrok.
Structure
Although we are going to focus on the Kubernetes here, you can run go run -ldflags "-s -w -X main.commit=`git rev-parse --short HEAD`" -race main.go command to run the application in your local environment and access it with http://0.0.0.0:3000.
├── .env
├── .gitignore
├── .dockerignore
├── .github
│ └── workflows
│ ├── continuous_deployment.yaml
│ ├── manual_deployment.yaml
│ └── pull_request_builder.yaml
├── deploy
│ └── k8s
│ ├── prod.yaml
│ ├── qa.yaml
│ └── stage.yaml
├── docker
│ └── ci
│ └── Dockerfile
├── go.mod
├── go.sum
├── main.go
└── main_test.go
Files
.env
# All these values are used for the local environment, not K8S!
ENV=dev
ADR=:3000
KEY=dev-secret
.dockerignore
bin/
.dockerignore
.gitignore
*.md
.env
.git/
.idea/
.DS_Store/
Dockerfile
FROM golang:1.15-alpine3.12 as build
WORKDIR /source
COPY . .
ARG COMMIT
RUN CGO_ENABLED=0 go build -ldflags "-s -w -X main.commit=${COMMIT}" -o bin/pipeline main.go
FROM alpine:3.12
COPY --from=build /source/bin/pipeline /bin/pipeline
EXPOSE 8080
ENTRYPOINT ["./bin/pipeline"]
main.go
package main
import (
"fmt"
"log"
"net/http"
"os"
"github.com/joho/godotenv"
)
var commit string
func main() {
// Load environment variables
if err := godotenv.Load(".env"); err != nil {
log.Fatal(err)
}
env, _ := os.LookupEnv("ENV")
adr, _ := os.LookupEnv("ADR")
key, _ := os.LookupEnv("KEY")
// Bootstrap HTTP handler
hom := home{commit, env, adr, key}
// Bootstrap HTTP router.
rtr := http.DefaultServeMux
rtr.HandleFunc("/", hom.handler)
// Start HTTP server.
log.Printf("running pipeline: %+v\n", hom)
log.Fatalln(http.ListenAndServe(adr, rtr))
}
type home struct {
com string
env string
adr string
key string
}
func (h home) handler(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(fmt.Sprintf("%+v", h)))
}
main_test.go
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func Test_home_handler(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/", nil)
res := httptest.NewRecorder()
home{}.handler(res, req)
if res.Code != http.StatusOK {
t.Error("expected 200 but got", res.Code)
}
}
GitHub workflows
I have kept the jobs as minimal as possible otherwise this post would be very long. As a result certain steps are ignored such as linter so on.
pull_request_builder.yaml
# Trigger the workflow only when:
# - a new pull request with any name/type is opened against the master, develop, hotfix/* or release/* branch
# - a commit is directly pushed to the pull request
name: Pull request builder
on:
pull_request:
branches:
- master
- develop
- hotfix/*
- release/*
jobs:
setup:
runs-on: ubuntu-latest
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum
test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...
continuous_deployment.yaml
# Trigger the workflow to deploy to qa environment only when:
# - an existing pull request with any name/type is merged to the master or develop branch
# - a commit is directly pushed to the master or develop branch
name: Continuous deployment
on:
push:
branches:
- master
- develop
jobs:
setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum
test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...
docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:qa
build-args: COMMIT=${{ needs.setup.outputs.commit }}
deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/qa.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-qa \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"
manual_deployment.yaml
# Trigger the workflow to deploy a specific git reference to a specific environment only when:
# - the `workflow_dispatch` event is used in the UI.
# This is ideal for environment such as production, staging or sandbox if you wish to make the
# deployment manual.
name: Manual deployment
on:
workflow_dispatch:
inputs:
env:
description: "Environment to deploy - options: qa|stage|prod"
required: true
ref:
description: "Git reference to deploy - example: branch/tag/sha"
required: true
jobs:
setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Deployment info
run: echo "Deploying '${{ github.event.inputs.ref }}' to '${{ github.event.inputs.env }}' environment"
- name: Verifying environment
run: |
envs=("qa stage prod")
[[ ${envs[*]} =~ ${{ github.event.inputs.env }} ]] || { echo "Invalid environment"; exit 1; }
- name: Use repository
uses: actions/checkout@v2
with:
ref: ${{ github.event.inputs.ref }}
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum
test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...
docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:${{ github.event.inputs.env }}
build-args: COMMIT=${{ needs.setup.outputs.commit }}
deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
case ${{ github.event.inputs.env }} in
qa) echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config ;;
stage) echo "${{ secrets.KUBE_STAGE_CONFIG }}" > $HOME/.kube/config ;;
prod) echo "${{ secrets.KUBE_PROD_CONFIG }}" > $HOME/.kube/config ;;
*) echo "Invalid environment"; exit 1;;
esac
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/${{ github.event.inputs.env }}.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-${{ github.event.inputs.env }} \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"
Kubernetes configuration
As shown below, there are three environments and each of which is isolated within their dedicated namespaces. The rollout and imagePullPolicy properties make sure that the new image is pulled at every deployment.
qa.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pipeline-qa
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-qa
data:
ENV: qa
ADR: :8080
---
apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-qa
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-qa
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:qa
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env
stage.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pipeline-stage
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-stage
data:
ENV: stage
ADR: :8080
---
apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-stage
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-stage
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:stage
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env
prod.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pipeline-prod
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-prod
data:
ENV: prod
ADR: :8080
---
apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-prod
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-prod
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 2
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:prod
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env
Logic
Pull Request
- First time PR creation runs pull_request_builder.yaml
- Pushing a commit to an existing PR runs pull_request_builder.yaml
- Merging an existing PR to develop/master runs continuous_deployment.yaml
- Direct commit push to develop/master runs continuous_deployment.yaml
Release/Hotfix
- First time release/hotfix creation against develop/master runs pull_request_builder.yaml
- Pushing a commit to an existing release/hotfix runs pull_request_builder.yaml
- Creating a new PR against the release/hotfix runs pull_request_builder.yaml
- Pushing a commit to an existing PR that tracks a release/hotfix runs pull_request_builder.yaml
- Merging an existing PR to a release/hotfix runs pull_request_builder.yaml
- Direct commit push to release/hotfix runs pull_request_builder.yaml
- Merging an existing release/hotfix to develop/master runs continuous_deployment.yaml
- Merging an existing release/hotfix to develop/master in CLI and pushing develop/master runs continuous_deployment.yaml
Tag
- Manually creating a tag does nt trigger any pipeline
- Pushing a tag from local does not trigger any pipeline
Manual Deployment
- Manually deploying a specific git ref (branch/tag/sha) to a specific environment
Others
- Closed PRs will not trigger any pipeline
- Rollback is same as manual deployment so deploy a previous tag
Setup
Add Docker secrets and Kube config to GitHub
This will let your workflow to push images to DockerHub registry.
DOCKERHUB_USERNAME
DOCKERHUB_TOKEN
This will let your workflows to deploy the application to your local K8S cluster.
// Enable routing public requests to K8S API.
$ kubectl proxy --disable-filter=true
Starting to serve on 127.0.0.1:8001
(Optionally $ kubectl proxy --port=8001 --accept-hosts='.*\.ngrok.io')
// Expose kubectl proxy to the Internet with ngrok.
$ ./ngrok http 127.0.0.1:8001
Forwarding http://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001
Forwarding https://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001
// Create a modified copy of your local kube config.
$ kubectl config view --flatten > ~/Desktop/kube_config
Remove `certificate-authority-data` line
Add `insecure-skip-tls-verify: true` line
Replace `server` value to `https://c309b9b1359f.ngrok.io`
Finally it should look like something like below.
```
apiVersion: v1
clusters:
- cluster:
insecure-skip-tls-verify: true
server: https://c309b9b1359f.ngrok.io
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0.......
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0.......
```
Copy this content into `KUBE_QA_CONFIG` GitHub secret. Also additional ones for the other environments.
This is what you should have.
Prepare local cluster
You have to repeat these steps for each three environments.
// Create namespace
kubectl create namespace pipeline-qa
// Create application secrets for the deployment beforehand
# ./Desktop/.env
KEY=qa-secret
$ kubectl --namespace=pipeline-qa create secret generic pipeline-secret --save-config --from-file=./Desktop/.env
secret/pipeline-secret created
$ kubectl --namespace=pipeline-qa get secret/pipeline-secret -o jsonpath='{.data}'
{".env":"S0VZPXFhLXNlY3JldAo="}
$ echo 'S0VZPXFhLXNlY3JldAo=' | base64 --decode
KEY=qa-secret
Test
Assuming that you have completed all these steps then pull request builder and deployment succeeded. Lets test the API. Very first deployment will have two replicas however the consequent ones will have only one as we patch "rollout" annotation.
Cluster info
$ kubectl --namespace=pipeline-qa get all
NAME READY STATUS RESTARTS AGE
pod/pipeline-deployment-5767699c9c-dzp2m 1/1 Running 0 26s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/pipeline-service ClusterIP 10.111.146.6880/TCP 28s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/pipeline-deployment 1/1 1 1 27s
NAME DESIRED CURRENT READY AGE
replicaset.apps/pipeline-deployment-5767699c9c 1 1 1 26s
replicaset.apps/pipeline-deployment-5d5c66fcf6 0 0 0 27s
Logs
$ kubectl --namespace=pipeline-qa logs pod/pipeline-deployment-5767699c9c-dzp2m
2021/08/03 19:45:37 running pipeline: {com:3f07885 env:qa adr::8080 key:qa-secret}
Accessing the application
$ kubectl --namespace=pipeline-qa port-forward service/pipeline-service 3000:80
Forwarding from 127.0.0.1:3000 -> 8080
Forwarding from [::1]:3000 -> 8080
Handling connection for 3000
This is what you should get as a result.
$ curl http://0.0.0.0:3000
{com:3f07885 env:qa adr::8080 key:qa-secret}
Pull request builder
Continuous deployment
Manual deployment
DockerHub tags
