GitHub eylemlerini kullanarak bir Golang uygulamasını Kubernetes kümesine dağıtma
01/08/2021 - DOCKER, GIT, GO, KUBERNETES
Bu örnekte, Golang uygulamamızı bir Kubernetes kümesine dağıtmak için GitHub eylemlerini (actions) kullanacağız. qa, stage ve prod Kubernetes ortamlarımız var. Bu ortamlarla çalışmak için üç farklı iş akışımız var. Yerel bir Minikube kümesi kullanacağım, bu yüzden onu Ngrok ile herkese açık hale getirmiz gerekecek, ki gerekli bilgi aşağıdaki "kurulum" bölümünde olacak.
Yapı
Burada Kubernetes'e odaklanacak olsak da, go run -ldflags "-s -w -X main.commit=`git rev-parse --short HEAD`" -race main.go komutu ile uygulamayı yerel ortamda test amaçlı çalıştırıp, http://0.0.0.0:3000 adresinden test edebilirsiniz.
├── .env
├── .gitignore
├── .dockerignore
├── .github
│ └── workflows
│ ├── continuous_deployment.yaml
│ ├── manual_deployment.yaml
│ └── pull_request_builder.yaml
├── deploy
│ └── k8s
│ ├── prod.yaml
│ ├── qa.yaml
│ └── stage.yaml
├── docker
│ └── ci
│ └── Dockerfile
├── go.mod
├── go.sum
├── main.go
└── main_test.go
Dosyalar
.env
# All these values are used for the local environment, not K8S!
ENV=dev
ADR=:3000
KEY=dev-secret
.dockerignore
bin/
.dockerignore
.gitignore
*.md
.env
.git/
.idea/
.DS_Store/
Dockerfile
FROM golang:1.15-alpine3.12 as build
WORKDIR /source
COPY . .
ARG COMMIT
RUN CGO_ENABLED=0 go build -ldflags "-s -w -X main.commit=${COMMIT}" -o bin/pipeline main.go
FROM alpine:3.12
COPY --from=build /source/bin/pipeline /bin/pipeline
EXPOSE 8080
ENTRYPOINT ["./bin/pipeline"]
main.go
package main
import (
"fmt"
"log"
"net/http"
"os"
"github.com/joho/godotenv"
)
var commit string
func main() {
// Load environment variables
if err := godotenv.Load(".env"); err != nil {
log.Fatal(err)
}
env, _ := os.LookupEnv("ENV")
adr, _ := os.LookupEnv("ADR")
key, _ := os.LookupEnv("KEY")
// Bootstrap HTTP handler
hom := home{commit, env, adr, key}
// Bootstrap HTTP router.
rtr := http.DefaultServeMux
rtr.HandleFunc("/", hom.handler)
// Start HTTP server.
log.Printf("running pipeline: %+v\n", hom)
log.Fatalln(http.ListenAndServe(adr, rtr))
}
type home struct {
com string
env string
adr string
key string
}
func (h home) handler(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(fmt.Sprintf("%+v", h)))
}
main_test.go
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func Test_home_handler(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/", nil)
res := httptest.NewRecorder()
home{}.handler(res, req)
if res.Code != http.StatusOK {
t.Error("expected 200 but got", res.Code)
}
}
GitHub iş akışları
İşleri olabildiğince az tuttum, aksi takdirde bu yazı çok uzun olurdu. Sonuç olarak, linter gibi bazı adımlar göz ardı ettim.
pull_request_builder.yaml
# Trigger the workflow only when:
# - a new pull request with any name/type is opened against the master, develop, hotfix/* or release/* branch
# - a commit is directly pushed to the pull request
name: Pull request builder
on:
pull_request:
branches:
- master
- develop
- hotfix/*
- release/*
jobs:
setup:
runs-on: ubuntu-latest
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum
test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...
continuous_deployment.yaml
# Trigger the workflow to deploy to qa environment only when:
# - an existing pull request with any name/type is merged to the master or develop branch
# - a commit is directly pushed to the master or develop branch
name: Continuous deployment
on:
push:
branches:
- master
- develop
jobs:
setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum
test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...
docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:qa
build-args: COMMIT=${{ needs.setup.outputs.commit }}
deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/qa.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-qa \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"
manual_deployment.yaml
# Trigger the workflow to deploy a specific git reference to a specific environment only when:
# - the `workflow_dispatch` event is used in the UI.
# This is ideal for environment such as production, staging or sandbox if you wish to make the
# deployment manual.
name: Manual deployment
on:
workflow_dispatch:
inputs:
env:
description: "Environment to deploy - options: qa|stage|prod"
required: true
ref:
description: "Git reference to deploy - example: branch/tag/sha"
required: true
jobs:
setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Deployment info
run: echo "Deploying '${{ github.event.inputs.ref }}' to '${{ github.event.inputs.env }}' environment"
- name: Verifying environment
run: |
envs=("qa stage prod")
[[ ${envs[*]} =~ ${{ github.event.inputs.env }} ]] || { echo "Invalid environment"; exit 1; }
- name: Use repository
uses: actions/checkout@v2
with:
ref: ${{ github.event.inputs.ref }}
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum
test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...
docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:${{ github.event.inputs.env }}
build-args: COMMIT=${{ needs.setup.outputs.commit }}
deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
case ${{ github.event.inputs.env }} in
qa) echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config ;;
stage) echo "${{ secrets.KUBE_STAGE_CONFIG }}" > $HOME/.kube/config ;;
prod) echo "${{ secrets.KUBE_PROD_CONFIG }}" > $HOME/.kube/config ;;
*) echo "Invalid environment"; exit 1;;
esac
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/${{ github.event.inputs.env }}.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-${{ github.event.inputs.env }} \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"
Kubernetes yapılandırması
Aşağıda gösterildiği gibi, üç ortam vardır ve bunların her biri kendilerine ayrılmış ad alanlarında yalıtılmıştır. rollout ve imagePullPolicy özellikleri, her dağıtımda yeni görüntünün (Docker imaj) alınmasını sağlar.
qa.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pipeline-qa
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-qa
data:
ENV: qa
ADR: :8080
---
apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-qa
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-qa
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:qa
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env
stage.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pipeline-stage
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-stage
data:
ENV: stage
ADR: :8080
---
apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-stage
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-stage
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:stage
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env
prod.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pipeline-prod
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-prod
data:
ENV: prod
ADR: :8080
---
apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-prod
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-prod
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 2
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:prod
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env
Mantık
Pull Request
- First time PR creation runs pull_request_builder.yaml
- Pushing a commit to an existing PR runs pull_request_builder.yaml
- Merging an existing PR to develop/master runs continuous_deployment.yaml
- Direct commit push to develop/master runs continuous_deployment.yaml
Release/Hotfix
- First time release/hotfix creation against develop/master runs pull_request_builder.yaml
- Pushing a commit to an existing release/hotfix runs pull_request_builder.yaml
- Creating a new PR against the release/hotfix runs pull_request_builder.yaml
- Pushing a commit to an existing PR that tracks a release/hotfix runs pull_request_builder.yaml
- Merging an existing PR to a release/hotfix runs pull_request_builder.yaml
- Direct commit push to release/hotfix runs pull_request_builder.yaml
- Merging an existing release/hotfix to develop/master runs continuous_deployment.yaml
- Merging an existing release/hotfix to develop/master in CLI and pushing develop/master runs continuous_deployment.yaml
Tag
- Manually creating a tag does nt trigger any pipeline
- Pushing a tag from local does not trigger any pipeline
Manuel Dağıtım (Manual Deployment)
- Manually deploying a specific git ref (branch/tag/sha) to a specific environment
Diğerleri
- Closed PRs will not trigger any pipeline
- Rollback is same as manual deployment so deploy a previous tag
Kurulum
GitHub'a Docker sırları ve Kube yapılandırması ekleyin
Bu, iş akışınızın görüntüleri DockerHub kayıt defterine göndermesini sağlar.
DOCKERHUB_USERNAME
DOCKERHUB_TOKEN
Bu, iş akışlarınızın uygulamayı yerel K8S kümenize dağıtmasına olanak tanır.
// Enable routing public requests to K8S API.
$ kubectl proxy --disable-filter=true
Starting to serve on 127.0.0.1:8001
(Optionally $ kubectl proxy --port=8001 --accept-hosts='.*\.ngrok.io')
// Expose kubectl proxy to the Internet with ngrok.
$ ./ngrok http 127.0.0.1:8001
Forwarding http://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001
Forwarding https://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001
// Create a modified copy of your local kube config.
$ kubectl config view --flatten > ~/Desktop/kube_config
Remove `certificate-authority-data` line
Add `insecure-skip-tls-verify: true` line
Replace `server` value to `https://c309b9b1359f.ngrok.io`
Finally it should look like something like below.
```
apiVersion: v1
clusters:
- cluster:
insecure-skip-tls-verify: true
server: https://c309b9b1359f.ngrok.io
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0.......
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0.......
```
Copy this content into `KUBE_QA_CONFIG` GitHub secret. Also additional ones for the other environments.
Sahip olmanız gereken şey bu.
Yerel kümeyi hazırlayın
Bu adımları her üç ortam için tekrarlamanız gerekir.
// Create namespace
kubectl create namespace pipeline-qa
// Create application secrets for the deployment beforehand
# ./Desktop/.env
KEY=qa-secret
$ kubectl --namespace=pipeline-qa create secret generic pipeline-secret --save-config --from-file=./Desktop/.env
secret/pipeline-secret created
$ kubectl --namespace=pipeline-qa get secret/pipeline-secret -o jsonpath='{.data}'
{".env":"S0VZPXFhLXNlY3JldAo="}
$ echo 'S0VZPXFhLXNlY3JldAo=' | base64 --decode
KEY=qa-secret
Test
Tüm bu adımları tamamladığınızı varsayıyorum, ayrıca pull request ve dağıtımın başarılı olduğunu varsayıyorum. API'yi test edelim. İlk dağıtımın iki kopyası olacak, ancak "rollout" ek açıklamasını yamaladığımız için sonrakiler yalnızca bir kopyaya sahip olacak.
Küme bilgisi
$ kubectl --namespace=pipeline-qa get all
NAME READY STATUS RESTARTS AGE
pod/pipeline-deployment-5767699c9c-dzp2m 1/1 Running 0 26s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/pipeline-service ClusterIP 10.111.146.6880/TCP 28s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/pipeline-deployment 1/1 1 1 27s
NAME DESIRED CURRENT READY AGE
replicaset.apps/pipeline-deployment-5767699c9c 1 1 1 26s
replicaset.apps/pipeline-deployment-5d5c66fcf6 0 0 0 27s
Kütükler
$ kubectl --namespace=pipeline-qa logs pod/pipeline-deployment-5767699c9c-dzp2m
2021/08/03 19:45:37 running pipeline: {com:3f07885 env:qa adr::8080 key:qa-secret}
Uygulamaya erişim
$ kubectl --namespace=pipeline-qa port-forward service/pipeline-service 3000:80
Forwarding from 127.0.0.1:3000 -> 8080
Forwarding from [::1]:3000 -> 8080
Handling connection for 3000
Sonuç olarak almanız gereken şey bu.
$ curl http://0.0.0.0:3000
{com:3f07885 env:qa adr::8080 key:qa-secret}
Pull request builder
Continuous deployment
Manual deployment
DockerHub tags
