Herkese merhaba!

Uzun yıllardır bol miktarda kişisel zaman ve enerji harcayarak bilgimizi hepinizle paylaşıyoruz. Ancak şu andan itibaren bu blogu çalışır durumda tutabilmek için yardımınıza ihtiyacımız var. Yapmanız gereken tek şey, sitedeki reklamlardan birine tıklamak olacaktır, aksi takdirde hosting vb. masraflar nedeniyle maalesef yayından kaldırılacaktır. Teşekkürler.

Bu örnekte, Golang uygulamamızı bir Kubernetes kümesine dağıtmak için GitHub eylemlerini (actions) kullanacağız. qa, stage ve prod Kubernetes ortamlarımız var. Bu ortamlarla çalışmak için üç farklı iş akışımız var. Yerel bir Minikube kümesi kullanacağım, bu yüzden onu Ngrok ile herkese açık hale getirmiz gerekecek, ki gerekli bilgi aşağıdaki "kurulum" bölümünde olacak.


Yapı


Burada Kubernetes'e odaklanacak olsak da, go run -ldflags "-s -w -X main.commit=`git rev-parse --short HEAD`" -race main.go komutu ile uygulamayı yerel ortamda test amaçlı çalıştırıp, http://0.0.0.0:3000 adresinden test edebilirsiniz.


├── .env
├── .gitignore
├── .dockerignore
├── .github
│   └── workflows
│   ├── continuous_deployment.yaml
│   ├── manual_deployment.yaml
│   └── pull_request_builder.yaml
├── deploy
│   └── k8s
│   ├── prod.yaml
│   ├── qa.yaml
│   └── stage.yaml
├── docker
│   └── ci
│   └── Dockerfile
├── go.mod
├── go.sum
├── main.go
└── main_test.go

Dosyalar


.env


# All these values are used for the local environment, not K8S!
ENV=dev
ADR=:3000
KEY=dev-secret

.dockerignore


bin/

.dockerignore
.gitignore
*.md
.env

.git/
.idea/
.DS_Store/

Dockerfile


FROM golang:1.15-alpine3.12 as build

WORKDIR /source
COPY . .

ARG COMMIT
RUN CGO_ENABLED=0 go build -ldflags "-s -w -X main.commit=${COMMIT}" -o bin/pipeline main.go

FROM alpine:3.12

COPY --from=build /source/bin/pipeline /bin/pipeline

EXPOSE 8080

ENTRYPOINT ["./bin/pipeline"]

main.go


package main

import (
"fmt"
"log"
"net/http"
"os"

"github.com/joho/godotenv"
)

var commit string

func main() {
// Load environment variables
if err := godotenv.Load(".env"); err != nil {
log.Fatal(err)
}
env, _ := os.LookupEnv("ENV")
adr, _ := os.LookupEnv("ADR")
key, _ := os.LookupEnv("KEY")

// Bootstrap HTTP handler
hom := home{commit, env, adr, key}

// Bootstrap HTTP router.
rtr := http.DefaultServeMux
rtr.HandleFunc("/", hom.handler)

// Start HTTP server.
log.Printf("running pipeline: %+v\n", hom)
log.Fatalln(http.ListenAndServe(adr, rtr))
}

type home struct {
com string
env string
adr string
key string
}

func (h home) handler(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(fmt.Sprintf("%+v", h)))
}

main_test.go


package main

import (
"net/http"
"net/http/httptest"
"testing"
)

func Test_home_handler(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/", nil)
res := httptest.NewRecorder()

home{}.handler(res, req)

if res.Code != http.StatusOK {
t.Error("expected 200 but got", res.Code)
}
}

GitHub iş akışları


İşleri olabildiğince az tuttum, aksi takdirde bu yazı çok uzun olurdu. Sonuç olarak, linter gibi bazı adımlar göz ardı ettim.


pull_request_builder.yaml


# Trigger the workflow only when:
# - a new pull request with any name/type is opened against the master, develop, hotfix/* or release/* branch
# - a commit is directly pushed to the pull request

name: Pull request builder

on:
pull_request:
branches:
- master
- develop
- hotfix/*
- release/*

jobs:

setup:
runs-on: ubuntu-latest
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum

test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...

continuous_deployment.yaml


# Trigger the workflow to deploy to qa environment only when:
# - an existing pull request with any name/type is merged to the master or develop branch
# - a commit is directly pushed to the master or develop branch

name: Continuous deployment

on:
push:
branches:
- master
- develop

jobs:

setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Use repository
uses: actions/checkout@v2
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum

test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...

docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:qa
build-args: COMMIT=${{ needs.setup.outputs.commit }}

deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/qa.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-qa \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"

manual_deployment.yaml


# Trigger the workflow to deploy a specific git reference to a specific environment only when:
# - the `workflow_dispatch` event is used in the UI.
# This is ideal for environment such as production, staging or sandbox if you wish to make the
# deployment manual.

name: Manual deployment

on:
workflow_dispatch:
inputs:
env:
description: "Environment to deploy - options: qa|stage|prod"
required: true
ref:
description: "Git reference to deploy - example: branch/tag/sha"
required: true

jobs:

setup:
runs-on: ubuntu-latest
outputs:
repo: ${{ steps.vars.outputs.repo }}
commit: ${{ steps.vars.outputs.commit }}
steps:
- name: Deployment info
run: echo "Deploying '${{ github.event.inputs.ref }}' to '${{ github.event.inputs.env }}' environment"
- name: Verifying environment
run: |
envs=("qa stage prod")
[[ ${envs[*]} =~ ${{ github.event.inputs.env }} ]] || { echo "Invalid environment"; exit 1; }
- name: Use repository
uses: actions/checkout@v2
with:
ref: ${{ github.event.inputs.ref }}
- name: Build variables
id: vars
run: |
echo "::set-output name=repo::$GITHUB_REPOSITORY"
echo "::set-output name=commit::$(git rev-parse --short "$GITHUB_SHA")"
- name: Upload repository
uses: actions/upload-artifact@v2
with:
name: repository
path: |
${{ github.workspace }}/deploy
${{ github.workspace }}/docker
${{ github.workspace }}/.dockerignore
${{ github.workspace }}/main.go
${{ github.workspace }}/main_test.go
${{ github.workspace }}/go.mod
${{ github.workspace }}/go.sum

test:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Use Golang 1.16
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Run tests
run: go test -v -race -timeout=180s -count=1 -cover ./...

docker:
needs: [setup, test]
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v2
with:
push: true
file: ./docker/ci/Dockerfile
tags: ${{ needs.setup.outputs.repo }}:${{ github.event.inputs.env }}
build-args: COMMIT=${{ needs.setup.outputs.commit }}

deploy:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Download repository
uses: actions/download-artifact@v2
with:
name: repository
- name: Install kubectl
run: |
curl -LO https://dl.k8s.io/release/v1.21.0/bin/linux/amd64/kubectl
curl -LO "https://dl.k8s.io/v1.21.0/bin/linux/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
- name: Create kube config
run: |
mkdir -p $HOME/.kube/
case ${{ github.event.inputs.env }} in
qa) echo "${{ secrets.KUBE_QA_CONFIG }}" > $HOME/.kube/config ;;
stage) echo "${{ secrets.KUBE_STAGE_CONFIG }}" > $HOME/.kube/config ;;
prod) echo "${{ secrets.KUBE_PROD_CONFIG }}" > $HOME/.kube/config ;;
*) echo "Invalid environment"; exit 1;;
esac
- name: Deploy
run: |
kubectl --kubeconfig $HOME/.kube/config apply -f deploy/k8s/${{ github.event.inputs.env }}.yaml
kubectl --kubeconfig $HOME/.kube/config patch deployment pipeline-deployment --namespace=pipeline-${{ github.event.inputs.env }} \
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rollout\":\"`date +'%s'`\"}}}}}"

Kubernetes yapılandırması


Aşağıda gösterildiği gibi, üç ortam vardır ve bunların her biri kendilerine ayrılmış ad alanlarında yalıtılmıştır. rollout ve imagePullPolicy özellikleri, her dağıtımda yeni görüntünün (Docker imaj) alınmasını sağlar.


qa.yaml


apiVersion: v1
kind: Namespace
metadata:
name: pipeline-qa

---

apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-qa
data:
ENV: qa
ADR: :8080

---

apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-qa
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-qa
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:qa
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env

stage.yaml


apiVersion: v1
kind: Namespace
metadata:
name: pipeline-stage

---

apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-stage
data:
ENV: stage
ADR: :8080

---

apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-stage
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-stage
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 1
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:stage
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env

prod.yaml


apiVersion: v1
kind: Namespace
metadata:
name: pipeline-prod

---

apiVersion: v1
kind: ConfigMap
metadata:
name: pipeline-configmap
namespace: pipeline-prod
data:
ENV: prod
ADR: :8080

---

apiVersion: v1
kind: Service
metadata:
name: pipeline-service
namespace: pipeline-prod
spec:
type: ClusterIP
selector:
app: pipeline
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: pipeline-deployment
namespace: pipeline-prod
labels:
app: pipeline
annotations:
rollout: ""
spec:
replicas: 2
selector:
matchLabels:
app: pipeline
template:
metadata:
labels:
app: pipeline
spec:
containers:
- name: golang
image: you/pipeline:prod
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
envFrom:
- configMapRef:
name: pipeline-configmap
volumeMounts:
- name: dotenv
mountPath: ./.env
subPath: .env
readOnly: true
volumes:
- name: dotenv
secret:
secretName: pipeline-secret
items:
- key: .env
path: ./.env

Mantık


Pull Request


- First time PR creation runs pull_request_builder.yaml
- Pushing a commit to an existing PR runs pull_request_builder.yaml
- Merging an existing PR to develop/master runs continuous_deployment.yaml
- Direct commit push to develop/master runs continuous_deployment.yaml

Release/Hotfix


- First time release/hotfix creation against develop/master runs pull_request_builder.yaml
- Pushing a commit to an existing release/hotfix runs pull_request_builder.yaml
- Creating a new PR against the release/hotfix runs pull_request_builder.yaml
- Pushing a commit to an existing PR that tracks a release/hotfix runs pull_request_builder.yaml
- Merging an existing PR to a release/hotfix runs pull_request_builder.yaml
- Direct commit push to release/hotfix runs pull_request_builder.yaml
- Merging an existing release/hotfix to develop/master runs continuous_deployment.yaml
- Merging an existing release/hotfix to develop/master in CLI and pushing develop/master runs continuous_deployment.yaml

Tag


- Manually creating a tag does nt trigger any pipeline
- Pushing a tag from local does not trigger any pipeline

Manuel Dağıtım (Manual Deployment)


- Manually deploying a specific git ref (branch/tag/sha) to a specific environment

Diğerleri


- Closed PRs will not trigger any pipeline
- Rollback is same as manual deployment so deploy a previous tag

Kurulum


GitHub'a Docker sırları ve Kube yapılandırması ekleyin


Bu, iş akışınızın görüntüleri DockerHub kayıt defterine göndermesini sağlar.


DOCKERHUB_USERNAME
DOCKERHUB_TOKEN

Bu, iş akışlarınızın uygulamayı yerel K8S kümenize dağıtmasına olanak tanır.


// Enable routing public requests to K8S API.

$ kubectl proxy --disable-filter=true
Starting to serve on 127.0.0.1:8001
(Optionally $ kubectl proxy --port=8001 --accept-hosts='.*\.ngrok.io')

// Expose kubectl proxy to the Internet with ngrok.

$ ./ngrok http 127.0.0.1:8001
Forwarding http://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001
Forwarding https://c309b9b1359f.ngrok.io -> http://127.0.0.1:8001

// Create a modified copy of your local kube config.

$ kubectl config view --flatten > ~/Desktop/kube_config
Remove `certificate-authority-data` line
Add `insecure-skip-tls-verify: true` line
Replace `server` value to `https://c309b9b1359f.ngrok.io`

Finally it should look like something like below.
```
apiVersion: v1
clusters:
- cluster:
insecure-skip-tls-verify: true
server: https://c309b9b1359f.ngrok.io
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0.......
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0.......
```

Copy this content into `KUBE_QA_CONFIG` GitHub secret. Also additional ones for the other environments.

Sahip olmanız gereken şey bu.



Yerel kümeyi hazırlayın


Bu adımları her üç ortam için tekrarlamanız gerekir.


// Create namespace

kubectl create namespace pipeline-qa

// Create application secrets for the deployment beforehand

# ./Desktop/.env
KEY=qa-secret

$ kubectl --namespace=pipeline-qa create secret generic pipeline-secret --save-config --from-file=./Desktop/.env
secret/pipeline-secret created

$ kubectl --namespace=pipeline-qa get secret/pipeline-secret -o jsonpath='{.data}'
{".env":"S0VZPXFhLXNlY3JldAo="}
$ echo 'S0VZPXFhLXNlY3JldAo=' | base64 --decode
KEY=qa-secret

Test


Tüm bu adımları tamamladığınızı varsayıyorum, ayrıca pull request ve dağıtımın başarılı olduğunu varsayıyorum. API'yi test edelim. İlk dağıtımın iki kopyası olacak, ancak "rollout" ek açıklamasını yamaladığımız için sonrakiler yalnızca bir kopyaya sahip olacak.


Küme bilgisi


$ kubectl --namespace=pipeline-qa get all
NAME READY STATUS RESTARTS AGE
pod/pipeline-deployment-5767699c9c-dzp2m 1/1 Running 0 26s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/pipeline-service ClusterIP 10.111.146.68 80/TCP 28s

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/pipeline-deployment 1/1 1 1 27s

NAME DESIRED CURRENT READY AGE
replicaset.apps/pipeline-deployment-5767699c9c 1 1 1 26s
replicaset.apps/pipeline-deployment-5d5c66fcf6 0 0 0 27s

Kütükler


$ kubectl --namespace=pipeline-qa logs pod/pipeline-deployment-5767699c9c-dzp2m
2021/08/03 19:45:37 running pipeline: {com:3f07885 env:qa adr::8080 key:qa-secret}

Uygulamaya erişim


$ kubectl --namespace=pipeline-qa port-forward service/pipeline-service 3000:80
Forwarding from 127.0.0.1:3000 -> 8080
Forwarding from [::1]:3000 -> 8080
Handling connection for 3000

Sonuç olarak almanız gereken şey bu.


$ curl http://0.0.0.0:3000
{com:3f07885 env:qa adr::8080 key:qa-secret}

Pull request builder



Continuous deployment



Manual deployment




DockerHub tags