GitHub Actions, ArgoCD, Helm ve Kubernetes kullanarak GitOps tarzı CI/CD kurulumu

20/03/2022 - DOCKER, GIT, GO, HELM, KUBERNETES, ARGOCD

Daha önce böyle bir konu hakkında yazmıştım ama bu biraz farklı, çünkü uygulama deposunda artık CI/CD ile ilgili konfigürasyon dosyaları yok. Hepsi özel depolarında saklanır. Bu ayrımın nedeni, uygun bir GitOps pratiği elde etmek istememizdir. Kısacası, uygulama ve yapılandırma deposu birbirinden ayrılmıştır.


Mevcut kurulum


argocd, dev, sbox ve prod olmak üzere dört Kubernetes kümemiz var. ArgoCD, argocd kümesinde çalışır ve diğer üç kümeye dağıtım yapar.


Dağıtım stratejisi



Kurulum


GitHub'ı hazırlayın


pacman adında bir uygulama havuzunuz olduğunu varsayıyorum.


  1. Okuma, yazma ve silme izinleriyle GITHUB_ACTIONS adlı Docker erişim belirteci oluşturun. Sonuç: 8d8937fe-753b-4f2b-96bc-b3603e4ed2b0

  2. "Repo" kapsamlarını kullanarak ARGOCD adlı PAT oluşturun. Sonuç: ghp_fhjkiuUYuy456frreSgrtry2

  3. "Repo" kapsamlarını kullanarak GITHUB_ACTIONS adlı PAT oluşturun. Sonuç: ghp_re543tgtu7hgaTHhyh6757

  4. ghp_re543tgtu7hgaTHhyh6757 kullanarak ACTIONS_TOKEN adlı pacman deposu sırrı oluşturun.

  5. 8d8937fe-753b-4f2b-96bc-b3603e4ed2b0 kullanarak DOCKERHUB_TOKEN adlı pacman deposu sırrı oluşturun.

  6. Docker kullanıcı adınızı kullanarak DOCKERHUB_USER adlı pacman deposu sırrı oluşturun.

Kubernetes'i hazırlayın


$ minikube start -p argocd --vm-driver=virtualbox --memory=2000
$ minikube start -p dev --vm-driver=virtualbox --memory=2000
$ minikube start -p sbox --vm-driver=virtualbox --memory=2000
$ minikube start -p prod --vm-driver=virtualbox --memory=2000

$ kubectl config get-contexts
CURRENT   NAME     CLUSTER   AUTHINFO   NAMESPACE
*         argocd   argocd    argocd
          dev      dev       dev
          prod     prod      prod
          sbox     sbox      sbox

ArgoCD'yi hazırlayın


Terminalde kurun

$ kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.2.4/manifests/install.yaml

Şifreyi bulun

$ kubectl get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo
// gYzYfJ5kxzJsdZGK

Login UI

$ kubectl port-forward svc/argd-server 8443:443
Forwarding from 127.0.0.1:8443 -> 8080

// Visit 127.0.0.1:8443 and use admin:gYzYfJ5kxzJsdZGK

Login CLI

$ argocd --insecure login 127.0.0.1:8443

Cluster ekleme

$ argocd cluster add dev
$ argocd cluster add sbox
$ argocd cluster add prod

Repository ekleme

Bu isteğe bağlıdır, "default" kullanabilirsiniz.


$ argocd repo add https://github.com/you/config \
    --type git \
    --name config \
    --project config \
    --username you \
    --password ghp_fhjkiuUYuy456frreSgrtry2

Project ekleme

Bu isteğe bağlıdır, "default" kullanabilirsiniz.


$ argocd proj create --file ~/local/file/system/config/infra/argocd/project.yaml

Application ekleme

Bunun yerine ApplicationSet'i kullanabilirsiniz.


$ argocd app create --file ~/local/file/system/config/infra/argocd/pacman/dev.yaml
$ argocd app create --file ~/local/file/system/config/infra/argocd/pacman/sbox.yaml
$ argocd app create --file ~/local/file/system/config/infra/argocd/pacman/prod.yaml

Uygulama deposu


├── .github
│   └── workflows
│       ├── merge.yaml
│       ├── pull_request.yaml
│       └── release.yaml
├── docker
│   └── Dockerfile
├── .dockerignore
├── main.go
└── main_test.go

Dosyalar


.dockerignore

.dockerignore
.gitignore
*.md
pacman
.DS_Store

docker/
.git/
.idea/

main.go

package main

import (
	"log"
	"net/http"
	"os"
)

var ver string

func main() {
	rtr := http.DefaultServeMux
	rtr.HandleFunc("/", home{}.handle)

	addr := os.Getenv("HTTP_ADDR")

	log.Printf("%s: info: http listen and serve pacman: %s", ver, addr)
	if err := http.ListenAndServe(addr, rtr); err != nil && err != http.ErrServerClosed {
		log.Printf("%s: error: http listen and serve pacman: %s", ver, err)
	}
}

type home struct{}

func (h home) handle(w http.ResponseWriter, r *http.Request) {
	log.Printf("%s: info: X-Request-ID: %s\n", ver, r.Header.Get("X-Request-ID"))
	_, _ = w.Write([]byte("pacman" + ver))
}

main_test.go

package main

import (
	"net/http"
	"net/http/httptest"
	"testing"
)

func Test_home_handle(t *testing.T) {
	req := httptest.NewRequest(http.MethodGet, "/", nil)
	res := httptest.NewRecorder()

	home{}.handle(res, req)

	if res.Code != http.StatusOK {
		t.Error("expected 200 but got", res.Code)
	}
}

Dockerfile

FROM golang:1.17.5-alpine3.15 as build
WORKDIR /source
COPY . .
ARG VER
RUN CGO_ENABLED=0 go build -ldflags "-s -w -X main.ver=${VER}" -o pacman main.go

FROM alpine:3.15
COPY --from=build /source/pacman /pacman
EXPOSE 8080
ENTRYPOINT ["./pacman"]

pull_request.yaml

# Trigger the workflow only when:
# - a new pull request with any name/type is opened against the master, develop, hotfix/* or release/* branch
# - a commit is directly pushed to the pull request

name: Pull Request

on:
  pull_request:
    branches:
      - master
      - develop
      - hotfix/*
      - release/*

jobs:

  setup:
    runs-on: ubuntu-latest
    steps:
      - name: Use repository
        uses: actions/checkout@v2
      - name: Upload repository
        uses: actions/upload-artifact@v2
        with:
          name: repository
          path: |
            ${{ github.workspace }}/main.go
            ${{ github.workspace }}/main_test.go
            ${{ github.workspace }}/go.mod
            ${{ github.workspace }}/go.sum

  test:
    needs: setup
    runs-on: ubuntu-latest
    steps:
      - name: Use Golang 1.17
        uses: actions/setup-go@v2
        with:
          go-version: 1.17
      - name: Download repository
        uses: actions/download-artifact@v2
        with:
          name: repository
      - name: Run tests
        run: go test -v -race -timeout=180s -count=1 -cover ./...

merge.yaml

Bu, dev dağıtımını tetiklemeye zorlamak için yapılandırma deposuna bir commit iletecektir.


# Trigger the workflow only when:
# - an existing pull request with any name/type is merged to the master or develop branch
# - a commit is directly pushed to the master or develop branch

name: Merge

on:
  push:
    branches:
      - master
      - develop

jobs:

  setup:
    runs-on: ubuntu-latest
    outputs:
      ver: ${{ steps.vars.outputs.ver }}
    steps:
      - name: Use repository
        uses: actions/checkout@v2
      - name: Build variables
        id: vars
        run: |
          echo "::set-output name=ver::$(git rev-parse --short "$GITHUB_SHA")"
      - name: Upload repository
        uses: actions/upload-artifact@v2
        with:
          name: repository
          path: |
            ${{ github.workspace }}/docker
            ${{ github.workspace }}/.dockerignore
            ${{ github.workspace }}/main.go
            ${{ github.workspace }}/main_test.go
            ${{ github.workspace }}/go.mod
            ${{ github.workspace }}/go.sum

  test:
    needs: setup
    runs-on: ubuntu-latest
    steps:
      - name: Use Golang 1.17
        uses: actions/setup-go@v2
        with:
          go-version: 1.17
      - name: Download repository
        uses: actions/download-artifact@v2
        with:
          name: repository
      - name: Run tests
        run: go test -v -race -timeout=180s -count=1 -cover ./...

  docker:
    needs: [setup, test]
    runs-on: ubuntu-latest
    steps:
      - name: Download repository
        uses: actions/download-artifact@v2
        with:
          name: repository
      - name: Login to DockerHub
        uses: docker/login-action@v1
        with:
          username: ${{ secrets.DOCKERHUB_USER }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - name: Build and push image
        uses: docker/build-push-action@v2
        with:
          push: true
          file: docker/Dockerfile
          tags: ${{ github.repository }}:latest
          build-args: VER=${{ needs.setup.outputs.ver }}

  config:
    needs: [setup, docker]
    runs-on: ubuntu-latest
    steps:
      - name: Use config repository
        uses: actions/checkout@v2
        with:
          repository: ${{ github.repository_owner }}/config
          ref: master
          token: ${{ secrets.ACTIONS_TOKEN }}
      - name: Push commit hash to config repository
        run: |
          echo ${{ needs.setup.outputs.ver }} > infra/helm/pacman/crds/vcs/hash
          git config user.name $(git log -n 1 --pretty=format:%an)
          git config user.email $(git log -n 1 --pretty=format:%ae)
          git commit infra/helm/pacman/crds/vcs/hash -m "pacman ${{ needs.setup.outputs.ver }}"
          git push origin HEAD

release.yaml

Bu, sbox dağıtımını tetiklemeye zorlamak için yapılandırma deposuna bir commit iletecektir.


# Trigger the workflow only when:
# - a new release is released which excludes pre-release and draft

name: Release

on:
  release:
    types:
      - released

jobs:

  setup:
    runs-on: ubuntu-latest
    steps:
      - name: Use repository
        uses: actions/checkout@v2
      - name: Upload repository
        uses: actions/upload-artifact@v2
        with:
          name: repository
          path: |
            ${{ github.workspace }}/.docker
            ${{ github.workspace }}/.dockerignore
            ${{ github.workspace }}/main.go
            ${{ github.workspace }}/main_test.go
            ${{ github.workspace }}/go.mod
            ${{ github.workspace }}/go.sum

  test:
    needs: setup
    runs-on: ubuntu-latest
    steps:
      - name: Use Golang 1.17
        uses: actions/setup-go@v2
        with:
          go-version: 1.17
      - name: Download repository
        uses: actions/download-artifact@v2
        with:
          name: repository
      - name: Run tests
        run: go test -v -race -timeout=180s -count=1 -cover ./...

  docker:
    needs: [setup, test]
    runs-on: ubuntu-latest
    steps:
      - name: Download repository
        uses: actions/download-artifact@v2
        with:
          name: repository
      - name: Login to DockerHub
        uses: docker/login-action@v1
        with:
          username: ${{ secrets.DOCKERHUB_USER }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - name: Build and push image
        uses: docker/build-push-action@v2
        with:
          push: true
          file: docker/Dockerfile
          tags: ${{ github.repository }}:${{ github.event.release.tag_name }}
          build-args: VER=${{ github.event.release.tag_name }}

  config:
    needs: docker
    runs-on: ubuntu-latest
    steps:
      - name: Use config repository
        uses: actions/checkout@v2
        with:
          repository: ${{ github.repository_owner }}/config
          ref: master
          token: ${{ secrets.ACTIONS_TOKEN }}
      - name: Push release tag to config repository
        run: |
          echo ${{ github.event.release.tag_name }} > infra/helm/pacman/crds/vcs/tag
          git config user.name $(git log -n 1 --pretty=format:%an)
          git config user.email $(git log -n 1 --pretty=format:%ae)
          git commit infra/helm/pacman/crds/vcs/tag -m "pacman ${{ github.event.release.tag_name }}"
          git push origin HEAD

Yapılandırma deposu


└── infra
    ├── argocd
    │   └── pacman
    │       ├── dev.yaml
    │       ├── prod.yaml
    │       ├── project.yaml
    │       └── sbox.yaml
    └── helm
        └── pacman
            ├── Chart.yaml
            ├── crds
            │   └── vcs
            │       ├── hash
            │       └── tag
            ├── dev.yaml
            ├── prod.yaml
            ├── sbox.yaml
            └── templates
                ├── configmap.yaml
                ├── deployment.yaml
                └── service.yaml

Dosyalar


argocd/dev.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: pacman-dev

spec:
  project: pacman
  source:
    repoURL: https://github.com/you/config
    path: infra/helm/pacman
    targetRevision: HEAD
    helm:
      valueFiles:
        - dev.yaml
  destination:
    name: dev
    namespace: default
  syncPolicy:
    syncOptions:
      - ApplyOutOfSyncOnly=true
      - CreateNamespace=true
    automated:
      prune: true
      selfHeal: true

argocd/sbox.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: pacman-sbox

spec:
  project: pacman
  source:
    repoURL: https://github.com/you/config
    path: infra/helm/pacman
    targetRevision: HEAD
    helm:
      valueFiles:
        - sbox.yaml
  destination:
    name: sbox
    namespace: default
  syncPolicy:
    syncOptions:
      - ApplyOutOfSyncOnly=true
      - CreateNamespace=true
    automated:
      prune: true
      selfHeal: true

argocd/prod.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: pacman-prod

spec:
  project: pacman
  source:
    repoURL: https://github.com/you/config
    path: infra/helm/pacman
    targetRevision: HEAD
    helm:
      valueFiles:
        - prod.yaml
  destination:
    name: prod
    namespace: default
  syncPolicy:
    syncOptions:
      - ApplyOutOfSyncOnly=true
      - CreateNamespace=true

argocd/project.yaml

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: pacman

spec:
  destinations:
    - name: '*'
      namespace: '*'
      server: '*'
  clusterResourceWhitelist:
    - group: '*'
      kind: '*'
  orphanedResources:
    warn: true
  sourceRepos:
    - https://github.com/you/config

helm/hash

dev dağıtımı için ayrılmıştır. Bu dosya, GitHub Actions "merge" iş akışı tarafından otomatik olarak güncellenir.


1234567

helm/tag

dev dağıtımı için ayrılmıştır. Bu dosya, GitHub Actions "release" iş akışı tarafından otomatik olarak güncellenir.


latest

helm/Chart.yaml

apiVersion: v2
name: pacman
version: 0.0.0

helm/dev.yaml

env:
  HTTP_ADDR: :8080

image:
  name: you/pacman
  tag: latest
  pull: Always

deployment:
  force: true
  replicas: 1
  container:
    name: go
    port: 8080

service:
  type: ClusterIP
  port: 8080

helm/sbox.yaml

env:
  HTTP_ADDR: :8080

image:
  name: you/pacman
  tag: ""
  pull: IfNotPresent

deployment:
  force: false
  replicas: 2
  container:
    name: go
    port: 8080

service:
  type: ClusterIP
  port: 8080

helm/prod.yaml

env:
  HTTP_ADDR: :8080

image:
  name: you/pacman
  tag: ""
  pull: IfNotPresent

deployment:
  force: false
  replicas: 2
  container:
    name: go
    port: 8080

service:
  type: ClusterIP
  port: 8080

helm/configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Chart.Name }}
  namespace: default

data:
  HTTP_ADDR: {{ .Values.env.HTTP_ADDR }}

helm/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: {{ .Chart.Name }}
  namespace: default

spec:
  type: {{ .Values.service.type }}
  selector:
    app: {{ .Chart.Name }}
  ports:
    - port: {{ .Values.service.port }}
      targetPort: {{ .Values.deployment.container.port }}

helm/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Chart.Name }}
  namespace: default
  labels:
    app: {{ .Chart.Name }}

spec:
  replicas: {{ .Values.deployment.replicas }}
  selector:
    matchLabels:
      app: {{ .Chart.Name }}
  template:
    metadata:
      labels:
        app: {{ .Chart.Name }}
      {{- if .Values.deployment.force }}
      annotations:
        roller: {{ .Files.Get "crds/vcs/hash" | trim }}
      {{- end }}
    spec:
      containers:
        - name: {{ .Values.deployment.container.name }}
          image: "{{ .Values.image.name }}:{{ .Values.image.tag | default (.Files.Get "crds/vcs/tag" | trim) }}"
          imagePullPolicy: {{ .Values.image.pull }}
          ports:
            - containerPort: {{ .Values.deployment.container.port }}
          envFrom:
            - configMapRef:
                name: {{ .Chart.Name }}

CI/CD ekran görüntüleri


AcroCD


Cluster


Repository


Project


Application


GitHub


Merge


Yapılandırma deposunda commit!



Release


Yapılandırma deposunda commit!



Docker


Etiket