26/11/2020 - KUBERNETES
In this example we are going to mount secret and non-secret config files into Pods. For the secrets, there are two ways we are going to use. Using secret.yaml file and manually creating the secrets in the Kubernetes cluster. The problem with the first option is that, if you commit secret.yaml file, you expose your secrets so it is safer to with the manual option. Just make sure that the secrets are already in Kubernetes otherwise Pod will crash.
We are going to mount:
Files under secret folder and .env file are not committed because they have secrets for development environment.
├── Makefile
├── config
│ └── finders.yaml
├── deploy
│ └── k8s
│ ├── configmap.yaml
│ ├── deployment.yaml
│ └── secret.yaml
├── docker
│ └── dev
│ └── Dockerfile
├── .env
├── .env.dist
├── .gitignore
├── main.go
└── secret
├── credentials.conf
└── keys.yaml
## Build application binary.
.PHONY: build
build:
go build -race -ldflags "-s -w" -o bin/main main.go
## Build application binary and run it.
.PHONY: run
run: build
bin/main
## -----------------------------------------------------------------------------------
## Build, tag and push application image to registry then clean up.
.PHONY: push
push:
docker build -t you/address-finder:latest -f docker/dev/Dockerfile .
docker push you/address-finder:latest
docker rmi you/address-finder:latest
docker system prune --volumes --force
## Deploy application to kubernetes cluster.
.PHONY: deploy
deploy:
# This secret command is not needed for manual secret interactions (preferred).
kubectl apply -f deploy/k8s/secret.yaml
kubectl apply -f deploy/k8s/configmap.yaml
kubectl apply -f deploy/k8s/deployment.yaml
ENV_VAR_X=x_secret_from_repo_non_dist
ENV_VAR_Y=y_secret_from_repo_non_dist
ENV_VAR_Z=z_non_secret_from_repo_non_dist
ENV_VAR_X=x_secret_from_repo_dist
ENV_VAR_Y=y_secret_from_repo_dist
ENV_VAR_Z=z_non_secret_from_repo_dist
secret/
.env
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"time"
"github.com/joho/godotenv"
)
func main() {
fmt.Println("--- Secret credentials file ---------------------")
scrConf, err := ioutil.ReadFile("secret/credentials.conf")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(scrConf))
fmt.Println("--- Secret keys file ----------------------------")
kysConf, err := ioutil.ReadFile("secret/keys.yaml")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(kysConf))
fmt.Println("--- Public finders file -------------------------")
finConf, err := ioutil.ReadFile("config/finders.yaml")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(finConf))
fmt.Println("--- System environment variables ----------------")
_ = godotenv.Load()
fmt.Println(value("ENV_VAR_X"))
fmt.Println(value("ENV_VAR_Y"))
fmt.Println(value("ENV_VAR_Z"))
time.Sleep(time.Hour)
}
func value(key string) string {
v, ok := os.LookupEnv(key)
if !ok {
log.Fatalf("the environment var %s was not found", key)
}
return v
}
#
# STAGE 1: build
#
FROM golang:1.15-alpine3.12 as build
WORKDIR /source
COPY . .
RUN CGO_ENABLED=0 go build -ldflags "-s -w" -o bin/main main.go
#
# STAGE 2: run
#
FROM alpine:3.12 as run
COPY --from=build /source/bin/main /main
ENTRYPOINT ["/main"]
finders:
- name: postcode
api: https://api.postcodes.io/postcodes
method: GET
- name: ip
api: http://ip-api.com/json
method: GET
title: this is credentials.conf secret content (local)
line_1: hello (local)
line_2: world (local)
KEY_1: key 1 (local)
KEY_2: key 2 (local)
apiVersion: v1
kind: ConfigMap
metadata:
name: address-finder-configmap
data:
finders.yaml: |
finders:
- name: address
api: https://api.getAddress.io/find
method: GET
- name: postcode
api: https://api.postcodes.io/postcodes
method: GET
- name: ip
api: http://ip-api.com/json
method: GET
apiVersion: apps/v1
kind: Deployment
metadata:
name: address-finder-deployment
labels:
app: address-finder
spec:
replicas: 1
selector:
matchLabels:
app: address-finder
template:
metadata:
labels:
app: address-finder
spec:
containers:
- name: golang
image: you/address-finder:latest
volumeMounts:
# Refers to a single file
- name: config
mountPath: ./config/finders.yaml
subPath: finders.yaml
readOnly: true
# Refers to all the secret files in a folder
- name: secret
mountPath: ./secret
readOnly: true
# Refers to single secret file to app root
- name: dotenv
mountPath: ./.env
subPath: .env # Without this, .env will be created as a folder, not a file!
readOnly: true
volumes:
- name: config
configMap:
name: address-finder-configmap
- name: secret
secret:
secretName: address-finder-secret
- name: dotenv
secret:
secretName: address-finder-secret
items:
- key: .env
path: ./.env
Remember this not preferred way of handling secrets. You will delete this file and manually create address-finder-secret component with kubectl command in Kubernetes cluster in options 2. What you are seeing here is the option 1.
apiVersion: v1
kind: Secret
metadata:
name: address-finder-secret
type: Opaque
data:
credentials.conf: | # cat secret/credentials.conf | base64
dGl0bGU6IHRoaXMgaXMgY3JlZGVudGlhbHMuY29uZiBzZWNyZXQgY29udGVudCAobG9jYWwpCmxpbmVfMTogaGVsbG8gKGxvY2FsKQoKbGluZV8yOiB3b3JsZCAobG9jYWwpCg==
keys.yaml: | # cat secret/keys.yaml | base64
S0VZXzE6IGtleSAxIChsb2NhbCkKS0VZXzI6IGtleSAyIChsb2NhbCkK
.env: | # cat .env | base64
RU5WX1ZBUl9YPXhfc2VjcmV0X2Zyb21fcmVwb19ub25fZGlzdApFTlZfVkFSX1k9eV9zZWNyZXRfZnJvbV9yZXBvX25vbl9kaXN0CkVOVl9WQVJfWj16X25vbl9zZWNyZXRfZnJvbV9yZXBvX25vbl9kaXN0Cg==
This is what terminal output look like if we have run the application in local environment. As you can see the content matches the files we listed above.
$ make run
go build -race -ldflags "-s -w" -o bin/main main.go
bin/main
--- Secret credentials file ---------------------
title: this is credentials.conf secret content (local)
line_1: hello (local)
line_2: world (local)
--- Secret keys file ----------------------------
KEY_1: key 1 (local)
KEY_2: key 2 (local)
--- Public finders file -------------------------
finders:
- name: postcode
api: https://api.postcodes.io/postcodes
method: GET
- name: ip
api: http://ip-api.com/json
method: GET
--- System environment variables ----------------
x_secret_from_repo_non_dist
y_secret_from_repo_non_dist
z_non_secret_from_repo_non_dist
This is where we keep secret.yaml in repository and rely on it to create secrets automatically. Not preferred! Deploy the application below.
$ make deploy
# This secret command is not needed for manual secret interactions (preferred).
kubectl apply -f deploy/k8s/secret.yaml
secret/address-finder-secret created
kubectl apply -f deploy/k8s/configmap.yaml
configmap/address-finder-configmap created
kubectl apply -f deploy/k8s/deployment.yaml
deployment.apps/address-finder-deployment created
Check application logs below.
$ kubectl logs pod/address-finder-deployment-7f6d6959db-w57gn
--- Secret credentials file ---------------------
title: this is credentials.conf secret content (local)
line_1: hello (local)
line_2: world (local)
--- Secret keys file ----------------------------
KEY_1: key 1 (local)
KEY_2: key 2 (local)
--- Public finders file -------------------------
finders:
- name: address
api: https://api.getAddress.io/find
method: GET
- name: postcode
api: https://api.postcodes.io/postcodes
method: GET
- name: ip
api: http://ip-api.com/json
method: GET
--- System environment variables ----------------
x_secret_from_repo_non_dist
y_secret_from_repo_non_dist
z_non_secret_from_repo_non_dist
As you can see above, secrets have been decoded (they are encoded as local copy) and mounted in the Pod. Also the finders config is coming from the configmap.yaml file instead of the application repository. Environment variables are coming from the .env file not from the environment because we didn't create them as normal env variables. Let's check the Pod content.
$ kubectl exec -it pod/address-finder-deployment-7f6d6959db-w57gn sh
/ # ls -la
total 1816
-rw-r--r-- 1 root root 118 Nov 26 19:26 .env
drwxr-xr-x 2 root root 4096 Nov 26 19:26 config
drwxrwxrwt 3 root root 140 Nov 26 19:26 secret
...
/ # cat .env
ENV_VAR_X=x_secret_from_repo_non_dist
ENV_VAR_Y=y_secret_from_repo_non_dist
ENV_VAR_Z=z_non_secret_from_repo_non_dist
/ # ls -l config/
total 4
-rw-r--r-- 1 root root 223 Nov 26 19:26 finders.yaml
/ # cat config/finders.yaml
finders:
- name: address
api: https://api.getAddress.io/find
method: GET
- name: postcode
api: https://api.postcodes.io/postcodes
method: GET
- name: ip
api: http://ip-api.com/json
method: GET
/ # ls -l secret/
total 0
lrwxrwxrwx 1 root root 23 Nov 26 19:26 credentials.conf -> ..data/credentials.conf
lrwxrwxrwx 1 root root 16 Nov 26 19:26 keys.yaml -> ..data/keys.yaml
/ # cat secret/credentials.conf
title: this is credentials.conf secret content (local)
line_1: hello (local)
line_2: world (local)
/ # cat secret/keys.yaml
KEY_1: key 1 (local)
KEY_2: key 2 (local)
Let's check what secrets components look like. We will manually create this later in option 2.
$ kubectl get secrets
NAME TYPE DATA AGE
address-finder-secret Opaque 3 10m
$ kubectl describe secret address-finder-secret
Name: address-finder-secret
Namespace: default
Labels:
Annotations:
Type: Opaque
Data
====
keys.yaml: 42 bytes
.env: 118 bytes
credentials.conf: 100 bytes
This is where we don't have secret.yaml at all and rely on manually creating the secrets in Kubernetes before deployment - preferred! Here we manually create the secret files. I already have them so just showing the contents.
$ cat ./Desktop/credentials.conf
title: this is credentials.conf secret content (desktop)
line_1: hello (desktop)
line_2: world (dektop)
$ cat ./Desktop/keys.yaml
KEY_1: key 1 (desktop)
KEY_2: key 2 (desktop)
$ cat ./Desktop/.env
ENV_VAR_X=x_secret_from_desktop
ENV_VAR_Y=y_secret_from_desktop
ENV_VAR_Z=z_non_secret_from_desktop
Let's turn these files into Kubernetes secret component.
$ kubectl create secret generic address-finder-secret --save-config \
> --from-file=./Desktop/credentials.conf \
> --from-file=./Desktop/keys.yaml \
> --from-file=./Desktop/.env
As you can see below, this command matches what secret.yaml created previously in option 1.
$ kubectl get secrets
NAME TYPE DATA AGE
address-finder-secret Opaque 3 21s
$ kubectl describe secret address-finder-secret
Name: address-finder-secret
Namespace: default
Labels:
Annotations:
Type: Opaque
Data
====
.env: 101 bytes
credentials.conf: 105 bytes
keys.yaml: 47 bytes
Let's deploy the app now. I removed the secret.yaml command from the make command.
$ make deploy
kubectl apply -f deploy/k8s/configmap.yaml
configmap/address-finder-configmap created
kubectl apply -f deploy/k8s/deployment.yaml
Let's see application log/output. Focus on the secrets.
$ kubectl logs pod/address-finder-deployment-7f6d6959db-f4rcr
--- Secret credentials file ---------------------
title: this is credentials.conf secret content (desktop)
line_1: hello (desktop)
line_2: world (dektop)
--- Secret keys file ----------------------------
KEY_1: key 1 (desktop)
KEY_2: key 2 (desktop)
--- Public finders file -------------------------
finders:
- name: address
api: https://api.getAddress.io/find
method: GET
- name: postcode
api: https://api.postcodes.io/postcodes
method: GET
- name: ip
api: http://ip-api.com/json
method: GET
--- System environment variables ----------------
x_secret_from_desktop
y_secret_from_desktop
z_non_secret_from_desktop
As you can see above, it worked fine.
If you wish to update the secrets later on, you can use kubectl edit secret address-finder-secret command. Just remember it requires base64 encoded data.
You can use command below for base64 encoding/decoding.
// encode
$ cat {file-path} | base64
// decode
$ echo -n {plain-secret-goes-here} | base64