17/03/2021 - DOCKER, GO
In this example we are going to use Docker to run the HashiCorp Vault server. After that, we will manually set it up so our Golang application connects to it. This is my personal suggestion. It is better to use HashiCorp Vault at application bootstrap for loading secrets once. This is because the interaction is a bit slow. If your application doesn't mind slow operations then that's fine.
├── cmd
│ └── bank
│ └── main.go
├── docker
│ ├── docker-compose.yaml
│ └── vault
│ ├── config
│ │ └── config.json
│ ├── data
│ ├── logs
│ └── policies
└── internal
└── vault
└── vault.go
version: "3.4"
services:
bank-vault:
container_name: "bank-vault"
image: "vault:1.6.3"
command: "server"
ports:
- "8200:8200"
cap_add:
- "IPC_LOCK"
environment:
VAULT_ADDR: "http://0.0.0.0:8200"
VAULT_API_ADDR: "http://0.0.0.0:8200"
SKIP_SETCAP: "true"
SKIP_CHOWN: "true"
volumes:
- "./vault/config:/vault/config"
- "./vault/data:/vault/data"
- "./vault/logs:/vault/logs"
- "./vault/policies:/vault/policies"
{
"storage": {
"file": {
"path": "vault/data"
}
},
"listener": {
"tcp": {
"address": "0.0.0.0:8200",
"tls_disable": true
}
},
"ui": true,
"max_lease_ttl": "8760h",
"default_lease_ttl": "8760h",
"disable_mlock": true
}
package vault
import (
"encoding/json"
"fmt"
"log"
"github.com/hashicorp/vault/api"
)
// API v2 specific path. Exclude /data for API v1.
const base = "/secret/data"
type Config struct {
Token string
Address string
Path string
Debug bool
}
type Vault struct {
client *api.Client
config Config
}
func New(config Config) (Vault, error) {
client, err := api.NewClient(&api.Config{Address: config.Address})
if err != nil {
return Vault{}, err
}
client.SetToken(config.Token)
return Vault{client: client, config: config}, nil
}
// Write upserts a new secret to a path.
func (v Vault) Write(key string, value map[string]interface{}) error {
scr, err := v.client.Logical().Write(
fmt.Sprintf("%s%s/%s", base, v.config.Path, key),
map[string]interface{}{"data": value},
)
if err != nil {
return fmt.Errorf("write: %w", err)
}
if v.config.Debug {
dat, err := json.Marshal(scr)
if err != nil {
return fmt.Errorf("debug: %w", err)
}
log.Println(string(dat))
}
return nil
}
// Read retrieves the most recent version of an existing secret from the path.
func (v Vault) Read(key string) (map[string]interface{}, error) {
scr, err := v.client.Logical().Read(fmt.Sprintf("%s%s/%s", base, v.config.Path, key))
if err != nil {
return nil, fmt.Errorf("read: %w", err)
}
if scr == nil {
return nil, fmt.Errorf("path: not found")
}
if v.config.Debug {
dat, err := json.Marshal(scr)
if err != nil {
return nil, fmt.Errorf("debug: %w", err)
}
log.Println(string(dat))
}
dat, ok := scr.Data["data"]
if !ok {
return nil, fmt.Errorf("secret: not found")
}
return dat.(map[string]interface{}), nil
}
package main
import (
"log"
"github.com/you/client/internal/vault"
)
func main() {
vault, err := vault.New(vault.Config{
Token: "s.fg2cEykAegqOiV1euwuQCGAz",
Address: "http://0.0.0.0:8200",
Path: "/bank/dev", // /{application}/{environment}
Debug: false,
})
if err != nil {
log.Fatal(err)
}
// EXAMPLE 1 (single)
// Write
if err := vault.Write("TOKEN", map[string]interface{}{"token": "XYZabc000"}); err != nil {
log.Fatal(err)
}
// Retrieve
token, err := vault.Read("TOKEN")
if err != nil {
log.Fatal(err)
}
log.Println("TOKEN:", token)
// EXAMPLE 2 (multiple)
// Write
if err := vault.Write("CLIENT", map[string]interface{}{"id": "inanzzz", "secret": "123123"}); err != nil {
log.Fatal(err)
}
// Retrieve
client, err := vault.Read("CLIENT")
if err != nil {
log.Fatal(err)
}
log.Println("CLIENT:", client)
}
When you run docker, the output will look like below. You can then access UI via http://0.0.0.0:8200/ui address if you wish. Note: UI is not for production so you should disable it.
bank-vault | ==> Vault server configuration:
bank-vault |
bank-vault | Api Address: http://0.0.0.0:8200
bank-vault | Cgo: disabled
bank-vault | Cluster Address: https://0.0.0.0:8201
bank-vault | Go Version: go1.15.7
bank-vault | Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
bank-vault | Log Level: info
bank-vault | Mlock: supported: true, enabled: false
bank-vault | Recovery Mode: false
bank-vault | Storage: file
bank-vault | Version: Vault v1.6.3
bank-vault | Version Sha: b540be4b7ec48d0dd7512c8d8df9399d6bf84d76
bank-vault |
bank-vault | ==> Vault server started! Log data will stream in below:
bank-vault |
bank-vault | 2021-03-16T14:36:52.079Z [INFO] proxy environment: http_proxy= https_proxy= no_proxy=
The token below will be used for HTTP requests.
/ # vault operator init
Unseal Key 1: J/PkYie13YkYEUDzZRsijjqJ1NaxsOQRexEuQ8hMkDQB
Unseal Key 2: ocsR3Yd0UHNklQgQULJl3qlQ+OcekBcZDBJFU3e6dW74
Unseal Key 3: otOySb77wUs78wrVWELQG/EQ2NzWC9TPgPLK3Aqsobug
Unseal Key 4: o0KdstrE+R0D7eFbw0v0r1MnvuoG7VVgq4IiJ8zRQgiB
Unseal Key 5: vDIxuDcI2CVt9OvavaO6ewMkt4ZtTsEpmT+ZwvoF//kP
Initial Root Token: s.fg2cEykAegqOiV1euwuQCGAz
Run command below three times with three different unseal keys as listed above.
/ # vault operator unseal
Unseal Key (will be hidden):
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.6.3
Storage Type file
Cluster Name vault-cluster-b31c4221
Cluster ID 1ac98691-a94b-0d34-ddbb-0084cf0faef1
HA Enabled false
Use "Initial Root Token" to login.
/ # vault login
Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token s.fg2cEykAegqOiV1euwuQCGAz
token_accessor 107eqsoOahYMrfFncIa27jCQ
token_duration ∞
token_renewable false
token_policies ["root"]
identity_policies []
policies ["root"]
This is the engine where our secrets will be stored. We are using V2 here.
/ # vault secrets enable -path=secret -version=2 -description="application secret storage" kv
Success! Enabled the kv secrets engine at: secret/
/ # vault secrets list -detailed
Path Type Accessor Description
---- ---- -------- -----------
cubbyhole/ cubbyhole cubbyhole_cf0c91e4 per-token private secret storage
identity/ identity identity_03475516 identity store
secret/ kv kv_53d2a365 application secret storage
sys/ system system_bc07ceb9 system endpoints used for control, policy and debugging
Obviously you will normally do more than this in order to fine-tune the setup but I am only showing you the necessary ones to get us going. For more details check out my previous post Managing application secrets with Hashicorp Vault.
$ go run -race cmd/bank/main.go
2021/03/17 16:10:32 TOKEN: map[token:XYZabc000]
2021/03/17 16:10:32 CLIENT: map[id:inanzzz secret:123123]