»Vault Agent Injector Examples

The following are different configuration examples to support a variety of deployment models.

»Before Using the Vault Agent Injector

Before applying Vault Agent injection annotations to pods, the following requirements should be satisfied.

»Connectivity

  • the Kubernetes API can connect to the Vault Agent injector service on port 443, and the injector can connect to the Kubernetes API,
  • Vault can connect to the Kubernetes API,
  • Pods in the Kubernetes cluster can connect to Vault.

»Kubernetes and Vault Configuration

  • Kubernetes auth method should be configured and enabled in Vault,
  • Pod should have a service account,
  • desired secrets exist within Vault,
  • the service account should be bound to a Vault role with a policy enabling access to desired secrets.

For more information on configuring the Vault Kubernetes auth method, see the official documentation.

»Debugging

If an error occurs with a mutation request, Kubernetes will attach the error to the owner of the pod. Check the following for errors:

  • If the pod was created by a deployment or statefulset, check for errors in the replicaset that owns the pod.
  • If the pod was created by a job, check the job for errors.

»Patching Existing Pods

To patch existing pods, a Kubernetes patch can be applied to add the required annotations to pods. When applying a patch, the pods will be rescheduled.

First, create the patch:

cat <<EOF >> ./patch.yaml
spec:
  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/agent-inject-status: "update"
        vault.hashicorp.com/agent-inject-secret-db-creds: "database/creds/db-app"
        vault.hashicorp.com/agent-inject-template-db-creds: |
          {{- with secret "database/creds/db-app" -}}
          postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/appdb?sslmode=disable
          {{- end }}
        vault.hashicorp.com/role: "db-app"
        vault.hashicorp.com/ca-cert: "/vault/tls/ca.crt"
        vault.hashicorp.com/client-cert: "/vault/tls/client.crt"
        vault.hashicorp.com/client-key: "/vault/tls/client.key"
        vault.hashicorp.com/tls-secret: "vault-tls-client"
EOF
cat <<EOF >> ./patch.yamlspec:  template:    metadata:      annotations:        vault.hashicorp.com/agent-inject: "true"        vault.hashicorp.com/agent-inject-status: "update"        vault.hashicorp.com/agent-inject-secret-db-creds: "database/creds/db-app"        vault.hashicorp.com/agent-inject-template-db-creds: |          {{- with secret "database/creds/db-app" -}}          postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/appdb?sslmode=disable          {{- end }}        vault.hashicorp.com/role: "db-app"        vault.hashicorp.com/ca-cert: "/vault/tls/ca.crt"        vault.hashicorp.com/client-cert: "/vault/tls/client.crt"        vault.hashicorp.com/client-key: "/vault/tls/client.key"        vault.hashicorp.com/tls-secret: "vault-tls-client"EOF

Next, apply the patch:

kubectl patch deployment <MY DEPLOYMENT> --patch "$(cat patch.yaml)"
kubectl patch deployment <MY DEPLOYMENT> --patch "$(cat patch.yaml)"

The pod should now be rescheduled with additional containers. The pod can be inspected using the kubectl describe command:

kubectl describe pod <name of pod>
kubectl describe pod <name of pod>

»Deployments, StatefulSets, etc.

The annotations for configuring Vault Agent injection must be on the pod specification. Since higher level resources such as Deployments wrap pod specification templates, Vault Agent Injector can be used with all of these higher level constructs, too.

An example Deployment below shows how to enable Vault Agent injection:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-example
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-example-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app-example
  template:
    metadata:
      labels:
        app: app-example
      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/agent-inject-secret-db-creds: 'database/creds/db-app'
        vault.hashicorp.com/agent-inject-template-db-creds: |
          {{- with secret "database/creds/db-app" -}}
          postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/appdb?sslmode=disable
          {{- end }}
        vault.hashicorp.com/role: 'db-app'
        vault.hashicorp.com/ca-cert: '/vault/tls/ca.crt'
        vault.hashicorp.com/client-cert: '/vault/tls/client.crt'
        vault.hashicorp.com/client-key: '/vault/tls/client.key'
        vault.hashicorp.com/tls-secret: 'vault-tls-client'
    spec:
      containers:
        - name: app
          image: 'app:1.0.0'
      serviceAccountName: app-example
---apiVersion: v1kind: ServiceAccountmetadata:  name: app-example---apiVersion: apps/v1kind: Deploymentmetadata:  name: app-example-deploymentspec:  replicas: 1  selector:    matchLabels:      app: app-example  template:    metadata:      labels:        app: app-example      annotations:        vault.hashicorp.com/agent-inject: 'true'        vault.hashicorp.com/agent-inject-secret-db-creds: 'database/creds/db-app'        vault.hashicorp.com/agent-inject-template-db-creds: |          {{- with secret "database/creds/db-app" -}}          postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/appdb?sslmode=disable          {{- end }}        vault.hashicorp.com/role: 'db-app'        vault.hashicorp.com/ca-cert: '/vault/tls/ca.crt'        vault.hashicorp.com/client-cert: '/vault/tls/client.crt'        vault.hashicorp.com/client-key: '/vault/tls/client.key'        vault.hashicorp.com/tls-secret: 'vault-tls-client'    spec:      containers:        - name: app          image: 'app:1.0.0'      serviceAccountName: app-example

»ConfigMap Example

The following example creates a deployment that mounts a Kubernetes ConfigMap containing Vault Agent configuration files. For a complete list of the Vault Agent configuration settings, see the Agent documentation.

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-example
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-example-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app-example
  template:
    metadata:
      labels:
        app: app-example
      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/agent-configmap: 'my-configmap'
        vault.hashicorp.com/tls-secret: 'vault-tls-client'
    spec:
      containers:
        - name: app
          image: 'app:1.0.0'
      serviceAccountName: app-example
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-configmap
data:
  config.hcl: |
    "auto_auth" = {
      "method" = {
        "config" = {
          "role" = "db-app"
        }
        "type" = "kubernetes"
      }

      "sink" = {
        "config" = {
          "path" = "/home/vault/.token"
        }

        "type" = "file"
      }
    }

    "exit_after_auth" = false
    "pid_file" = "/home/vault/.pid"

    "template" = {
      "contents" = "{{- with secret \"database/creds/db-app\" -}}postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/mydb?sslmode=disable{{- end }}"
      "destination" = "/vault/secrets/db-creds"
    }

    "vault" = {
      "address" = "https://vault.demo.svc.cluster.local:8200"
      "ca_cert" = "/vault/tls/ca.crt"
      "client_cert" = "/vault/tls/client.crt"
      "client_key" = "/vault/tls/client.key"
    }
  config-init.hcl: |
    "auto_auth" = {
      "method" = {
        "config" = {
          "role" = "db-app"
        }
        "type" = "kubernetes"
      }

      "sink" = {
        "config" = {
          "path" = "/home/vault/.token"
        }

        "type" = "file"
      }
    }

    "exit_after_auth" = true
    "pid_file" = "/home/vault/.pid"

    "template" = {
      "contents" = "{{- with secret \"database/creds/db-app\" -}}postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/mydb?sslmode=disable{{- end }}"
      "destination" = "/vault/secrets/db-creds"
    }

    "vault" = {
      "address" = "https://vault.demo.svc.cluster.local:8200"
      "ca_cert" = "/vault/tls/ca.crt"
      "client_cert" = "/vault/tls/client.crt"
      "client_key" = "/vault/tls/client.key"
    }
---apiVersion: v1kind: ServiceAccountmetadata:  name: app-example---apiVersion: apps/v1kind: Deploymentmetadata:  name: app-example-deploymentspec:  replicas: 1  selector:    matchLabels:      app: app-example  template:    metadata:      labels:        app: app-example      annotations:        vault.hashicorp.com/agent-inject: 'true'        vault.hashicorp.com/agent-configmap: 'my-configmap'        vault.hashicorp.com/tls-secret: 'vault-tls-client'    spec:      containers:        - name: app          image: 'app:1.0.0'      serviceAccountName: app-example---apiVersion: v1kind: ConfigMapmetadata:  name: my-configmapdata:  config.hcl: |    "auto_auth" = {      "method" = {        "config" = {          "role" = "db-app"        }        "type" = "kubernetes"      }
      "sink" = {        "config" = {          "path" = "/home/vault/.token"        }
        "type" = "file"      }    }
    "exit_after_auth" = false    "pid_file" = "/home/vault/.pid"
    "template" = {      "contents" = "{{- with secret \"database/creds/db-app\" -}}postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/mydb?sslmode=disable{{- end }}"      "destination" = "/vault/secrets/db-creds"    }
    "vault" = {      "address" = "https://vault.demo.svc.cluster.local:8200"      "ca_cert" = "/vault/tls/ca.crt"      "client_cert" = "/vault/tls/client.crt"      "client_key" = "/vault/tls/client.key"    }  config-init.hcl: |    "auto_auth" = {      "method" = {        "config" = {          "role" = "db-app"        }        "type" = "kubernetes"      }
      "sink" = {        "config" = {          "path" = "/home/vault/.token"        }
        "type" = "file"      }    }
    "exit_after_auth" = true    "pid_file" = "/home/vault/.pid"
    "template" = {      "contents" = "{{- with secret \"database/creds/db-app\" -}}postgres://{{ .Data.username }}:{{ .Data.password }}@postgres:5432/mydb?sslmode=disable{{- end }}"      "destination" = "/vault/secrets/db-creds"    }
    "vault" = {      "address" = "https://vault.demo.svc.cluster.local:8200"      "ca_cert" = "/vault/tls/ca.crt"      "client_cert" = "/vault/tls/client.crt"      "client_key" = "/vault/tls/client.key"    }

»Environment Variable Example

The following example demonstrates how templates can be used to create environment variables. A template should be created that exports a Vault secret as an environment variable and the application container should source those files during startup.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
  labels:
    app: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/role: 'web'
        vault.hashicorp.com/agent-inject-secret-config: 'secret/data/web'
        # Environment variable export template
        vault.hashicorp.com/agent-inject-template-config: |
          {{ with secret "secret/data/web" -}}
            export api_key="{{ .Data.data.payments_api_key }}"
          {{- end }}
    spec:
      serviceAccountName: web
      containers:
        - name: web
          image: alpine:latest
          args:
            ['sh', '-c', 'source /vault/secrets/config && <entrypoint script>']
          ports:
            - containerPort: 9090
---apiVersion: apps/v1kind: Deploymentmetadata:  name: web-deployment  labels:    app: webspec:  replicas: 1  selector:    matchLabels:      app: web  template:    metadata:      labels:        app: web      annotations:        vault.hashicorp.com/agent-inject: 'true'        vault.hashicorp.com/role: 'web'        vault.hashicorp.com/agent-inject-secret-config: 'secret/data/web'        # Environment variable export template        vault.hashicorp.com/agent-inject-template-config: |          {{ with secret "secret/data/web" -}}            export api_key="{{ .Data.data.payments_api_key }}"          {{- end }}    spec:      serviceAccountName: web      containers:        - name: web          image: alpine:latest          args:            ['sh', '-c', 'source /vault/secrets/config && <entrypoint script>']          ports:            - containerPort: 9090

»AppRole Authentication

The following example demonstrates how the AppRole authentication method can be used by Vault Agent for retrieving secrets. A Kubernetes secret containing the AppRole secret ID and role ID should be created first.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
  labels:
    app: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/agent-extra-secret: 'approle-example'
        vault.hashicorp.com/auth-type: 'approle'
        vault.hashicorp.com/auth-path: 'auth/approle'
        vault.hashicorp.com/auth-config-role-id-file-path: '/vault/custom/role-id'
        vault.hashicorp.com/auth-config-secret-id-file-path: '/vault/custom/secret-id'
        vault.hashicorp.com/agent-inject-secret-db-creds: 'database/creds/db-app'
        vault.hashicorp.com/agent-inject-template-db-creds: |
          {{- with secret "database/creds/db-app" -}}
          postgres://{{ .Data.username }}:{{ .Data.password }}@postgres.postgres.svc:5432/wizard?sslmode=disable
          {{- end }}
        vault.hashicorp.com/role: 'my-role'
        vault.hashicorp.com/tls-secret: 'vault-tls'
        vault.hashicorp.com/ca-cert: '/vault/tls/ca.crt'
    spec:
      serviceAccountName: web
      containers:
        - name: web
          image: alpine:latest
          args:
            ['sh', '-c', 'source /vault/secrets/config && <entrypoint script>']
          ports:
            - containerPort: 9090
---apiVersion: apps/v1kind: Deploymentmetadata:  name: web-deployment  labels:    app: webspec:  replicas: 1  selector:    matchLabels:      app: web  template:    metadata:      labels:        app: web      annotations:        vault.hashicorp.com/agent-inject: 'true'        vault.hashicorp.com/agent-extra-secret: 'approle-example'        vault.hashicorp.com/auth-type: 'approle'        vault.hashicorp.com/auth-path: 'auth/approle'        vault.hashicorp.com/auth-config-role-id-file-path: '/vault/custom/role-id'        vault.hashicorp.com/auth-config-secret-id-file-path: '/vault/custom/secret-id'        vault.hashicorp.com/agent-inject-secret-db-creds: 'database/creds/db-app'        vault.hashicorp.com/agent-inject-template-db-creds: |          {{- with secret "database/creds/db-app" -}}          postgres://{{ .Data.username }}:{{ .Data.password }}@postgres.postgres.svc:5432/wizard?sslmode=disable          {{- end }}        vault.hashicorp.com/role: 'my-role'        vault.hashicorp.com/tls-secret: 'vault-tls'        vault.hashicorp.com/ca-cert: '/vault/tls/ca.crt'    spec:      serviceAccountName: web      containers:        - name: web          image: alpine:latest          args:            ['sh', '-c', 'source /vault/secrets/config && <entrypoint script>']          ports:            - containerPort: 9090