External Secrets Operator¶
External Secrets Operator (ESO) bridges
OpenBao to native Kubernetes Secret objects. Workloads consume
secrets the standard way — envFrom, volumeMounts, imagePullSecrets —
with no sidecar, no init container and no code that knows what a Vault token
is.
At a glance¶
| Namespace | external-secrets |
| Depends on | OpenBao at runtime, though not to be installed |
| If it is down | Secrets already materialised keep working. Nothing rotates, and nothing new resolves |
| Health check | kubectl get clustersecretstore → Valid on all three |
| Files | payload/platform/external-secrets/; the stores in payload/platform/openbao/cluster-secret-store.yaml |
Configuration¶
sequenceDiagram
participant App as Workload
participant ES as ExternalSecret
participant ESO as ESO Controller
participant Bao as OpenBao
participant K as K8s Secret
ESO->>Bao: login (k8s SA JWT)
Bao-->>ESO: vault token
ESO->>Bao: read kv/path
Bao-->>ESO: secret data
ESO->>K: create/update Secret
App->>K: mount / envFrom
ESO mints a short-lived token for an external-secrets-vault ServiceAccount
through the TokenRequest API, which the chart's controller ClusterRole allows
in every namespace (cluster-secret-store.yaml binds its own
external-secrets-vault-token-creator ClusterRole, scoped to that
ServiceAccount name, beside it); OpenBao validates it against the TokenReview API
and issues a token bound to that store's policy. Platform namespaces use the
openbao store and the ServiceAccount in external-secrets; a workload uses
openbao-apps and its own, which confines it to kv/<namespace>/ — see
Kubernetes auth. No long-lived credential exists
anywhere. The stores use ESO's vault provider unchanged against the
in-cluster Service.
In application.yaml's valuesObject:
| Setting | Why |
|---|---|
No ClusterSecretStore here |
A store only validates against a running, unsealed OpenBao, so it ships with openbao, whose readiness it reports — see Bootstrap convergence |
| Webhook ServiceAccount token stays mounted | The webhook has no RoleBinding but builds an in-cluster client at startup and exits without the token (unable to load in-cluster config), which only shows when the pod is recreated |
| Two webhook replicas, a PodDisruptionBudget and a hostname spread | The webhook's failurePolicy is Fail, so while a single replica reschedules through a Kured drain every ExternalSecret, PushSecret and store apply fails and the Argo sync that carried it goes to retry — see Replica placement |
Usage¶
Adding a secret¶
-
Store the value in OpenBao, following the KV layout. A random value is not typed in at all: a
Passwordgenerator and aPushSecretwrite it, see Generated secrets. -
Declare an
ExternalSecretalongside the workload's other manifests; the parent Application picks it up. A workload namesopenbao-appsand ships the ServiceAccount that store logs in as; a platform component namesopenbaoinstead and needs no ServiceAccount:--- apiVersion: v1 kind: ServiceAccount metadata: name: external-secrets-vault namespace: my-app annotations: argocd.argoproj.io/sync-wave: "-2" automountServiceAccountToken: false --- apiVersion: external-secrets.io/v1 kind: ExternalSecret metadata: name: my-app-credentials namespace: my-app annotations: # ESO's CRDs arrive with the external-secrets Application; this # covers a sync before they have landed. argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true spec: refreshInterval: 1h secretStoreRef: name: openbao-apps kind: ClusterSecretStore target: name: my-app-credentials # name of the resulting K8s Secret creationPolicy: Owner data: - secretKey: key1 # key in the K8s Secret remoteRef: key: my-app/<purpose> # the namespace is the prefix property: key1 # key inside the KV entryTo pull every key without listing them, use
dataFrominstead ofdata: -
Consume the Secret:
refreshInterval polls OpenBao; 0 disables polling and re-syncs only on
resource changes. To sync now rather than wait:
Health check¶
Start with the store; if it is not Ready, nothing downstream is either.
openbao-apps and openbao-claude-agents report Valid without logging in,
so their auth failures show on the ExternalSecret instead:
| Cause | Fix |
|---|---|
| OpenBao is sealed | Unseal it |
| Kubernetes auth role missing | make bao-policies writes all three roles; it needs a root token |
Wrong serviceAccountRef |
external-secrets-vault, in external-secrets for openbao and without a namespace for the other two |
Then the individual object:
| Cause | Fix |
|---|---|
The KV path does not exist (bao kv get kv/<path> returns 404) |
Store the secret first |
property: matches no key in the KV entry |
bao kv get kv/<path> lists them |
permission denied from openbao-apps |
The path is outside kv/<namespace>/, or the namespace has no external-secrets-vault ServiceAccount |
using cluster store ... is not allowed from namespace |
The store's conditions exclude the namespace: a workload on openbao, or a platform namespace on openbao-apps |
Pitfalls¶
Rotation does not restart anything
A rotated value reaches the Secret within refreshInterval, but a pod that read it into an environment variable keeps the old one until something restarts it. A rollout, or Reloader-style tooling, is the other half of rotation.