Security Policies¶

Four cluster-wide controls: Pod Security Admission, default-deny ingress network policy, no API token on the default ServiceAccount, and two admission policies on image references. Each is easy to turn on and hard to turn on safely, so the pattern is the same for all of them: measure first, enforce second, one namespace at a time.

At a glance¶

Configuration¶

Pod Security Admission¶

A namespace that does not opt in runs at privileged, which enforces nothing. Each namespace in pod-security.yaml carries three labels:

Enforcement is set to what already works, so nothing running breaks, while warn and audit show what a stricter level would catch; tightening later is an informed change.

privileged here means "not yet reduced", not "unexamined": audit and warn are still baseline or restricted on all of them. Namespace objects are owned by this Application so the labels stay declarative rather than drifting after CreateNamespace=true.

Network policies¶

Ingress only, in ten namespaces, as CiliumNetworkPolicy in network-policies.yaml. Every policy also admits traffic from within the namespace and from host and remote-node for probes.

Why CiliumNetworkPolicy and not NetworkPolicy¶

Two kinds of traffic have no pod identity a standard NetworkPolicy can name: Gateway traffic, which reaches a backend from the per-node Envoy, and kubelet health probes, which come from the node. A plain default-deny therefore kills ingress and probes, and the pods restart forever in a way that looks like an application fault. Cilium's fromEntities names them: ingress for Envoy-proxied traffic, host and remote-node for the kubelet.

Scope¶

The monitoring callers are easy to lose: Kured blocks every reboot when its Prometheus query fails, the Authentik outpost is what prometheus.infra and alertmanager.infra resolve to, and the Ceph dashboard pulls from both. cnpg-system covers the operator only; each database lives in its workload's namespace, so the rule admitting the operator on port 8000 belongs to that namespace's policy — see Adding a Workload.

Egress is untouched: a default-deny there also needs DNS, the API server and every external endpoint, and getting it wrong takes the component down rather than leaving it exposed. Not covered, each for its own reason:

Default ServiceAccount tokens¶

A pod that names no serviceAccountName lands on default, which nothing configures, and gets an API token it has no use for. A workload reaching the API through default is already a misconfiguration, so default-serviceaccounts.yaml sets automountServiceAccountToken: false on it in every namespace with nothing to preserve, empty namespaces included. Left out: kube-system, where the control plane authenticates with client certificates and "very likely fine" is not the standard, and authentik, where authentik-server sets no serviceAccountName — an upstream chart default, to be fixed at the source.

Admission policies¶

Trivy reports a bad image after it is running; admission-policies.yaml refuses or records it at admission, with the built-in ValidatingAdmissionPolicy rather than a policy engine. Both match Pods and every controller that produces them, so a bad Deployment is rejected at kubectl apply rather than failing quietly in its ReplicaSet.

Both carry failurePolicy: Ignore: an expression that errors lets the request through rather than stopping every pod in the cluster. A real violation is still refused.

Usage¶

Promoting the registry allowlist¶

1. Confirm nothing running would be refused, over the full Loki retention:

   {job="kubernetes-audit"} |= "validation.policy.admission.k8s.io/validation_failure" |= "image-registry-allowed"
   

2. Change validationActions on the binding in admission-policies.yaml to ["Deny"].

Tightening a namespace¶

1. See what running pods a stricter level would reject, without changing anything:

   kubectl label --dry-run=server --overwrite ns cert-manager \
     pod-security.kubernetes.io/enforce=restricted
   

2. Catch what is not running right now — a nightly CronJob, say — from the audit label's records over the full Loki retention window, in Grafana:

   {job="kubernetes-audit"} |= "pod-security.kubernetes.io/audit-violations"
   

3. Raise enforce in pod-security.yaml.

Rolling out a network policy¶

One namespace at a time, watching Hubble between each:

kubectl -n kube-system port-forward svc/hubble-relay 4245:80
hubble observe --verdict DROPPED --namespace openbao --follow

Drops are also kept in Loki, so a policy that broke something overnight can be read back: {job="hubble", verdict="DROPPED"} — see Cilium.

If something legitimate is dropped, put a single endpoint into audit mode — decisions logged, not enforced — to find the missing rule without an outage:

kubectl -n kube-system exec ds/cilium -- \
  cilium endpoint config <endpoint-id> PolicyAuditMode=Enabled

Health check¶

kubectl get ns -L pod-security.kubernetes.io/enforce
kubectl get cnp -A
kubectl get validatingadmissionpolicy

Pitfalls¶