Platform¶
The core infrastructure components that run the cluster, all managed by ArgoCD. Every one of them exists because bare metal does not come with the thing a cloud provider would have handed you: no load balancer, no managed certificates, no block storage API, no identity provider, no backup service.
Components¶
One row per directory under payload/platform/, with the Application name
where it differs from the directory — that is the name kubectl -n argocd get
applications shows — and the namespace it lands in.
| Component | Namespace | What it does |
|---|---|---|
| alloy | logging |
Grafana Alloy, the log shipper — see Logging |
| arc-controller | arc-systems |
The Actions Runner Controller — see GitHub runners |
| arc-runners | arc-runners, arc-jobs |
Self-hosted GitHub Actions runners, one scale set per repository |
| argocd | argocd |
ArgoCD itself and the platform ApplicationSet |
| argocd-config | argocd |
The ExternalSecret for ArgoCD's OIDC client, and its Grafana dashboard; see ArgoCD |
| argocd-projects | argocd |
The AppProjects; see GitOps Strategy |
| authentik | authentik |
Single sign-on for every platform UI |
| backup | backup |
Velero's buckets, the VolumeSnapshotClass and an etcd snapshot CronJob — see Backups & Recovery |
| cert-manager | cert-manager |
The cert-manager chart |
| certificates | cert-manager |
The Let's Encrypt issuers and the Gateway wildcard certificates — see cert-manager |
| cilium | kube-system |
CNI, kube-proxy replacement, Gateway API, LoadBalancer addresses, WireGuard, Hubble |
| cloudnative-pg | cnpg-system |
The PostgreSQL operator every workload database runs on |
| descheduler | descheduler |
Evicts pods a drain or reboot left piled on the surviving nodes |
| external-dns | external-dns |
Publishes Route53 records from HTTPRoutes |
| external-secrets | external-secrets |
Bridges OpenBao to native Kubernetes Secrets |
| gateway-api | kube-system |
The two Gateways and the HTTP-to-HTTPS redirect |
| gateway-api-crds | kube-system |
The Gateway API CRDs — see Gateway API |
| kube-vip | kube-system |
Holds the control-plane VIP; adopts the static pod Ignition bootstraps — see Control Plane VIP |
| kubelet-csr-approver | kubelet-csr-approver |
Approves kubelet-serving CSRs against the inventory — see Metrics Server |
| kured | kured |
Drains and reboots nodes to apply staged OS and Kubernetes updates |
| logging | logging |
Loki's bucket, rules, dashboards and Grafana datasource |
| loki | logging |
The Loki chart — see Logging |
| metrics-server | kube-system |
The metrics.k8s.io API behind kubectl top and every HPA |
monitoring, Application kube-prometheus-stack |
monitoring, node-exporter |
Prometheus, Grafana, Alertmanager, kube-state-metrics; node-exporter alone in node-exporter — see Monitoring |
| openbao | openbao |
Cluster-wide secret store |
| prometheus-operator-crds | monitoring |
The Prometheus operator CRDs, separate so every chart that ships a ServiceMonitor can sync — see Monitoring |
| rook-ceph | rook-ceph |
The CSI driver, dashboards and HTTPRoute |
| rook-ceph-cluster | rook-ceph |
The CephCluster, pools and storage classes — see Rook-Ceph |
| rook-ceph-operator | rook-ceph |
The Rook operator chart — see Rook-Ceph |
| security | kube-system |
Pod Security Admission levels, the network policies and image admission policies — see Security Policies |
| snapshot-controller | backup |
The CSI snapshot controller Velero's volume snapshots need — see Backups & Recovery |
| tetragon | kube-system |
Runtime detection: execs, credential changes and sensitive file reads from the kernel |
| trivy-operator | trivy-system |
Vulnerability, misconfiguration, secret, RBAC and CIS scanning, every finding a CRD |
| velero | backup |
Volume and resource backups to the Ceph object store |
Across the platform, memory limits are set at roughly 2.5x the measured peak
working set and requests at steady state; CPU is requested but never limited.
The control-plane static pods get their memory requests through the kubeadm
patches in ansible/templates/butane_node_config.yaml.j2, applied at
kubeadm init, join and upgrade, so a running node picks them up at its
next reinstall; they carry no limits, because a limit on the API server or
etcd would OOM-kill the control plane and 2.5x their peak exceeds the node.
CoreDNS is kubeadm's too: the same directory holds a patch spreading its
replicas one per node, which kubeadm applies at init and upgrade apply
only, and its PodDisruptionBudget sits in the security component because
nothing else owns a CoreDNS manifest.
Components the cluster cannot run without — the CNI and API VIP, the
admission webhooks, the secret and certificate chain, the CSR approver, the
storage daemons and the node security agents — carry a system-* priority
class, so under node pressure the kubelet evicts workloads first:
system-node-critical for the DaemonSets a node needs,
system-cluster-critical for the rest.
Traffic Flow¶
flowchart LR
Client([Client]) --> LB[Cilium LoadBalancer]
LB --> GW[Gateway API]
subgraph Cluster
GW -->|HTTPRoute| Svc[Service]
Svc --> Pod[App Pod]
end
style Client fill:#f9f,stroke:#333
Four hops, and Cilium is three of them. When a hostname stops answering, ask
which hop stopped, in this order: does the Gateway still hold its LoadBalancer
IP, does the HTTPRoute still say Accepted, does the Service still have
endpoints.
HTTPRoute Locations¶
Routes sit with the component they expose, as a manifest where the component is plain YAML and as chart values where the chart renders its own:
| Service | URL | Route |
|---|---|---|
| ArgoCD | argo.infra.k8s.wlkr.ch |
server.httproute in payload/argocd/values.yaml |
| ArgoCD CLI (gRPC) | argo-grpc.infra.k8s.wlkr.ch |
server.grpcroute in payload/argocd/values.yaml |
| Authentik | auth.k8s.wlkr.ch |
payload/platform/authentik/httproute.yaml |
| Prometheus | prometheus.infra.k8s.wlkr.ch |
payload/platform/monitoring/httproute.yaml |
| Alertmanager | alertmanager.infra.k8s.wlkr.ch |
payload/platform/monitoring/httproute.yaml |
| Grafana | monitoring.infra.k8s.wlkr.ch |
grafana.route.main in payload/platform/monitoring/application.yaml |
| Hubble | hubble.infra.k8s.wlkr.ch |
payload/platform/cilium/httproute.yaml |
| OpenBao UI | vault.infra.k8s.wlkr.ch |
server.gateway.httpRoute in payload/platform/openbao/application.yaml |
| Rook dashboard | rook.infra.k8s.wlkr.ch |
payload/platform/rook-ceph/httproute.yaml |
| Ceph object store | s3.infra.k8s.wlkr.ch |
payload/platform/rook-ceph/httproute.yaml |
| Home Assistant | home.k8s.wlkr.ch |
home-assistant/httproute.yaml in homelab-apps |
Deployment¶
After that the argocd Application syncs the platform ApplicationSet, which
generates one Application per payload/platform/*/application.yaml plus the
workloads Application that deploys the apps ApplicationSet for the
workloads repository. Each component
directory holds exactly one application.yaml; everything else in it is what
that Application deploys. GitOps Strategy has the
structure and the sync policy.
Replica placement¶
Two replicas on one node are not redundancy. Every chart's default is a single
preferredDuringSchedulingIgnoredDuringExecution anti-affinity term — one
weight among the scheduler's scores, and on a cluster of one 16 GB worker
against two 8 GB ones, free memory outscores it. So every component with
more than one replica carries an explicit rule:
| Rule | When |
|---|---|
topologySpreadConstraints, maxSkew: 1, whenUnsatisfiable: ScheduleAnyway |
the component runs on the three workers only, where a hard rule would leave a replica Pending through a Kured drain |
podAntiAffinity requiredDuringSchedulingIgnoredDuringExecution |
the component tolerates the control planes, so every node is a candidate and a drain always leaves room — or its chart exposes no spread constraint |
Constraints carry nodeTaintsPolicy: Honor, so the skew is measured over the
nodes the pod can reach rather than over the tainted control planes. Where a
component carries a PodDisruptionBudget it says maxUnavailable: 1, so a drain
or the descheduler evicts one replica at a time;
maxUnavailable rather than minAvailable, so a component scaled down to one
replica never blocks a drain. The CloudNativePG operator runs two replicas
with no budget. Placement
is scored once, at scheduling time: pods already sharing a node stay there
until something restarts them, and the descheduler moves
them back only if its evictor is allowed to touch them.
Rollout order¶
Nothing enforces an order: every Application syncs as soon as it exists and converges by retry and health, as GitOps Strategy explains. This is the order a fresh cluster settles in, and what each layer is waiting for while it is not yet Healthy.
| Layer | Applications | Waits for |
|---|---|---|
| Projects and CRDs | argocd-projects, gateway-api-crds, prometheus-operator-crds |
— |
| Network | cilium, kube-vip |
the Gateway API CRDs Cilium's operator reads at startup |
| Controllers | arc-controller, cert-manager, cloudnative-pg, external-secrets, kubelet-csr-approver, rook-ceph-operator, snapshot-controller |
a network; each brings its own CRDs |
| Storage | rook-ceph, rook-ceph-cluster |
the Rook operator and its CRDs |
| Secrets | openbao |
rook-ceph-block for its volumes, then an operator — see Bootstrap pauses at OpenBao |
| Certificates | certificates, external-dns |
a working ClusterSecretStore and the Route53 credentials in OpenBao |
| Ingress | gateway-api |
the wildcard certificates the Gateways terminate TLS with |
| Services | arc-runners, argocd-config, authentik, backup, kube-prometheus-stack, logging, metrics-server |
secrets, storage, the Gateways, and approved kubelet certificates |
| Backends | loki, velero |
the buckets logging and backup claim |
| Agents | alloy, tetragon |
Loki, so the collector has somewhere to ship |
| Policy | descheduler, kured, security, trivy-operator |
namespaces to label, a Prometheus to scrape and metrics-server for node usage; kured reboots only on a staged update, never on a fresh install |
| Workloads | workloads |
nothing on its own; the workloads it generates wait for the platform resources they reference — see the workloads repository |
Why the ClusterSecretStore sits with OpenBao and the issuers are their own
Application is in
Bootstrap convergence.