Known Limitations¶
Things this cluster does not do, collected so they are findable before they are discovered. None is a bug; each is an accepted tradeoff or work not yet done.
| Limitation | Consequence | Mitigation / plan | Where documented |
|---|---|---|---|
API server endpoint is fixed at kubeadm init |
controlPlaneEndpoint is baked into the API server certificates. A cluster whose certificates name one node is single-homed on it: while that node is down no node can join, Cilium on every other node loses the API server (with kube-proxy replaced it cannot reach it through a Service), and output/kubeconfig points at nothing. Workloads keep running; control-plane access fails |
Provisioning puts a kube-vip VIP in front of the API servers. Cilium's k8sServiceHost must name an address that answers — pointing it at one that does not takes the CNI down cluster-wide — so it moves to the VIP only once the VIP is live |
Control Plane VIP, migrating a cluster built without one |
| OpenBao needs an operator to unseal it | Shamir seal, no auto-unseal: every restart (node reboot, Kured cycle, chart bump) leaves each replica sealed until 3 of 5 key shares are supplied. Sealed, no ExternalSecret resolves and cert-manager cannot renew. The cluster looks healthy; the consequence surfaces when a certificate expires |
Check bao status after every reboot. Auto-unseal against a KMS or a second OpenBao would remove the step at the price of an external hard dependency; the key shares stay in the operator's hands on purpose |
Unsealing after a restart, after any node reboot |
| Alerting reaches one mailbox | One receiver, one mailbox, one SMTP provider, unmonitored. If mail is rejected or the password expires, alerts stop silently; Watchdog proves the pipeline only as far as Alertmanager |
A second receiver on a different transport. Meanwhile, run the routine health check | Alerting, routine health check |
| Backups do not leave the cluster | Velero and the etcd snapshot CronJob write to the Ceph object store the cluster runs on: protection against a deleted PVC or a bad prune, none against losing the cluster |
RGW bucket replication or a second Velero BackupStorageLocation; neither is configured. Until then disaster recovery is Git plus the OpenBao unseal keys |
Backups & Recovery |
| Automatic updates stop at patch releases | Sysupdate is pinned to the Kubernetes and containerd major.minor in ansible/inventory.yaml; Kured drains and reboots one node at a time to apply what is staged, and refuses while Ceph or etcd is unhealthy. A minor Kubernetes upgrade is manual: change the inventory, push the sysupdate config to running nodes, run kubeadm upgrade. Kured blocks only on the alerts named in its config; anything outside that list does not stop a reboot |
By design | Nodes are pinned to a minor series, upgrading a minor deliberately, Kured |
| Network policy is partial, AppProjects are permissive | Ten namespaces have default-deny ingress and every platform namespace has Pod Security Admission labels. All egress is unrestricted, other namespaces allow all ingress, and the infra AppProject allows every kind in every namespace. A pod in authentik can still talk to a pod in kured |
Extend default-deny, then egress; narrow the projects | Security Policies, Security Posture |
| Reprovisioning needs the boot server on the segment | Rebuilding a node means make serve on a machine on the nodes' L2 segment, the external DHCP pointing at it, make reinstall LIMIT=<node>, and a power cycle. There is no remote rebuild, and the deployment host is not part of the cluster |
Only reprovisioning: Flatcar is installed to disk, so a running node reboots, updates and rejoins with the boot server off | Boot & Bootstrap Process |
| Single region, site and rack | No failure domain larger than a node. Ceph replicates across nodes on one power feed; a site-level event takes everything | The honest limit of a homelab | — |