Known Limitations¶
Things this cluster does not do, collected so they are findable before they are discovered. None is a bug; each is an accepted tradeoff or work not yet done.
| Limitation | Consequence | Mitigation / plan | Where documented |
|---|---|---|---|
The API endpoint address is fixed at kubeadm init |
controlPlaneEndpoint is the kube-vip VIP, and kubeadm bakes it into the API server certificates, every node's kubeconfig and the stored ClusterConfiguration; Cilium's k8sServiceHost names the same address. Moving the VIP to another address is therefore a certificate regeneration on every control-plane node and a kubeconfig rewrite on every node, not a config change that can be synced |
Choose the address once, outside any DHCP range and apart from the LoadBalancer pools. The in-place procedure exists for the day it has to move; the next build uses a name | Choosing the address, migrating in place |
| OpenBao needs an operator to unseal it | Shamir seal, no auto-unseal: every restart (node reboot, Kured cycle, chart bump) leaves each replica sealed until the threshold of key shares is supplied. Sealed, no ExternalSecret resolves and cert-manager cannot renew. The cluster looks healthy; the consequence surfaces when a certificate expires |
Check bao status after every reboot. Auto-unseal against a KMS or a second OpenBao would remove the step at the price of an external hard dependency; the key shares stay in the operator's hands on purpose |
Unsealing after a restart, after any node reboot |
| Alerting reaches one mailbox | One receiver, one mailbox, one SMTP provider, unmonitored. If mail is rejected or the password expires, alerts stop silently; Watchdog proves the pipeline only as far as Alertmanager |
A second receiver on a different transport. Meanwhile, run the routine health check | Alerting, routine health check |
| Backups do not leave the cluster | Velero and the etcd snapshot CronJob write to the Ceph object store the cluster runs on: protection against a deleted PVC or a bad prune, none against losing the cluster |
RGW bucket replication or a second Velero BackupStorageLocation; neither is configured. Until then disaster recovery is Git plus the OpenBao unseal keys |
Backups & Recovery |
| Automatic updates stop at patch releases | Sysupdate is pinned to the Kubernetes major.minor in ansible/inventory.yaml; Kured drains and reboots one node at a time to apply what is staged, and refuses while Ceph or etcd is unhealthy. A minor Kubernetes upgrade is manual: change the inventory, push the sysupdate config to running nodes, run kubeadm upgrade. Kured blocks only on the alerts named in its config; anything outside that list does not stop a reboot |
By design | Nodes are pinned to a minor series, upgrading a minor deliberately, Kured |
The infra AppProject is permissive |
The network policies are enforced, but the infra AppProject allows every kind in every namespace, so a platform application.yaml can deploy anything anywhere |
Narrow the projects | Security Posture |
| Reprovisioning needs the boot server on the segment | Rebuilding a node means make serve on a machine on the nodes' L2 segment, the external DHCP pointing at it, make reinstall LIMIT=<node>, and a power cycle. There is no remote rebuild, and the deployment host is not part of the cluster |
Only reprovisioning: Flatcar is installed to disk, so a running node reboots, updates and rejoins with the boot server off | Boot & Bootstrap Process |
| The object store survives one host down, no more | During every Kured reboot the RGW data pool, which holds the Velero backups and Loki chunks, has no redundancy left, while rbd-pool still has some |
Accepted, no migration: Ceph cannot change the erasure profile of an existing pool, so changing it means a new pool and moving every bucket | Object storage |
| Single region, site and rack | No failure domain larger than a node. Ceph replicates across nodes on one power feed; a site-level event takes everything | The honest limit of a homelab | — |