Control Plane VIP¶

The API server is reached through a virtual IP held by kube-vip, not through one named node: whichever control-plane node wins the leader election answers on control_plane_vip from ansible/inventory.yaml, and another takes over if it goes away. The address is fixed at kubeadm init — see Known Limitations; a cluster built before the VIP is migrated in place.

How it is wired¶

kube-vip runs in ARP mode with leader election, advertising a /32, with svc_enable: "false": LoadBalancer services belong to Cilium's L2 announcements, and two components answering ARP for the same range produces intermittent, host-dependent failures.

Design¶

The static pod is only the bootstrap: once ArgoCD syncs payload/platform/kube-vip/, a DaemonSet takes over, so upgrading kube-vip is a merged PR rather than a file rewritten on each node. kube-vip publishes a chart; it is not used because the handover below needs the adopt init container, which the chart has no hook for. A DaemonSet can hold the VIP here because the control-plane kubelets use their own node's API server (server: in /etc/kubernetes/kubelet.conf) and so does kube-vip (KUBERNETES_SERVICE_HOST is the host IP, which is in the certificate SANs), so both start after a full power loss without the VIP or Cilium. kube_vip_version in the inventory is therefore the bootstrap version only, and the VIP settings in the inventory and daemonset.yaml must be kept in step. The first sync replaces all three static pods at once and drops the VIP for a few seconds; later updates roll one node at a time. kube-vip's ServiceAccount may only get and update its own Lease, and the Application does not prune and its objects carry Delete=false, because removing them takes the API away from the workers and from every kubeconfig.

The handover¶

The adopt trade-off¶

adopt and its writable hostPath stay after the handover, although on an adopted node they do nothing and every pod start still runs a container that can delete the control plane's static pod manifests. A reinstalled control-plane node gets the static manifest back from Ignition and needs adoption again, and forgetting to re-add it for that one reinstall leaves two kube-vips contending for the API VIP; the exposure is limited to the pinned image.

Endpoint name¶

The next cluster build uses a DNS name as controlPlaneEndpoint, so moving the VIP later no longer means rewriting every node's kubeconfig; certSANs still needs regenerating for the new address. The running cluster keeps the bare VIP: there is no migration, because its certificates and stored configuration do not carry a name.

To build with a name:

1. Set control_plane_endpoint_host in ansible/inventory.yaml to the name, before the first make config of the build, and keep it there.

   control_plane_endpoint_host: k8s-api.<lan domain>
   

2. Continue with the Quickstart. Everything else follows from that variable.

Check that nodes and the cluster agree once the first control plane is up.

kubectl -n kube-public get configmap cluster-info -o jsonpath='{.data.kubeconfig}' | grep server

Choosing the address¶

A free address on the nodes' subnet, outside any DHCP range, and distinct from the Cilium LoadBalancer pools. Nothing validates this: a collision shows up as an API server unreachable after provisioning, or reachable from some machines and not others depending on whose ARP cache won.

Migrating a cluster built without a VIP¶

A cluster provisioned before the VIP has its first control-plane node's address baked into the API server certificates, so this is not a config change you can sync.

Rebuild (simplest)¶

Reprovision from the Quickstart; the VIP is in place from kubeadm init onwards. Restore state per Backups & Recovery.

In place¶

One node at a time, with a second terminal holding a working kubeconfig, because several steps can leave you unable to open a new one.

1. Confirm the address is free.

   ping -c2 <vip>        # must not answer
   arping -c2 <vip>      # from a node on the segment
   

2. Add the VIP under apiServer.certSANs in the stored ClusterConfiguration.

   kubectl -n kube-system edit configmap kubeadm-config
   

3. Regenerate the API server certificate on each control-plane node.

   ssh core@<node>
   sudo mv /etc/kubernetes/pki/apiserver.crt /etc/kubernetes/pki/apiserver.crt.bak
   sudo mv /etc/kubernetes/pki/apiserver.key /etc/kubernetes/pki/apiserver.key.bak
   sudo kubeadm init phase certs apiserver
   sudo crictl ps | grep kube-apiserver   # confirm it restarted
   

4. Deploy the kube-vip static pod to each control-plane node, generated from the template's values or copied from a node provisioned with the new configuration, and verify the VIP answers.

   curl -k https://<vip>:6443/healthz
   

5. Repoint the cluster: update controlPlaneEndpoint in the kubeadm-config ConfigMap, then on every node rewrite the server address in /etc/kubernetes/*.conf and /var/lib/kubelet/kubeconfig and restart the kubelet.

6. Only now change k8sServiceHost in payload/platform/cilium/values.yaml to the VIP and commit. The merge rolls every agent, so the VIP must answer before it lands; watch the rollout.

   kubectl -n kube-system rollout status ds/cilium
   

7. Re-fetch your kubeconfig so it uses the VIP.

   make kubeconfig
   

Verifying¶

# which node currently holds the VIP
kubectl -n kube-system get lease plndr-cp-lock -o jsonpath='{.spec.holderIdentity}'; echo
kubectl -n kube-system get pods -l app.kubernetes.io/name=kube-vip -o wide

# the endpoint your kubeconfig actually uses
kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'; echo

Test failover once, deliberately: reboot the leader per Rebooting a node and confirm kubectl keeps working after a few seconds.