Sonnet 5 analysis below from a real issue I experienced. Please verify @privateip.
What needs to happen
A VRF that fails to wire into a node's kernel on its first attempt needs to actually surface that failure and keep retrying — not report healthy forever while carrying no traffic.
Why this matters
Found while debugging an unreachable Datum Cloud compute deploy (milo-os-com in project drewr-y4nd1b). The workload's NetworkInterface, BGPAdvertisement, and BGPVRFInstance objects all reported clean success, the compute node's own kernel VRF table was correctly populated, and a same-node connectivity test worked — but the edge node that's supposed to forward inbound traffic toward the compute node never sent a single packet. edge-1e72b4-us-central-1's BGPVRFInstance for that VPC (80000035-edge-1e72b4-us-central-1) was stuck at generation: 1 — created once and never touched again — despite the router reconciling other VPCs every ~30 seconds and despite the workload being destroyed and redeployed four times. Every other VPC's BGPVRFInstance on the same node had generation 2–14, actively reconciling. At the time, edge-1e72b4 had zero kernel VRF interfaces at all (ip -d link show type vrf returned nothing), yet every BGPVRFInstance on it reported Ready: True / "VRF instance accepted".
This is the same underlying pattern as #451 (closed "not planned" without a code fix): a VRF's kernel-side setup fails once, is never retried, and nothing in status ever reflects the failure. #451 hit this on a route-reflector's own underlay VRF; this is the same mechanism recurring on a tenant/EVPN VRF on a different node, which #451 didn't cover.
The only workaround found was moving the workload to a brand-new network (a fresh VPC ID), which gave the edge router a clean first attempt and worked — the new BGPVRFInstance reconciled correctly and reached generation 11 immediately. That's a workaround for one workload, not a fix — any other tenant whose VPC happens to hit this on first attach is stuck the same way, silently, with nothing in its own status ever telling them why.
Desired outcome
- A VRF that fails its first kernel setup attempt (interface resolution, BPF probe, or anything else in that path) is retried on a later reconcile rather than permanently frozen at generation 1 with no further attempts.
BGPVRFInstance/BGPAdvertisement Ready/Advertised conditions reflect whether the VRF is actually usable, not just "the reconcile ran without a Go error" — so a tenant (or an operator) can tell from the object's own status that something is wrong, instead of everything reading healthy while no traffic moves.
Sonnet 5 analysis below from a real issue I experienced. Please verify @privateip.
What needs to happen
A VRF that fails to wire into a node's kernel on its first attempt needs to actually surface that failure and keep retrying — not report healthy forever while carrying no traffic.
Why this matters
Found while debugging an unreachable Datum Cloud compute deploy (
milo-os-comin projectdrewr-y4nd1b). The workload'sNetworkInterface,BGPAdvertisement, andBGPVRFInstanceobjects all reported clean success, the compute node's own kernel VRF table was correctly populated, and a same-node connectivity test worked — but the edge node that's supposed to forward inbound traffic toward the compute node never sent a single packet.edge-1e72b4-us-central-1'sBGPVRFInstancefor that VPC (80000035-edge-1e72b4-us-central-1) was stuck atgeneration: 1— created once and never touched again — despite the router reconciling other VPCs every ~30 seconds and despite the workload being destroyed and redeployed four times. Every other VPC'sBGPVRFInstanceon the same node had generation 2–14, actively reconciling. At the time,edge-1e72b4had zero kernel VRF interfaces at all (ip -d link show type vrfreturned nothing), yet everyBGPVRFInstanceon it reportedReady: True / "VRF instance accepted".This is the same underlying pattern as #451 (closed "not planned" without a code fix): a VRF's kernel-side setup fails once, is never retried, and nothing in status ever reflects the failure. #451 hit this on a route-reflector's own underlay VRF; this is the same mechanism recurring on a tenant/EVPN VRF on a different node, which #451 didn't cover.
The only workaround found was moving the workload to a brand-new network (a fresh VPC ID), which gave the edge router a clean first attempt and worked — the new
BGPVRFInstancereconciled correctly and reached generation 11 immediately. That's a workaround for one workload, not a fix — any other tenant whose VPC happens to hit this on first attach is stuck the same way, silently, with nothing in its own status ever telling them why.Desired outcome
BGPVRFInstance/BGPAdvertisementReady/Advertisedconditions reflect whether the VRF is actually usable, not just "the reconcile ran without a Go error" — so a tenant (or an operator) can tell from the object's own status that something is wrong, instead of everything reading healthy while no traffic moves.