fix(podman): resolve local callbacks to loopback on Linux - #3435
politerealism wants to merge 1 commit into
Conversation
Since RFC 0012 (NVIDIA#2942), the callback-capable supervisor always runs in the host network namespace, for both rootful and rootless Podman. The driver still requested a callback listener via bridge-network gateway inspection (rootful) or default-route negotiation (rootless pasta), and aliased host.containers.internal/host.openshell.internal through Podman's host-gateway resolver. On multi-homed hosts, Podman's host-gateway resolution and the driver's own negotiation can each pick a different interface than the one the supervisor actually shares with the gateway (loopback), leaving sandbox creation unreachable. Request loopback directly for the automatic local-callback path on Linux, and resolve the host aliases to 127.0.0.1 to match, regardless of rootful/rootless mode or which rootless network helper Podman reports. An explicit host_gateway_ip override is unaffected. macOS Podman Machine behavior (gvproxy host-loopback forwarding) is unchanged. Fixes NVIDIA#3412 Signed-off-by: politerealism <burdcat17@gmail.com>
|
With the new split, I think I we can actually remove all the callback machinery and remove this class of bug. Could you see if #3365 resolve your issue? |
Manual verificationBeyond the unit tests, I ran this against a real local Podman-backed gateway built from this branch to confirm the fix end-to-end. Test 1 — default config (loopback-bound gateway): Inspected the supervisor container directly: Test 2 — gateway primary listener bound to a real interface IP ( Gateway log: The Sandbox creation against that gateway also reached Supervisor container again showed Cleaned up both test sandboxes and the local gateway afterward; no leftover containers. |
Summary
On Linux, the Podman driver's automatic local-callback path negotiated a bridge-network gateway address (rootful) or the host's default-route interface (rootless pasta), and aliased
host.containers.internal/host.openshell.internalthrough Podman'shost-gatewayresolver. Since RFC 0012 (#2942), the callback-capable supervisor always runs in the host network namespace and shares loopback with the gateway, so on multi-homed hosts these two independent interface selections can disagree, leaving the supervisor unable to reach the gateway.Related Issue
Fixes #3412
Changes
driver.rs: automatic Linux callback path now always requests a loopback gateway listener, for both rootful and rootless Podman; removed the now-obsolete bridge-gateway-IP discovery and rootless-pasta-only gating. Explicithost_gateway_ipoverride path is simplified and now works uniformly regardless of rootless network helper.container.rs: host aliases resolve to127.0.0.1directly on Linux instead of Podman'shost-gatewaymagic value, matching the requested loopback listener. macOS Podman Machine behavior (gvproxy host-loopback forwarding) is unchanged.client.rs: removed the now-deadnetwork_gateway_ipbridge-inspection method (no remaining callers).config.rs,NETWORKING.md,architecture/gateway.md,docs/reference/gateway-config.mdx,docs/reference/sandbox-compute-drivers.mdx,skills/debug-openshell-cluster/SKILL.md: updated to describe the new loopback-based default and troubleshooting guidance.Out of scope (left for follow-up, per the issue's own implementation plan): removing PR #2874's delayed-bind protobuf/server machinery (a separate, wire-breaking proto cleanup), and the misleading
ContainerExited: code 0diagnostic when the supervisor exits before the workload.Testing
mise run pre-commitpassescrates/openshell-driver-podman/src/driver.rs,container.rs) — 224 passing, including new/updated coverage for: rootful and rootless automatic loopback selection, explicit override still works with any rootless helper, constructor no longer inspects the bridge network for local callbacks.cargo fmt --all -- --checkandcargo clippy -p openshell-driver-podman --all-targets -- -D warningscleanmise run markdown:lintcleanChecklist