Skip to content

fix(mxc): harden governed proxy lifecycle (NVBug 6783325) - #3472

Open
prekshivyas wants to merge 2 commits into
NVIDIA:windowsfrom
prekshivyas:6783325-mxc-proxy-startup/pv
Open

prekshivyas wants to merge 2 commits into
NVIDIA:windowsfrom
prekshivyas:6783325-mxc-proxy-startup/pv

Conversation

@prekshivyas

@prekshivyas prekshivyas commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Observability and regression hardening for NVIDIA NVBug 6783325. An explicit network_policies rule activates governed egress, while a policy with no network rules leaves the host listener and proxy environment absent; the lifecycle also emits a successful startup log and the mock path exercises the real host proxy.

The headline "explicit policy never starts" symptom no longer reproduces on the current windows tip: explicit rules already created a real listener and injected proxy variables, and a normally completed one-shot workload removed the listener after AgentCompleted. The patch makes those contracts durable because the default/no-network-policy case still activated the proxy unconditionally, successful startup was not logged, and the mock lifecycle explicitly skipped listener startup.

Base: fb2980e077288b61ef03a2e6187e162d158526aa (origin/windows, re-fetched immediately before commit).

Related Issue

NVIDIA NVBug 6783325. No public GitHub issue required; this is a localized Windows MXC bug fix.

NVBug 6782969 remains separate. This PR does not change policy matching or enforcement and sends no request through the deny/no-match scenario.

Root Cause

  • egress_proxy = true alone produced a per-sandbox proxy address, MXC loopback redirect, credentials, and proxy environment for every policy, including policies without network rules.
  • The mock invoker was explicitly excluded from start_host_proxy, so tests could observe redirect/env serialization but could not catch a missing host listener or verify cleanup.
  • Successful proxy startup had no lifecycle log.

Changes

  • Gate the governed-egress address on an explicit network policy (and preserve fail-closed middleware rejection).
  • Reject egress_proxy + pc_network_allow when no explicit network rules exist, preventing a silent unrestricted-egress fallback; explicit mixed policies remain proxy-backed and default-deny.
  • Start the actual authenticated loopback proxy in mocked lifecycle tests and log successful startup with sandbox and bound address.
  • Add positive coverage for allow and deny/no-match policies plus negative coverage for the default/no-network-policy case, including listener ownership, proxy env injection, and stop/delete cleanup.
  • Update MXC documentation and affected examples so configurations that need proxy/audit behavior include an explicit rule, while the loopback-only relay example no longer claims governed Internet egress.

Exact Before/After Evidence

Environment: Yukon Windows 10.0.28000 ARM64, native Rust 1.95.0 aarch64-pc-windows-msvc, signed Microsoft wxc-exec.exe 0.8.0 (DDE1C592270E9A659B01DCCAD70362DA7B99FEC114885FA4D625507AA775A503). Workloads printed only presence markers; proxy credentials and values were not logged.

Unmodified base, gateway PID 25388:

  • Explicit allow: Ready; listener 127.0.0.1:51990; env count 17 with HTTP_PROXY=SET / HTTPS_PROXY=SET; no successful startup log; stop completed in 67 ms and closed the listener.
  • Explicit deny/no-match (allowed.invalid:443): Ready; listener 127.0.0.1:65434; the same proxy env markers; stop/delete cleaned up.
  • No network policy: Ready, but unexpectedly opened 127.0.0.1:60489 and injected the same 17-variable proxy environment.
  • The new tests failed before the production change: explicit-policy lifecycle had no host_proxy under mock, and the default case unexpectedly had a proxy address.

Patched commit, gateway PID 32012, identical policies/config/workload:

  • Explicit allow: Ready; startup log MXC host egress proxy started sandbox=n6783325-allow3 address=127.0.0.1:59127; listener owned by the gateway; env count 17 with both proxy markers; stop completed in 61 ms and the listener was gone after 250 ms.
  • Explicit deny/no-match: Ready; startup log at 127.0.0.1:50743; listener/env present; stop/delete cleaned up.
  • No network policy: Ready; no startup log, no per-sandbox listener, no proxy markers, and env count 5 (Windows bootstrap variables only).

Security Impact

This narrows exposure: policies without explicit network governance no longer receive a host loopback listener, per-sandbox proxy credential, broad loopback MXC allowance, or proxy environment. A conflicting unrestricted pc_network_allow fallback is rejected synchronously instead of weakening that boundary. Explicit governed egress remains fail-closed on startup/mapping errors. Provider credential resolution still requires both egress_proxy and an explicit network policy. No permissions or policy-enforcement rules are broadened.

Testing

  • mise run pre-commit passes
    • Rust formatting and Markdown lint passed, and the SPDX check passed independently (981 files).
    • Aggregate task is blocked on this ARM64 host by Python 3.14 building grpcio-tools from source: MSVC rejects upstream /std:c++17 plus /std:c11 flags.
  • Unit tests added/updated
    • cargo test -p openshell-driver-mxc --target aarch64-pc-windows-msvc: 95 unit + 57 integration tests passed; 12 real-MXC tests ignored by the normal suite.
    • mise run --skip-tools windows:test:arm64: 4,943 passed, 26 skipped, zero failures.
    • Focused ARM64 package Clippy passed with the repository/base warning allowances; no new diagnostics remain.
    • cargo fmt --all -- --check, git diff --check, PowerShell parser check: passed.
  • E2E tests added/updated (if applicable)
    • Staged ARM64 MXC mock runner: 4/4 scenarios passed.
    • Signed real ARM64 MXC before/after lifecycle matrix passed as detailed above.
    • Release ARM64 build passed before (8m51s) and after (1m08s incremental).

Additional infrastructure notes:

  • windows:check:arm64 reached z3-sys but the configured prebuilt Z3 download returned HTTP 403.
  • The repository Tier-2 real-MXC wrapper was queued behind another independent task whose external wxc-exec probe hung; only this PR's queued wrapper was canceled. The direct signed-MXC matrix above completed and is the scenario-specific real-runtime proof.
  • The first patched real-MXC allow attempt opened/logged its listener, then MXC transiently rejected its own configuration as unsupported by the selected backend; deleting and repeating the identical command succeeded. No OpenShell code changed between attempts.

Residual Risk / Platform Limits

  • The existing host proxy remains an environment-driven path for proxy-aware clients; direct network enforcement and NVBug 6782969 are out of scope.
  • The real MXC backend showed the single transient rejection documented above.
  • Validation was native ARM64; no x64-only NemoClaw qualification was claimed.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (MXC behavior documented in the driver README)

@copy-pr-bot

copy-pr-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@prekshivyas

Copy link
Copy Markdown
Contributor Author

Qualification before/after summary

NVBug: 6783325 — MXC host-proxy activation/lifecycle does not match explicit policy

Before: Explicit allow and deny/no-match policies started a listener and injected proxy variables, but the no-network-policy case incorrectly did the same. Mock lifecycle skipped the real listener and successful startup had no durable log.

After: Explicit allow and deny/no-match retain governed proxy startup; the no-network-policy case has no listener, proxy credentials, or proxy environment. Real ARM64 lifecycle proof confirmed listener ownership and cleanup. Results include 94 unit + 57 integration tests, 4,942 Windows ARM64 tests passed with 26 skipped, mock 4/4, and the signed real-MXC matrix passing.

Qualification status: Focused scenario PASS; full combined qualification pending. A full two-cycle OpenClaw qualification should run after combining/rebasing this change with #3434. Hosted CI also requires a maintainer /ok to test <head SHA> mirror authorization.

@shailendra-nv shailendra-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the two inline findings below. The proxy activation test and startup log are valuable, but the empty-policy gate must not silently change mixed-config egress from loopback-only to unrestricted. Please also align the relay qualification documentation with the new non-proxy configuration. For NVBug 6783325, the PR should be treated as observability/regression hardening: the explicit-policy production path already worked on the base revision, and the original clean-exit reproduction still removes the listener after AgentCompleted.

policy: Option<&SandboxPolicy>,
) -> Result<Option<SocketAddr>, tonic::Status> {
let configured = configured_egress_addr(config)?;
Ok(configured.filter(|_| policy_activates_governed_egress(policy)))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filtering the configured proxy address to None changes precedence with the existing pc_network_allow fallback. With egress_proxy=true, pc_network_allow=true, and no network rules, the base used loopback-only governed egress; this revision reaches the fallback that emits egress.default=allow. Preserve default-deny, reject the conflicting configuration, or explicitly document and test this security-sensitive behavior.

@prekshivyas prekshivyas Sep 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 8e79706. The mixed configuration now fails closed during sandbox validation: when egress_proxy = true and pc_network_allow = true, a policy without explicit network rules returns InvalidArgument before lifecycle startup instead of falling through to unrestricted defaultPolicy = "allow". empty_network_policy_rejects_unrestricted_fallback covers that branch. I also run the explicit deny/no-match lifecycle case with pc_network_allow = true; it remains proxy-backed with network.egress.default = deny, so explicit governed policies retain their existing precedence. The full native ARM64 workspace passed: 4,943 passed, 26 skipped.

egress_proxy_addr = "127.0.0.1:18080"
# No governed Internet egress is needed. The relay reaches the driver's
# on-demand loopback listener through privateNetworkClientServer above.
egress_proxy = false

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This configuration now disables egress_proxy and relies on privateNetworkClientServer, but companion qualification comments still say mxc-ws-gateway.toml enables the proxy and that the relay connects through it (run-ws-agent-test.ps1, relay.rs, and mxc-ws-agent.rs). Update those references so qualification attributes connectivity to the correct mechanism.

@prekshivyas prekshivyas Sep 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 8e79706. I updated all three named references (run-ws-agent-test.ps1, relay.rs, and mxc-ws-agent.rs) plus clarified mxc-ws-gateway.toml: this qualification path connects directly to the route-selected private-interface relay listener through privateNetworkClientServer; governed egress and the host CONNECT proxy are not involved. The PowerShell parser check and the complete MXC package suite pass.

@prekshivyas prekshivyas changed the title fix(mxc): gate proxy startup on explicit policy (NVBug 6783325) fix(mxc): harden governed proxy lifecycle (NVBug 6783325) Sep 19, 2026
@prekshivyas

prekshivyas commented Sep 19, 2026

Copy link
Copy Markdown
Contributor Author

Implemented both requested changes in 8e79706:

  • fail closed on egress_proxy + pc_network_allow when the sandbox policy has no explicit network rules, with a new negative regression test;
  • prove explicit deny/no-match plus pc_network_allow stays proxy-backed and default-deny;
  • align the relay qualification docs/comments around direct private-network relay access (no governed CONNECT proxy).

Validation after the review update:

  • MXC package: 152 passed, 12 real-MXC tests ignored by the normal suite;
  • native ARM64 workspace: 4,943 passed, 26 skipped, zero failures;
  • focused package Clippy, Rust formatting, diff check, and PowerShell parser: passed.

I also reframed the title/description as observability and regression hardening, explicitly noting that current-base explicit-policy startup and clean-exit listener removal already worked.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@prekshivyas
prekshivyas force-pushed the 6783325-mxc-proxy-startup/pv branch from 119aa9f to 8e79706 Compare September 19, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants