feat(sandbox): add control and boundary supervisor modes

Cargo.toml

@@ -38,7 +38,7 @@ http-body-util = "0.1"
 tokio-rustls = { version = "0.26", default-features = false, features = ["logging", "tls12", "ring"] }
 rustls = { version = "0.23", default-features = false, features = ["std", "logging", "tls12", "ring"] }
 rustls-pemfile = "2"
-rcgen = { version = "0.13", features = ["crypto", "pem"] }
+rcgen = { version = "0.13", features = ["crypto", "pem", "x509-parser"] }
 webpki-roots = "1"
 rustls-native-certs = "0.8"
 

architecture/compute-runtimes.md

@@ -1,20 +1,26 @@
 # Compute Runtimes
 
 Compute runtimes create, stop, start, delete, and watch sandbox workloads for the
-gateway. Supervisor-controlled runtimes start a workload that runs the
-`openshell-sandbox` supervisor, which enforces the sandbox contract locally.
-Driver-controlled runtimes apply the canonical sandbox policy while
-provisioning and report workload readiness directly.
+gateway. They do not replace sandbox policy enforcement. A runtime either starts
+the co-located `openshell-sandbox` supervisor or provisions its RFC 0012
+`--mode=control` and `--mode=boundary` placements. Runtimes without the standard
+supervisor apply the canonical sandbox policy while provisioning and report
+workload readiness directly.
 
 ## Driver Contract
 
 Each runtime receives a sandbox spec and canonical policy from the gateway and
 is responsible for:
 
 - Selecting the sandbox image.
-- For supervisor-controlled runtimes, injecting sandbox identity and gateway
-  callback configuration, supplying callback credentials, and providing the
-  supervisor binary or image.
+- Injecting sandbox identity and gateway callback configuration.
+- Supplying TLS or secret material for supervisor callbacks.
+- Providing the supervisor binary or image in the workload.
+- Provisioning protected control and boundary configs plus a private Unix socket,
+  TLS-authenticated TCP, or vsock transport when the supervisor is separated.
+  Runtime-specific code supplies immutable resource claims and transport
+  coordinates; the shared boundary protocol supplies lifecycle, exec, signaling,
+  forwarding, and binary identity semantics.
 - For runtimes without the standard supervisor, validating and applying the
   canonical policy before launching the workload.
 - Forwarding the exact canonical main-process argv and TTY mode without shell
@@ -247,10 +253,10 @@ delete, reconciliation removes the row; otherwise it can remain `Deleting`.
 
 | Runtime | Best fit | Sandbox boundary | Notes |
 |---|---|---|---|
-| Docker | Local development with Docker available. | Container plus nested sandbox namespace. | Uses host networking so loopback gateway endpoints work from the supervisor. Advertises the combined-supervisor policy-DNS and transparent-TCP substrate. |
+| Docker | Local development with Docker available. | Container plus nested sandbox namespace. | Creates the workload with Docker `network_mode=none`; a host control process mediates egress and access over a private bind-mounted Unix socket. |
 | Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API and CDI GPU devices when available. Delivers the supervisor via OCI image volume by default; falls back to extracting the binary to a host-side cache and bind-mounting it when `userns` is configured (overlay does not support idmapped mounts). Advertises the combined-supervisor policy-DNS and transparent-TCP substrate. |
-| Kubernetes | Cluster deployment through Helm. | Pod plus nested sandbox namespace. | Uses Kubernetes API objects, service accounts, secrets, PVC-backed workspace storage, and GPU resources. |
-| VM | Experimental microVM isolation. | Per-sandbox libkrun VM. | Managed endpoint-backed driver. The gateway spawns `openshell-driver-vm`, waits for its Unix socket, and then consumes it through the same remote `compute_driver.proto` path used by unmanaged endpoint drivers. The VM driver boots a cached bootstrap `rootfs.ext4`, prepares requested OCI images inside a bootstrap VM with `umoci`, attaches the prepared image disk read-only, and gives each sandbox a writable `overlay.ext4` for merged-root changes and runtime material. The driver persists each accepted launch request beside the overlay and restarts those VMs on driver startup without recreating the overlay. |
+| Kubernetes | Cluster deployment through Helm. | Pod plus nested sandbox namespace. | The proxy-pod topology fences workload egress with NetworkPolicy, runs boundary mode as the workload entrypoint, and runs control mode in a separate zero-capability pod over per-boundary TLS. It requires an enforcing CNI and trusted sandbox namespace. |
+| VM | Experimental microVM isolation. | Per-sandbox libkrun VM. | The guest is NIC-less and runs boundary mode as PID 1; host control owns gateway networking and reaches the guest over vsock. The driver persists each accepted launch request and writable overlay for restart. |
 | Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. A launch-time endpoint may use a canonical built-in name to preserve its driver-config key while replacing in-process construction. The gateway connects to an operator-provisioned UDS, snapshots `GetCapabilities`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |
 
 Per-sandbox CPU and memory values currently enter the driver layer through

architecture/sandbox.md

@@ -1,8 +1,7 @@
 # Sandbox
 
-A sandbox is the runtime boundary where agent code executes. It is created by a
-compute runtime and managed inside the workload by `openshell-sandbox`, the
-sandbox supervisor.
+A sandbox is the runtime boundary where agent code executes. A compute runtime
+creates it, and `openshell-sandbox` operates it as one logical supervisor.
 
 ## Runtime Model
 
@@ -21,20 +20,52 @@ container-granted capabilities. This is fail-closed: the supervisor retains
 aborts unless the bounding set ends up empty. A `setpcap` `EPERM` is tolerated
 only when the set is already empty; any other outcome fails the spawn.
 
+Separated runtimes split the same logical supervisor into two placements:
+
+| Mode | Placement and authority |
+|---|---|
+| `openshell-sandbox --mode=control` | Runs outside the workload boundary. Owns gateway credentials, the admitted policy, RFC 0012 lifecycle, network-policy decisions, SSH, and gateway relays. |
+| `openshell-sandbox --mode=boundary --boundary-config <path>` | Runs inside or adjacent to the workload. Owns process launch and observation, exec signaling, PTY control, loopback forwarding, egress capture, and binary identity. It has no gateway credentials or independent policy authority. |
+
+The compute driver provisions the two protected configurations and their private
+Unix-socket, TLS-authenticated TCP, or vsock transport. TCP across a shared or
+operator-managed network uses a driver-provisioned trust root and verified server
+name; NetworkPolicy alone is not a confidentiality boundary. All configurations carry the
+same boundary ID, bootstrap credential, protocol version, and immutable driver resource claims.
+The shared remote backend authenticates each request and rejects a resource-claim
+mismatch. Driver crates do not appear in generic supervisor lifecycle, network,
+SSH, or session code.
+
+Control mode can expose semantic TCP readiness after the boundary reaches
+`Running`. The bind IP is explicit and address-family neutral; Kubernetes
+drivers inject the pod IP so IPv4 and IPv6 probes reach the same listener.
+
+For cross-UID Unix-socket placements, the driver-owned parent directory limits
+socket reachability and the bootstrap credential authenticates every request.
+Boundary mode can re-own a bind-mounted config as root-only before starting the
+workload when its host ownership could otherwise coincide with the workload UID.
+
 ## Startup Flow
 
 1. The compute runtime starts the workload with sandbox identity, callback
    endpoint, TLS or secret material, image metadata, and initial command.
-2. The supervisor loads policy and runtime settings from local files or the
+2. Control mode loads policy and runtime settings from local files or the
    gateway, depending on mode.
-3. It prepares filesystem access, process restrictions, network namespace
-   routing, trust stores, provider credential resolution, and inference routes.
-4. It launches the persisted canonical main-process argv and retains its PTY
-   or pipes in the main-session multiplexer.
-5. It starts the policy proxy and local SSH server.
-6. It opens a supervisor session back to the gateway for connect, exec, file
+3. The isolation backend attaches and confirms standing enforcement. Boundary
+   mode prepares boundary-local filesystem, process, and egress controls.
+4. Control mode connects network mediation, then confirms enforcement and starts
+   the admitted main process through the backend. Boundary
+   mode applies launch-time controls before its first untrusted instruction.
+5. Control mode starts the local SSH server. Exec and loopback
+   streams cross the shared boundary protocol when the modes are separated.
+6. Control mode opens a supervisor session back to the gateway for connect, exec, file
    sync, config polling, and log push.
 
+When the admitted main process exits, its status and retained terminal output
+remain available. The confirmed boundary and control-owned access plane continue
+to serve policy-authorized exec and loopback forwarding until explicit stop or
+delete tears down the boundary and terminates any remaining workload processes.
+
 ## Isolation Layers
 
 OpenShell uses overlapping controls rather than a single sandbox primitive:
@@ -109,6 +140,15 @@ generation-pinned authorization form the transparent TCP security boundary.
 Docker and Podman do not currently advertise usable IPv6 egress for this
 substrate, so AAAA queries return NOERROR/NODATA and IPv6 DNS remains fenced.
 
+Network-rule ownership depends on supervisor placement. In legacy combined
+mode, `openshell-sandbox` creates the workload network namespace and owns its
+`ip`, iptables, and nftables setup and cleanup. In delegated control/boundary
+mode, the compute driver provisions the protected network substrate before
+control attaches; boundary mode owns only boundary-local observation and the
+listener handed to it. Control mode evaluates policy but must not mutate a
+driver-owned namespace or assume that legacy `ip`/nft cleanup applies. Drivers
+must tear down their own rules and namespace resources with the workload.
+
 Provider credential placeholders are resolved through the live provider state
 for each HTTP request, after destination and L7 policy admission. A static
 credential resolves only when the request host, port, and path match an endpoint
@@ -527,7 +567,11 @@ engine with a gateway policy revision.
   to new connections or the next parsed HTTP request where the proxy can safely
   re-evaluate.
 - If the supervisor relay drops, the sandbox can keep running, but connect and
-  exec operations fail until the supervisor registers again.
+  exec operations fail until the supervisor registers again. In a separated
+  topology, a replacement control process replays the identical boundary
+  lifecycle and receives the existing process handle. The boundary rejects
+  changed launch inputs and releases the single main-process attachment when
+  the old control transport closes.
 - If the canonical main process exits, the supervisor durably reports the
   normalized result immediately. A foreground create declares a one-shot main
   attachment, so the supervisor accepts it even after a fast process exits,

crates/openshell-core/src/provider_credentials.rs

@@ -359,6 +359,17 @@ impl ProviderCredentialState {
     ///    here so SDKs can read them at startup.
     /// 3. Everything else stays as placeholders for proxy-time resolution.
     pub fn child_env_with_gcp_resolved(&self) -> HashMap<String, String> {
+        self.child_env_snapshot_with_gcp_resolved().1
+    }
+
+    /// Return the current revision and its workload-facing environment from
+    /// one state snapshot.
+    ///
+    /// Remote isolation boundaries use the pair as a revisioned update.  The
+    /// revision must describe the exact environment sent across the boundary,
+    /// so callers must not obtain the two values through separate lock
+    /// acquisitions.
+    pub fn child_env_snapshot_with_gcp_resolved(&self) -> (u64, HashMap<String, String>) {
         use crate::google_cloud;
 
         let inner = self
@@ -376,7 +387,7 @@ impl ProviderCredentialState {
             .any(|key| env.contains_key(*key) && inner.non_secret_environment_keys.contains(*key));
 
         if !has_gcp_metadata && !has_gcp_config {
-            return env;
+            return (inner.current.revision, env);
         }
 
         if has_gcp_metadata {
@@ -414,7 +425,44 @@ impl ProviderCredentialState {
             }
         }
 
-        env
+        (inner.current.revision, env)
+    }
+
+    /// Compare and install a workload-facing environment snapshot.
+    ///
+    /// Provider environment revisions are opaque content identities, not
+    /// ordered counters. The expected revision makes retries idempotent while
+    /// rejecting updates based on a stale view of the boundary state.
+    pub fn compare_and_install_child_env_snapshot(
+        &self,
+        expected_revision: u64,
+        revision: u64,
+        mut child_env: HashMap<String, String>,
+    ) -> u64 {
+        let mut inner = self
+            .inner
+            .write()
+            .expect("provider credential state poisoned");
+        if revision == inner.current.revision || expected_revision != inner.current.revision {
+            return inner.current.revision;
+        }
+
+        for key in &inner.suppressed_keys {
+            child_env.remove(key);
+        }
+        inner.current = Arc::new(ProviderCredentialSnapshot {
+            revision,
+            child_env,
+            dynamic_credentials: HashMap::new(),
+        });
+        inner.generations.clear();
+        inner.current_resolver = None;
+        inner.combined_resolver = None;
+        inner.non_secret_environment_keys.clear();
+        inner.static_credential_bindings.clear();
+        inner.known_static_credential_keys.clear();
+        inner.static_credential_identity_epochs.clear();
+        revision
     }
 
     /// Return the GCP token placeholder and its remaining lifetime in seconds.
@@ -2117,6 +2165,40 @@ mod tests {
         );
     }
 
+    #[test]
+    fn child_env_snapshot_update_uses_opaque_revision_cas() {
+        let state = ProviderCredentialState::from_child_env_snapshot(
+            4,
+            HashMap::from([("TOKEN".to_string(), "four".to_string())]),
+        );
+
+        assert_eq!(
+            state.compare_and_install_child_env_snapshot(
+                4,
+                6,
+                HashMap::from([("TOKEN".to_string(), "six".to_string())]),
+            ),
+            6
+        );
+        assert_eq!(
+            state.compare_and_install_child_env_snapshot(
+                4,
+                5,
+                HashMap::from([("TOKEN".to_string(), "stale".to_string())]),
+            ),
+            6
+        );
+        assert_eq!(
+            state.compare_and_install_child_env_snapshot(6, 2, HashMap::new()),
+            2,
+            "opaque revisions may move numerically backwards"
+        );
+
+        let (revision, env) = state.child_env_snapshot_with_gcp_resolved();
+        assert_eq!(revision, 2);
+        assert!(env.is_empty(), "an empty snapshot must revoke the old env");
+    }
+
     #[test]
     fn stale_generation_falls_back_to_current_credential_after_retention_window() {
         let state = ProviderCredentialState::from_environment(

crates/openshell-core/src/sandbox_env.rs

@@ -26,6 +26,9 @@ pub const SSH_SOCKET_PATH: &str = "OPENSHELL_SSH_SOCKET_PATH";
 /// Log level for the sandbox supervisor (e.g. `"debug"`, `"info"`, `"warn"`).
 pub const LOG_LEVEL: &str = "OPENSHELL_LOG_LEVEL";
 
+/// IP address for the control-mode semantic readiness listener.
+pub const HEALTH_BIND_IP: &str = "OPENSHELL_HEALTH_BIND_IP";
+
 /// Versioned specification for the exact canonical main process.
 ///
 /// Most drivers use JSON directly. Transports that cannot preserve spaces in
@@ -122,6 +125,13 @@ pub const TELEMETRY_ENABLED: &str = "OPENSHELL_TELEMETRY_ENABLED";
 /// `"sidecar"`; the default combined supervisor path omits it.
 pub const SUPERVISOR_TOPOLOGY: &str = "OPENSHELL_SUPERVISOR_TOPOLOGY";
 
+/// The isolation backend admitted by the deployment configuration (RFC 0012).
+///
+/// Delivered on a channel separate from the topology descriptor so descriptor
+/// verification against the admitted backend is not self-referential. Required
+/// whenever a topology descriptor is supplied.
+pub const ADMITTED_ISOLATION_BACKEND: &str = "OPENSHELL_ADMITTED_ISOLATION_BACKEND";
+
 /// Network enforcement backend selected by the compute driver.
 pub const NETWORK_ENFORCEMENT_MODE: &str = "OPENSHELL_NETWORK_ENFORCEMENT_MODE";
 
@@ -154,6 +164,17 @@ pub const GATEWAY_TLS_SERVER_NAME: &str = "OPENSHELL_GATEWAY_TLS_SERVER_NAME";
 /// by workload child processes.
 pub const PROXY_TLS_DIR: &str = "OPENSHELL_PROXY_TLS_DIR";
 
+/// Optional path to a durable PEM-encoded interception CA certificate.
+/// Must be configured together with [`PROXY_CA_KEY`].
+pub const PROXY_CA_CERT: &str = "OPENSHELL_PROXY_CA_CERT";
+
+/// Optional path to the private key for [`PROXY_CA_CERT`].
+/// Must be configured together with the certificate path.
+pub const PROXY_CA_KEY: &str = "OPENSHELL_PROXY_CA_KEY";
+
+/// Whether the control-owned SSH Unix socket is shared across trusted UIDs.
+pub const SSH_SOCKET_SHARED: &str = "OPENSHELL_SSH_SOCKET_SHARED";
+
 /// Path to the CA certificate for mTLS communication with the gateway.
 pub const TLS_CA: &str = "OPENSHELL_TLS_CA";
 

crates/openshell-isolation-interface/Cargo.toml

@@ -13,9 +13,20 @@ repository.workspace = true
 [dependencies]
 openshell-core = { path = "../openshell-core", default-features = false }
 async-trait = "0.1"
+serde = { workspace = true }
+serde_json = { workspace = true }
+thiserror = { workspace = true }
 tokio = { workspace = true }
+rustls = { workspace = true }
+rustls-pemfile = { workspace = true }
+tokio-rustls = { workspace = true }
+socket2 = { workspace = true }
+
+[target.'cfg(unix)'.dependencies]
+libc = "0.2"
 
 [dev-dependencies]
+rcgen = { workspace = true }
 tokio = { workspace = true }
 
 [lints]