diff --git a/architecture/compute-runtimes.md b/architecture/compute-runtimes.md index a2bb25c312..d7a5635721 100644 --- a/architecture/compute-runtimes.md +++ b/architecture/compute-runtimes.md @@ -297,12 +297,15 @@ sandbox and can negate OpenShell workspace isolation and filesystem-policy controls. Driver-owned supervisor, token, and TLS bind mounts stay reserved. Network features follow the driver/substrate split. Drivers own only the outer -fence and protected channel. The sandbox owns seccomp notification, local DNS, -socket virtualization, process observation, and binary identity. The supervisor -owns DNS eligibility, policy authorization, destination filtering, upstream -dials, relay behavior, credential rewriting, and OCSF decisions. No supported -path requires nftables, a workload network namespace, proxy environment -variables, added capabilities, or an unconfined AppArmor profile. +fence and protected channel. The native Linux adapter owns seccomp notification, +local DNS, socket virtualization, process observation, and binary identity. The +Kubernetes gVisor adapter instead combines the sentry and zero-rule workload +`NetworkPolicy` with a workload-local explicit proxy whose streams cross the +protected channel. The supervisor owns DNS eligibility, policy authorization, +destination filtering, upstream dials, relay behavior, credential rewriting, +and OCSF decisions. Native mode does not require proxy environment variables; +gVisor mode injects loopback HTTP proxy variables and applies endpoint-only +policy without added capabilities or an unconfined AppArmor profile. The Kubernetes deployment packaging has two ownership boundaries. The gateway chart owns the gateway workload, configuration, Services, PKI, and diff --git a/architecture/sandbox.md b/architecture/sandbox.md index f12221fb0c..0c3ee856e5 100644 --- a/architecture/sandbox.md +++ b/architecture/sandbox.md @@ -11,8 +11,8 @@ Each sandbox has three trust levels: | Component | Role | |---|---| | Supervisor | Owns gateway credentials, admitted policy, L7 proxying, SSH, and gateway relays. It never executes inside the agent workload. | -| Sandbox | Runs as the same non-root identity as the agent, installs the workload seccomp listener, applies the Landlock baseline, owns child processes, and mediates the protected supervisor channel. | -| Agent child | Inherits the sandbox network listener and runs with zero capabilities, `no_new_privs`, Landlock, and the final syscall filter. | +| Sandbox | Runs as the same non-root identity as the agent, owns child processes, mediates the protected supervisor channel, and instantiates the driver-selected native Linux or gVisor adapter. | +| Agent child | Runs with zero capabilities behind the selected adapter. Native Linux children inherit seccomp mediation, `no_new_privs`, Landlock, and the final syscall filter; gVisor children rely on the sentry, OCI mounts, and the outer network fence. | The runtime grants neither trusted component nor agent child any Linux capability inside the workload. Drivers resolve one exact non-root UID, GID, @@ -71,8 +71,8 @@ replacement from granting authority. run untrusted code yet. 3. `openshell-supervisor` loads policy and runtime settings from the gateway, attaches to the sandbox, and verifies the driver's generation and evidence. -4. The sandbox installs its seccomp notification broker and Landlock baseline, - validates its mechanism-specific audit evidence, and reports backend-neutral +4. The sandbox starts the selected runtime adapter, validates its mechanism- + specific audit evidence, and reports backend-neutral enforcement properties. The supervisor must accept those properties and their immutable session and resource binding before it sends the launch permit. Other isolation backends may establish the same properties with @@ -131,6 +131,16 @@ OpenShell uses overlapping controls rather than a single sandbox primitive: | Driver outer fence | Docker `network_mode=none`, a NIC-less VM, or Kubernetes NetworkPolicy prevents any missed or unsupported kernel path from escaping. | | Policy proxy | Evaluates destination, binary identity, TLS/L7 rules, SSRF checks, and inference interception. | +The Kubernetes gVisor adapter uses the same authenticated lifecycle and process +backend with different enforcement mechanisms. The gVisor sentry and OCI mounts +provide the workload boundary. A zero-rule Kubernetes egress `NetworkPolicy` +blocks direct connections, while a workload-local HTTP/CONNECT listener reverse- +tunnels streams to the existing supervisor proxy. The supervisor loads policy +in endpoint-only mode because the tunnel authenticates the sandbox generation, +not an individual executable. This adapter intentionally does not claim native +Landlock path policy, nested child seccomp, transparent TCP, or per-binary +network attribution. + The supervisor may enrich baseline filesystem allowances for runtime-required paths, such as proxy support files or GPU device paths when a GPU is present. These internal allowances must stay sandbox-scoped and avoid exposing host diff --git a/crates/openshell-driver-docker/src/isolation.rs b/crates/openshell-driver-docker/src/isolation.rs index d462f78234..54e6b9ac00 100644 --- a/crates/openshell-driver-docker/src/isolation.rs +++ b/crates/openshell-driver-docker/src/isolation.rs @@ -16,8 +16,8 @@ use openshell_isolation_interface::contract::{ }; use openshell_sandbox_backend::GPU_RESOURCE_CLAIM; use openshell_sandbox_backend::boundary_protocol::{ - BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeDescriptor, - SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, + BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeAdapter, + SandboxRuntimeDescriptor, SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, }; use serde::Serialize; @@ -119,6 +119,7 @@ impl DockerBoundarySpec { resource_claims: resource_claims.clone(), resource_claim_files: BTreeMap::new(), workload_identity: self.workload_identity.clone(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: outer_fence.clone(), child_env: self.child_env, }, @@ -127,6 +128,7 @@ impl DockerBoundarySpec { generation: self.generation, session_id: self.session_id, workload_identity: self.workload_identity, + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Unix { socket_path: self.control_socket, }, diff --git a/crates/openshell-driver-kubernetes/README.md b/crates/openshell-driver-kubernetes/README.md index c24cd9c9dd..87976e4034 100644 --- a/crates/openshell-driver-kubernetes/README.md +++ b/crates/openshell-driver-kubernetes/README.md @@ -63,6 +63,16 @@ default seccomp profile. The sandbox installs a nested seccomp user-notification filter without requesting a capability in the Pod spec. Startup fails closed when the runtime blocks the required seccomp or Landlock operations. +An effective `pod.runtime_class_name` of `gvisor` selects the gVisor adapter in +the same sandbox backend. The workload Pod keeps the non-root, drop-all- +capabilities posture but omits the Kubernetes seccomp profile and custom sysctl +that GKE Sandbox does not support. The sandbox qualifies the gVisor sentry, +starts an explicit proxy on `127.0.0.1:3128`, and reverse-tunnels proxy streams +to the supervisor over the authenticated boundary protocol. The empty-egress +`NetworkPolicy` blocks direct workload connections. This mode applies endpoint- +only network policy and does not apply Landlock path rules, the nested child +seccomp filter, transparent TCP, or per-binary network attribution. + The supervisor Pod has a direct, non-controller owner reference to the Sandbox resource. This links its garbage-collection lifecycle to the sandbox without competing with the Agent Sandbox controller for workload-Pod ownership. diff --git a/crates/openshell-driver-kubernetes/src/driver.rs b/crates/openshell-driver-kubernetes/src/driver.rs index 713cb20ff3..3e45b9caea 100644 --- a/crates/openshell-driver-kubernetes/src/driver.rs +++ b/crates/openshell-driver-kubernetes/src/driver.rs @@ -1962,6 +1962,7 @@ impl KubernetesComputeDriver { return Err(fail("pair labels changed")); } let spec = pod.spec.as_ref().ok_or_else(|| fail("missing Pod spec"))?; + let gvisor = spec.runtime_class_name.as_deref() == Some("gvisor"); if spec.host_network == Some(true) || spec.host_pid == Some(true) || spec.host_ipc == Some(true) @@ -2013,10 +2014,12 @@ impl KubernetesComputeDriver { .supplemental_groups .as_deref() .is_some_and(|groups| !groups.is_empty()) - || security - .seccomp_profile - .as_ref() - .is_none_or(|profile| profile.type_ != "RuntimeDefault") + || (!gvisor + && security + .seccomp_profile + .as_ref() + .is_none_or(|profile| profile.type_ != "RuntimeDefault")) + || (gvisor && security.seccomp_profile.is_some()) { return Err(fail("numeric identity, groups, or seccomp profile changed")); } @@ -2046,9 +2049,17 @@ impl KubernetesComputeDriver { && sysctl.get("value").and_then(serde_json::Value::as_str) == Some("0") }) }); - if !unprivileged_port_sysctl { + if !gvisor && !unprivileged_port_sysctl { return Err(fail("safe unprivileged-port sysctl changed")); } + if gvisor + && pod_json + .pointer("/spec/securityContext/sysctls") + .and_then(serde_json::Value::as_array) + .is_some_and(|sysctls| !sysctls.is_empty()) + { + return Err(fail("gVisor workload must not request custom sysctls")); + } let check_container = |container: &k8s_openapi::api::core::v1::Container, name: &str| -> Result<(), KubernetesDriverError> { @@ -2265,6 +2276,16 @@ impl KubernetesComputeDriver { agent_gid, &names.sandbox_secret, )?; + let runtime_adapter = if workload_pod + .spec + .as_ref() + .and_then(|spec| spec.runtime_class_name.as_deref()) + == Some("gvisor") + { + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::Gvisor + } else { + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::NativeLinux + }; let workload_pod_uid = workload_pod.metadata.uid.clone().ok_or_else(|| { KubernetesDriverError::Message("workload Pod has no UID".to_string()) @@ -2308,6 +2329,22 @@ impl KubernetesComputeDriver { child_env.extend(spec.environment.clone()); } child_env.retain(|name, _| !name.starts_with("OPENSHELL_")); + if matches!( + runtime_adapter, + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::Gvisor + ) { + for name in ["HTTP_PROXY", "HTTPS_PROXY", "http_proxy", "https_proxy"] { + child_env.insert(name.to_string(), "http://127.0.0.1:3128".to_string()); + } + child_env.insert( + "NO_PROXY".to_string(), + "localhost,127.0.0.1,::1".to_string(), + ); + child_env.insert( + "no_proxy".to_string(), + "localhost,127.0.0.1,::1".to_string(), + ); + } let host_gateway_ip = self.config.host_gateway_ip.parse().ok(); let session_id = launch_authentication.supervisor.session_id; let tls = generate_sandbox_tls_material(session_id) @@ -2368,6 +2405,7 @@ impl KubernetesComputeDriver { }, host_gateway_ip, workload_identity, + adapter: runtime_adapter, child_env, } .provision() @@ -2471,7 +2509,7 @@ impl KubernetesComputeDriver { supervisor_uid: &str, agent_uid: u32, agent_gid: u32, - child_env: std::collections::HashMap, + mut child_env: std::collections::HashMap, launch_authentication: &openshell_core::jwt::SandboxLaunchAuthentication, ) -> Result<(), KubernetesDriverError> { let namespace_uid = Api::::all(self.client.clone()) @@ -2526,6 +2564,32 @@ impl KubernetesComputeDriver { agent_gid, &names.sandbox_secret, )?; + let runtime_adapter = if workload_pod + .spec + .as_ref() + .and_then(|spec| spec.runtime_class_name.as_deref()) + == Some("gvisor") + { + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::Gvisor + } else { + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::NativeLinux + }; + if matches!( + runtime_adapter, + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::Gvisor + ) { + for name in ["HTTP_PROXY", "HTTPS_PROXY", "http_proxy", "https_proxy"] { + child_env.insert(name.to_string(), "http://127.0.0.1:3128".to_string()); + } + child_env.insert( + "NO_PROXY".to_string(), + "localhost,127.0.0.1,::1".to_string(), + ); + child_env.insert( + "no_proxy".to_string(), + "localhost,127.0.0.1,::1".to_string(), + ); + } let workload_pod_uid = workload_pod.metadata.uid.clone().ok_or_else(|| { KubernetesDriverError::Message("workload Pod has no UID".to_string()) @@ -2607,6 +2671,7 @@ impl KubernetesComputeDriver { }, host_gateway_ip: self.config.host_gateway_ip.parse().ok(), workload_identity, + adapter: runtime_adapter, child_env, } .provision() @@ -5217,6 +5282,10 @@ fn apply_supervisor_sandbox_runtime_boundary( else { return; }; + let gvisor = spec + .get("runtimeClassName") + .and_then(serde_json::Value::as_str) + == Some("gvisor"); spec.insert("hostNetwork".to_string(), serde_json::json!(false)); spec.insert("hostPID".to_string(), serde_json::json!(false)); spec.insert("hostIPC".to_string(), serde_json::json!(false)); @@ -5240,20 +5309,22 @@ fn apply_supervisor_sandbox_runtime_boundary( ] }), ); - spec.insert( - "securityContext".to_string(), - serde_json::json!({ - "runAsUser": params.sandbox_uid, - "runAsGroup": params.sandbox_gid, - "runAsNonRoot": true, - "fsGroup": params.sandbox_gid, - "fsGroupChangePolicy": "OnRootMismatch", - "supplementalGroups": [], - "supplementalGroupsPolicy": "Strict", - "seccompProfile": {"type": "RuntimeDefault"}, - "sysctls": [{"name": "net.ipv4.ip_unprivileged_port_start", "value": "0"}] - }), - ); + let mut pod_security = serde_json::json!({ + "runAsUser": params.sandbox_uid, + "runAsGroup": params.sandbox_gid, + "runAsNonRoot": true, + "fsGroup": params.sandbox_gid, + "fsGroupChangePolicy": "OnRootMismatch", + "supplementalGroups": [], + "supplementalGroupsPolicy": "Strict" + }); + if !gvisor { + pod_security["seccompProfile"] = serde_json::json!({"type": "RuntimeDefault"}); + pod_security["sysctls"] = serde_json::json!([ + {"name": "net.ipv4.ip_unprivileged_port_start", "value": "0"} + ]); + } + spec.insert("securityContext".to_string(), pod_security); let volumes = spec .entry("volumes") .or_insert_with(|| serde_json::json!([])) @@ -8505,6 +8576,40 @@ mod tests { ); } + #[test] + fn gvisor_runtime_omits_incompatible_kernel_security_context() { + let pod_template = { + let params = SandboxPodParams { + default_runtime_class_name: "gvisor", + ..SandboxPodParams::default() + }; + sandbox_template_to_k8s( + &SandboxTemplate::default(), + false, + &std::collections::HashMap::new(), + true, + ¶ms, + ) + }; + + assert_eq!( + pod_template["spec"]["runtimeClassName"], + serde_json::json!("gvisor") + ); + assert!( + pod_template["spec"]["securityContext"]["seccompProfile"].is_null(), + "GKE Sandbox rejects Kubernetes seccomp profiles" + ); + assert!( + pod_template["spec"]["securityContext"]["sysctls"].is_null(), + "GKE Sandbox rejects custom sysctls" + ); + assert_eq!( + pod_template["spec"]["containers"][0]["securityContext"]["capabilities"]["drop"], + serde_json::json!(["ALL"]) + ); + } + #[test] fn template_runtime_class_name_overrides_config_default() { let template = SandboxTemplate { diff --git a/crates/openshell-driver-kubernetes/src/isolation.rs b/crates/openshell-driver-kubernetes/src/isolation.rs index 463924242b..b1efb4d003 100644 --- a/crates/openshell-driver-kubernetes/src/isolation.rs +++ b/crates/openshell-driver-kubernetes/src/isolation.rs @@ -24,8 +24,8 @@ use openshell_isolation_interface::contract::{ BackendError, OuterFenceGuarantee, OuterFenceGuarantees, ResolvedWorkloadIdentity, }; use openshell_sandbox_backend::boundary_protocol::{ - BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeDescriptor, - SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, + BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeAdapter, + SandboxRuntimeDescriptor, SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, }; use serde::Serialize; @@ -211,6 +211,7 @@ pub struct KubernetesSandboxRuntimeBoundarySpec { pub supervisor_tls: SandboxTlsClientConfig, pub host_gateway_ip: Option, pub workload_identity: ResolvedWorkloadIdentity, + pub adapter: SandboxRuntimeAdapter, pub child_env: HashMap, } @@ -275,6 +276,7 @@ impl KubernetesSandboxRuntimeBoundarySpec { self.workload_pod_uid_path, )]), workload_identity: self.workload_identity.clone(), + adapter: self.adapter, outer_fence: outer_fence.clone(), child_env: self.child_env, }, @@ -283,6 +285,7 @@ impl KubernetesSandboxRuntimeBoundarySpec { generation: self.generation, session_id: self.session_id, workload_identity: self.workload_identity, + adapter: self.adapter, transport: SandboxTransport::Tcp { authority: self.control_authority, addresses: vec![self.control_address], @@ -382,6 +385,7 @@ mod tests { "sandbox:sandbox-resource-uid".to_string(), ) .unwrap(), + adapter: SandboxRuntimeAdapter::NativeLinux, child_env: HashMap::new(), } } @@ -445,6 +449,22 @@ mod tests { ); } + #[test] + fn provisioning_binds_gvisor_adapter_on_both_protocol_sides() { + let mut boundary = spec(); + boundary.adapter = SandboxRuntimeAdapter::Gvisor; + let provisioned = boundary.provision().expect("provision gVisor boundary"); + + assert_eq!( + provisioned.boundary_config.adapter, + SandboxRuntimeAdapter::Gvisor + ); + assert_eq!( + provisioned.runtime_descriptor.adapter, + SandboxRuntimeAdapter::Gvisor + ); + } + #[test] fn network_fence_denies_all_workload_initiated_egress() { let fence = KubernetesSandboxRuntimeNetworkFenceSpec { diff --git a/crates/openshell-driver-podman/src/isolation.rs b/crates/openshell-driver-podman/src/isolation.rs index c7ff1a7ec7..13df78440f 100644 --- a/crates/openshell-driver-podman/src/isolation.rs +++ b/crates/openshell-driver-podman/src/isolation.rs @@ -14,8 +14,8 @@ use openshell_isolation_interface::contract::{ OuterFenceGuarantee, OuterFenceGuarantees, ResolvedWorkloadIdentity, }; use openshell_sandbox_backend::boundary_protocol::{ - BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeDescriptor, - SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, + BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeAdapter, + SandboxRuntimeDescriptor, SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, generate_sandbox_tls_material, }; use serde::{Deserialize, Serialize}; @@ -236,6 +236,7 @@ pub fn bootstrap_archives( resource_claims: resource_claims.clone(), resource_claim_files: BTreeMap::new(), workload_identity: identity.clone(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: outer_fence.clone(), child_env: child_env.clone(), }; @@ -253,6 +254,7 @@ pub fn bootstrap_archives( host_gateway_ip: None, resource_claims, workload_identity: identity.clone(), + adapter: SandboxRuntimeAdapter::default(), outer_fence, }; // Libpod resolves the requested upload destination once for a stopped diff --git a/crates/openshell-driver-vm/src/isolation/mod.rs b/crates/openshell-driver-vm/src/isolation/mod.rs index 5d3ba3cd30..577bbd5654 100644 --- a/crates/openshell-driver-vm/src/isolation/mod.rs +++ b/crates/openshell-driver-vm/src/isolation/mod.rs @@ -12,8 +12,8 @@ use openshell_isolation_interface::contract::{ BackendError, OuterFenceGuarantee, OuterFenceGuarantees, ResolvedWorkloadIdentity, }; use openshell_sandbox_backend::boundary_protocol::{ - BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeDescriptor, - SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, + BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeAdapter, + SandboxRuntimeDescriptor, SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport, }; use serde::Serialize; use std::collections::{BTreeMap, HashMap}; @@ -114,6 +114,7 @@ impl VmBoundarySpec { resource_claims: resource_claims.clone(), resource_claim_files: BTreeMap::new(), workload_identity: workload_identity.clone(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: outer_fence.clone(), child_env: self.child_env, }, @@ -122,6 +123,7 @@ impl VmBoundarySpec { generation: self.generation, session_id: self.session_id, workload_identity, + adapter: SandboxRuntimeAdapter::default(), transport: self.transport, tls: self.supervisor_tls, // The host-side control process is the network broker, so diff --git a/crates/openshell-isolation-interface/src/contract.rs b/crates/openshell-isolation-interface/src/contract.rs index f88072c857..4329deb05b 100644 --- a/crates/openshell-isolation-interface/src/contract.rs +++ b/crates/openshell-isolation-interface/src/contract.rs @@ -978,13 +978,34 @@ pub struct PendingTcpOpen { /// An `Err` means that mediation lane is unusable and fails closed. #[async_trait] pub trait NetworkMediationSource: Send + Sync { + /// Shape of the workload-side stream exposed by this backend. + fn mode(&self) -> NetworkMediationMode { + NetworkMediationMode::TransparentTcp + } + /// Await the next staged workload TCP open. async fn accept_tcp(&self) -> Result; + /// Await a raw HTTP/CONNECT proxy client stream. Backends using this mode + /// authenticate the boundary and enforce direct-egress denial externally; + /// the supervisor proxy performs endpoint-only authorization. + async fn accept_explicit_proxy(&self) -> Result { + Err(BackendError::Unsupported( + "explicit proxy streams are not supported by this backend".to_string(), + )) + } + /// Await the next workload DNS query. async fn accept_dns(&self) -> Result; } +/// Network stream shape exposed by an isolation backend. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum NetworkMediationMode { + TransparentTcp, + ExplicitProxy, +} + /// DNS transport used by one workload exchange. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] pub enum DnsTransport { diff --git a/crates/openshell-isolation-interface/src/linux/workload_launcher.rs b/crates/openshell-isolation-interface/src/linux/workload_launcher.rs index 8602d9df80..3f8976349b 100644 --- a/crates/openshell-isolation-interface/src/linux/workload_launcher.rs +++ b/crates/openshell-isolation-interface/src/linux/workload_launcher.rs @@ -26,6 +26,7 @@ type LaunchJob = Box; pub struct WorkloadLauncher { jobs: mpsc::SyncSender, alive: Arc, + native_linux_isolation: bool, } impl WorkloadLauncher { @@ -63,6 +64,12 @@ impl WorkloadLauncher { pub fn is_alive(&self) -> bool { self.alive.load(Ordering::Acquire) } + + /// Whether children need the native Landlock and seccomp launch prelude. + #[must_use] + pub const fn uses_native_linux_isolation(&self) -> bool { + self.native_linux_isolation + } } /// Start the only workload launcher and return its listener to an unfiltered @@ -108,11 +115,35 @@ pub fn start() -> io::Result<(WorkloadLauncher, NotificationListener)> { WorkloadLauncher { jobs: jobs_tx, alive, + native_linux_isolation: true, }, listener, )) } +/// Start a serialized launcher without installing a host-kernel seccomp +/// listener. This is used inside gVisor, where the sentry and the driver's +/// network fence provide the isolation boundary. +pub fn start_unfiltered() -> io::Result { + let (jobs_tx, jobs_rx) = mpsc::sync_channel::(64); + let alive = Arc::new(AtomicBool::new(true)); + let thread_alive = alive.clone(); + thread::Builder::new() + .name("openshell-workload-launcher".to_string()) + .spawn(move || { + while let Ok(job) = jobs_rx.recv() { + job(); + } + thread_alive.store(false, Ordering::Release); + }) + .map_err(|error| io::Error::other(format!("start workload launcher thread: {error}")))?; + Ok(WorkloadLauncher { + jobs: jobs_tx, + alive, + native_linux_isolation: false, + }) +} + #[cfg(test)] #[allow(unsafe_code)] mod tests { @@ -121,6 +152,14 @@ mod tests { use super::*; + #[test] + fn unfiltered_launcher_serializes_work_without_native_controls() { + let launcher = start_unfiltered().expect("start unfiltered launcher"); + assert!(!launcher.uses_native_linux_isolation()); + assert_eq!(launcher.execute(|| 42).expect("execute launch job"), 42); + assert!(launcher.is_alive()); + } + #[test] fn one_listener_mediates_launcher_and_inherited_child() { let (launcher, listener) = start().expect("start launcher"); diff --git a/crates/openshell-sandbox-backend/src/boundary_protocol.rs b/crates/openshell-sandbox-backend/src/boundary_protocol.rs index 13b87f8465..54dc852945 100644 --- a/crates/openshell-sandbox-backend/src/boundary_protocol.rs +++ b/crates/openshell-sandbox-backend/src/boundary_protocol.rs @@ -43,6 +43,20 @@ pub const STREAM_STDIN_CLOSED: u8 = 4; pub const STREAM_NETWORK_DECISION: u8 = 5; pub const MAX_STREAM_FRAME_BYTES: usize = 64 * 1024; +/// Workload-isolation mechanism used behind the shared `OpenShell` Sandbox +/// Protocol. +/// +/// The native adapter uses Linux Landlock and seccomp notification; the gVisor +/// adapter relies on the sentry boundary, the driver's outer network fence, +/// and an explicit proxy tunnel. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum SandboxRuntimeAdapter { + #[default] + NativeLinux, + Gvisor, +} + /// Capability masks measured from `/proc//status` by the `OpenShell` /// co-located runtime. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -182,6 +196,88 @@ impl NativeLinuxSandboxAuditEvidence { } } +/// Mechanism-specific evidence produced by the gVisor runtime adapter. +#[allow(clippy::struct_excessive_bools)] +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct GvisorSandboxAuditEvidence { + pub sentry_detected: bool, + pub capabilities: CapabilityEvidence, + pub sandbox_dumpable: bool, + pub core_limit_zero: bool, + pub workload_launcher_healthy: bool, + pub explicit_proxy_healthy: bool, +} + +impl GvisorSandboxAuditEvidence { + pub fn validate(&self) -> Result<(), BackendError> { + let complete = self.sentry_detected + && self.capabilities.is_empty() + && !self.sandbox_dumpable + && self.core_limit_zero + && self.workload_launcher_healthy + && self.explicit_proxy_healthy; + if complete { + Ok(()) + } else { + Err(BackendError::Confirm( + "gVisor sandbox audit evidence is incomplete".to_string(), + )) + } + } + + #[must_use] + pub fn properties(&self) -> BoundaryProperties { + BoundaryProperties { + filesystem_confinement: EnforcedProperty::new( + self.sentry_detected, + "gvisor-sentry-oci-mounts", + ), + egress_interception: EnforcedProperty::new( + self.explicit_proxy_healthy, + "authenticated-proxy-tunnel", + ), + request_attribution: EnforcedProperty::new( + self.explicit_proxy_healthy, + "authenticated-boundary-session+endpoint-policy", + ), + privilege_floor: EnforcedProperty::new( + self.sentry_detected && self.capabilities.is_empty(), + "gvisor-sentry+capability-free-container", + ), + } + } +} + +/// Tagged adapter evidence validated by the `OpenShell` runtime backend. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "adapter", content = "evidence", rename_all = "kebab-case")] +pub enum OpenShellSandboxAdapterAudit { + NativeLinux(NativeLinuxSandboxAuditEvidence), + Gvisor(GvisorSandboxAuditEvidence), +} + +impl OpenShellSandboxAdapterAudit { + pub fn validate_for(&self, adapter: SandboxRuntimeAdapter) -> Result<(), BackendError> { + match (adapter, self) { + (SandboxRuntimeAdapter::NativeLinux, Self::NativeLinux(evidence)) => { + evidence.validate() + } + (SandboxRuntimeAdapter::Gvisor, Self::Gvisor(evidence)) => evidence.validate(), + _ => Err(BackendError::Confirm( + "sandbox audit evidence does not match the selected runtime adapter".to_string(), + )), + } + } + + #[must_use] + pub fn properties(&self) -> BoundaryProperties { + match self { + Self::NativeLinux(evidence) => evidence.properties(), + Self::Gvisor(evidence) => evidence.properties(), + } + } +} + /// Ephemeral identity of the supervisor process that owns one sandbox runtime. /// /// The supervisor generates this value in memory and presents it on every @@ -414,6 +510,9 @@ pub struct SandboxRuntimeDescriptor { pub session_id: SandboxSessionId, /// Immutable numeric identity already applied to the sandbox workload. pub workload_identity: openshell_isolation_interface::contract::ResolvedWorkloadIdentity, + /// Mechanism adapter used inside the workload boundary. + #[serde(default)] + pub adapter: SandboxRuntimeAdapter, /// Driver-provisioned byte-stream endpoint. pub transport: SandboxTransport, /// Per-generation pinned TLS server identity. @@ -437,6 +536,7 @@ impl fmt::Debug for SandboxRuntimeDescriptor { .field("boundary_id", &self.boundary_id) .field("generation", &self.generation) .field("session_id", &self.session_id) + .field("adapter", &self.adapter) .field("transport", &self.transport) .field("tls", &self.tls) .field("host_gateway_ip", &self.host_gateway_ip) @@ -494,6 +594,9 @@ pub struct BoundaryConfig { pub resource_claim_files: std::collections::BTreeMap, /// Exact identity already applied by the runtime to the sandbox process. pub workload_identity: openshell_isolation_interface::contract::ResolvedWorkloadIdentity, + /// Mechanism adapter selected by the trusted compute driver. + #[serde(default)] + pub adapter: SandboxRuntimeAdapter, /// Backend-neutral projection of the driver-validated outer fence. pub outer_fence: OuterFenceGuarantees, /// Driver-resolved environment exposed only to workload processes. @@ -523,6 +626,7 @@ impl fmt::Debug for BoundaryConfig { .field("resource_claims", &self.resource_claims) .field("resource_claim_files", &self.resource_claim_files) .field("workload_identity", &self.workload_identity) + .field("adapter", &self.adapter) .field("outer_fence", &self.outer_fence) .field("child_env_keys", &self.child_env.keys().collect::>()) .finish() @@ -716,6 +820,8 @@ pub enum Request { /// plane. OpenMediation, AcceptNetwork, + /// Upgrade one authenticated stream into a raw explicit-proxy tunnel. + AcceptExplicitProxy, } impl Request { @@ -827,6 +933,7 @@ impl fmt::Debug for Request { .finish(), Self::OpenMediation => formatter.write_str("OpenMediation"), Self::AcceptNetwork => formatter.write_str("AcceptNetwork"), + Self::AcceptExplicitProxy => formatter.write_str("AcceptExplicitProxy"), } } } @@ -878,6 +985,7 @@ pub enum Response { policy_generation: u64, timing: MediationTimingWire, }, + ExplicitProxyConnected, Error { kind: BoundaryErrorKind, message: String, @@ -1380,6 +1488,37 @@ mod tests { assert!(!audit.properties().egress_interception.enforced); } + #[test] + fn gvisor_audit_requires_sentry_and_proxy() { + let mut evidence = GvisorSandboxAuditEvidence { + sentry_detected: true, + capabilities: CapabilityEvidence { + inheritable: 0, + permitted: 0, + effective: 0, + bounding: 0, + ambient: 0, + }, + sandbox_dumpable: false, + core_limit_zero: true, + workload_launcher_healthy: true, + explicit_proxy_healthy: true, + }; + evidence.validate().unwrap(); + let audit = OpenShellSandboxAdapterAudit::Gvisor(evidence.clone()); + audit.validate_for(SandboxRuntimeAdapter::Gvisor).unwrap(); + assert!(audit.properties().egress_interception.enforced); + assert!( + audit + .validate_for(SandboxRuntimeAdapter::NativeLinux) + .is_err() + ); + + evidence.explicit_proxy_healthy = false; + assert!(evidence.validate().is_err()); + assert!(!evidence.properties().egress_interception.enforced); + } + #[test] fn binary_identity_wire_rejects_ambiguous_or_invalid_shapes() { for encoded in [ diff --git a/crates/openshell-sandbox-backend/src/runtime.rs b/crates/openshell-sandbox-backend/src/runtime.rs index a1a8883883..6c2eb5098b 100644 --- a/crates/openshell-sandbox-backend/src/runtime.rs +++ b/crates/openshell-sandbox-backend/src/runtime.rs @@ -24,9 +24,9 @@ use openshell_isolation_interface::contract::{ BackendError, BoundBoundary, BoundaryDuplexStream, BoundaryExec, BoundaryExitStatus, BoundaryInput, BoundaryLoopbackConnector, BoundaryOutput, BoundaryProcess, BoundarySignal, BoundaryTerminal, ConfirmedBoundary, ExecSession, ExecSpec, IsolationBackend, LoopbackTarget, - MediationTiming, NetworkMediationSource, PendingDnsQuery, PendingTcpOpen, ProcessAttachment, - ReadyBoundary, RunningBoundary, SandboxContext, TcpOpenDecision, TcpOpenDenial, - VerifiedBackendDescriptor, + MediationTiming, NetworkMediationMode, NetworkMediationSource, PendingDnsQuery, PendingTcpOpen, + ProcessAttachment, ReadyBoundary, RunningBoundary, SandboxContext, TcpOpenDecision, + TcpOpenDenial, VerifiedBackendDescriptor, }; use sha2::{Digest as _, Sha256}; use tokio::io::{AsyncReadExt, AsyncWriteExt}; @@ -103,6 +103,7 @@ impl IsolationBackend for OpenShellRuntimeBackend { let generation = runtime_descriptor.generation.clone(); let session_id = runtime_descriptor.session_id; let outer_fence = runtime_descriptor.outer_fence.clone(); + let adapter = runtime_descriptor.adapter; let client = Arc::new(BoundaryClient::new( runtime_descriptor, self.sandbox_bearer.clone(), @@ -127,7 +128,7 @@ impl IsolationBackend for OpenShellRuntimeBackend { agent: sandbox.agent, policy: sandbox.policy, sandbox_id: sandbox.sandbox_id, - mediation: Arc::new(RemoteNetworkMediation { client }), + mediation: Arc::new(RemoteNetworkMediation { client, adapter }), host_gateway_ip, ca_file_paths: self.ca_file_paths.clone(), provider_credentials: self.provider_credentials.clone(), @@ -136,6 +137,7 @@ impl IsolationBackend for OpenShellRuntimeBackend { session_id, resource_claims, outer_fence, + adapter, })) } } @@ -281,6 +283,7 @@ struct RemoteBound { session_id: openshell_core::SandboxSessionId, resource_claims: std::collections::BTreeMap, outer_fence: openshell_isolation_interface::contract::OuterFenceGuarantees, + adapter: crate::boundary_protocol::SandboxRuntimeAdapter, } #[async_trait] @@ -308,17 +311,16 @@ impl BoundBoundary for RemoteBound { .to_string(), )); } - let audit: crate::boundary_protocol::NativeLinuxSandboxAuditEvidence = + let audit: crate::boundary_protocol::OpenShellSandboxAdapterAudit = serde_json::from_value(confirmation.backend_audit.clone()).map_err(|error| { BackendError::Confirm(format!( - "decode native Linux sandbox audit evidence: {error}" + "decode OpenShell sandbox adapter audit evidence: {error}" )) })?; - audit.validate()?; + audit.validate_for(self.adapter)?; if confirmation.properties != audit.properties() { return Err(BackendError::Confirm( - "sandbox confirmation properties do not match native Linux audit evidence" - .to_string(), + "sandbox confirmation properties do not match adapter audit evidence".to_string(), )); } let client = self.client.clone(); @@ -769,10 +771,22 @@ async fn pump_exec_input( /// head-of-line blocking during concurrent TLS handshakes. struct RemoteNetworkMediation { client: Arc, + adapter: crate::boundary_protocol::SandboxRuntimeAdapter, } #[async_trait] impl NetworkMediationSource for RemoteNetworkMediation { + fn mode(&self) -> NetworkMediationMode { + match self.adapter { + crate::boundary_protocol::SandboxRuntimeAdapter::NativeLinux => { + NetworkMediationMode::TransparentTcp + } + crate::boundary_protocol::SandboxRuntimeAdapter::Gvisor => { + NetworkMediationMode::ExplicitProxy + } + } + } + async fn accept_tcp(&self) -> Result { let (stream, response) = self.client.open_exchange(Request::AcceptNetwork).await?; let Response::NetworkConnected { @@ -805,6 +819,17 @@ impl NetworkMediationSource for RemoteNetworkMediation { }) } + async fn accept_explicit_proxy(&self) -> Result { + let (stream, response) = self + .client + .open_exchange(Request::AcceptExplicitProxy) + .await?; + if !matches!(response, Response::ExplicitProxyConnected) { + return Err(unexpected_response("explicit_proxy_connected", &response)); + } + Ok(stream) + } + async fn accept_dns(&self) -> Result { loop { let session = self.client.mediation_session().await?; @@ -1852,7 +1877,9 @@ mod tests { use std::task::{Context, Poll}; use super::*; - use crate::boundary_protocol::{ExitStatusWire, generate_sandbox_tls_material}; + use crate::boundary_protocol::{ + ExitStatusWire, SandboxRuntimeAdapter, generate_sandbox_tls_material, + }; use crate::proto::{ BoundaryChunk, isolation_boundary_server::{IsolationBoundary, IsolationBoundaryServer}, @@ -2005,6 +2032,10 @@ mod tests { kind: crate::boundary_protocol::BoundaryErrorKind::Unavailable, message: "no pending network request".to_string(), }, + Request::AcceptExplicitProxy => Response::Error { + kind: crate::boundary_protocol::BoundaryErrorKind::Unavailable, + message: "no pending explicit proxy stream".to_string(), + }, }, }) { Ok(response) => response, @@ -2378,6 +2409,7 @@ mod tests { generation: "test-generation".to_string(), session_id: test_session_id(), workload_identity: sandbox().identity, + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Tcp { authority: "sandbox.test".to_string(), addresses: vec![address], @@ -2484,7 +2516,7 @@ mod tests { } fn test_confirmation() -> openshell_isolation_interface::contract::BoundaryConfirmation { - let audit = crate::boundary_protocol::NativeLinuxSandboxAuditEvidence { + let native_audit = crate::boundary_protocol::NativeLinuxSandboxAuditEvidence { capabilities: crate::boundary_protocol::CapabilityEvidence { inheritable: 0, permitted: 0, @@ -2516,6 +2548,8 @@ mod tests { tcp_allow_round_trip: true, tcp_deny_round_trip: true, }; + let audit = + crate::boundary_protocol::OpenShellSandboxAdapterAudit::NativeLinux(native_audit); openshell_isolation_interface::contract::BoundaryConfirmation { generation: "test-generation".to_string(), identity: sandbox().identity, @@ -2563,6 +2597,7 @@ mod tests { sandbox_id: context.sandbox_id, mediation: Arc::new(RemoteNetworkMediation { client: client.clone(), + adapter: SandboxRuntimeAdapter::NativeLinux, }), host_gateway_ip: None, ca_file_paths: Arc::new(std::sync::Mutex::new(None)), @@ -2578,6 +2613,7 @@ mod tests { session_id: test_session_id(), resource_claims: std::collections::BTreeMap::new(), outer_fence, + adapter: SandboxRuntimeAdapter::NativeLinux, }; assert!(matches!( @@ -2594,9 +2630,14 @@ mod tests { #[tokio::test] async fn remote_confirm_rejects_invalid_native_audit_before_monitoring() { let mut confirmation = test_confirmation(); - let mut audit: crate::boundary_protocol::NativeLinuxSandboxAuditEvidence = + let mut audit: crate::boundary_protocol::OpenShellSandboxAdapterAudit = serde_json::from_value(confirmation.backend_audit.clone()).expect("decode test audit"); - audit.seccomp.notification_round_trip = false; + let crate::boundary_protocol::OpenShellSandboxAdapterAudit::NativeLinux(native) = + &mut audit + else { + panic!("test confirmation must contain native Linux evidence"); + }; + native.seccomp.notification_round_trip = false; confirmation.backend_audit = serde_json::to_value(audit).expect("encode test audit"); assert_remote_confirmation_rejected(confirmation).await; @@ -2641,6 +2682,7 @@ mod tests { generation: "test-generation".to_string(), session_id: test_session_id(), workload_identity: sandbox().identity, + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Unix { socket_path: PathBuf::from("/tmp/vsock.sock"), }, @@ -2661,6 +2703,7 @@ mod tests { generation: "test-generation".to_string(), session_id: test_session_id(), workload_identity: sandbox().identity, + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Unix { socket_path: PathBuf::from("/tmp/vsock.sock"), }, @@ -2682,6 +2725,7 @@ mod tests { generation: "test-generation".to_string(), session_id: test_session_id(), workload_identity: sandbox().identity, + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Tcp { authority: "sandbox.test".to_string(), addresses: vec!["0.0.0.0:5500".parse().expect("valid address")], @@ -2704,6 +2748,7 @@ mod tests { generation: "test-generation".to_string(), session_id: test_session_id(), workload_identity: sandbox().identity, + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Tcp { authority: "sandbox.test".to_string(), addresses: vec!["10.42.0.7:5500".parse().expect("valid address")], @@ -2898,6 +2943,7 @@ mod tests { generation: "test-generation".to_string(), session_id: test_session_id(), workload_identity: sandbox().identity, + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Tcp { authority: "sandbox.test".to_string(), addresses: vec![address], @@ -2986,6 +3032,7 @@ mod tests { generation: "test-generation".to_string(), session_id: test_session_id(), workload_identity: context.identity.clone(), + adapter: SandboxRuntimeAdapter::default(), transport: SandboxTransport::Unix { socket_path: socket_path.clone(), }, diff --git a/crates/openshell-sandbox/src/boundary_exec.rs b/crates/openshell-sandbox/src/boundary_exec.rs index 57d9778a6f..63d54c6216 100644 --- a/crates/openshell-sandbox/src/boundary_exec.rs +++ b/crates/openshell-sandbox/src/boundary_exec.rs @@ -92,7 +92,9 @@ impl LocalBoundaryExec { command.env(key, value); } } - crate::process::strip_proxy_env_std(&mut command); + if self.launcher.uses_native_linux_isolation() { + crate::process::strip_proxy_env_std(&mut command); + } for (key, value) in &spec.env { if !key.starts_with("OPENSHELL_") { command.env(key, value); @@ -109,6 +111,9 @@ impl LocalBoundaryExec { &self, workdir: Option<&str>, ) -> Result, BackendError> { + if !self.launcher.uses_native_linux_isolation() { + return Ok(None); + } crate::sandbox::linux::log_sandbox_readiness(&self.policy, workdir); let runtime_read_only = crate::process::ca_runtime_read_only_paths(self.ca_file_paths.as_deref()); @@ -127,9 +132,14 @@ impl LocalBoundaryExec { #[cfg(target_os = "linux")] let prepared = self.prepare_sandbox(effective_workdir)?; #[cfg(target_os = "linux")] - let child_hardening = - openshell_isolation_interface::linux::child_seccomp::prepare(std::process::id()) - .map_err(|error| BackendError::Process(error.to_string()))?; + let child_hardening = self + .launcher + .uses_native_linux_isolation() + .then(|| { + openshell_isolation_interface::linux::child_seccomp::prepare(std::process::id()) + }) + .transpose() + .map_err(|error| BackendError::Process(error.to_string()))?; crate::pty::install_dedicated_process_group(&mut command); crate::pty::install_pre_exec_no_pty( &mut command, @@ -238,9 +248,14 @@ impl LocalBoundaryExec { #[cfg(target_os = "linux")] let prepared = self.prepare_sandbox(effective_workdir)?; #[cfg(target_os = "linux")] - let child_hardening = - openshell_isolation_interface::linux::child_seccomp::prepare(std::process::id()) - .map_err(|error| BackendError::Process(error.to_string()))?; + let child_hardening = self + .launcher + .uses_native_linux_isolation() + .then(|| { + openshell_isolation_interface::linux::child_seccomp::prepare(std::process::id()) + }) + .transpose() + .map_err(|error| BackendError::Process(error.to_string()))?; crate::pty::install_pre_exec( &mut command, self.policy.clone(), diff --git a/crates/openshell-sandbox/src/boundary_server.rs b/crates/openshell-sandbox/src/boundary_server.rs index 86e9e1973d..9eba118349 100644 --- a/crates/openshell-sandbox/src/boundary_server.rs +++ b/crates/openshell-sandbox/src/boundary_server.rs @@ -59,11 +59,13 @@ mod linux { use openshell_sandbox_backend::boundary_protocol::{ AgentSpecWire, BinaryIdentityWire, BoundaryConfig, BoundaryErrorKind, BoundaryListener as BoundaryListenerConfig, DnsQueryResultWire, ExecSpecWire, - ExitStatusWire, MediationTimingWire, NativeLinuxSandboxAuditEvidence, OutputWindowWire, + ExitStatusWire, GvisorSandboxAuditEvidence, MediationTimingWire, + NativeLinuxSandboxAuditEvidence, OpenShellSandboxAdapterAudit, OutputWindowWire, ProcessKindWire, ProcessSnapshotWire, Request, RequestEnvelope, Response, ResponseEnvelope, STREAM_EXIT, STREAM_NETWORK_DECISION, STREAM_STDERR, STREAM_STDIN, STREAM_STDIN_CLOSED, - STREAM_STDOUT, SandboxPolicyWire, SessionSnapshotWire, SignalWire, encode_frame, - read_frame, read_stream_frame, validate_resource_claims, write_frame, write_stream_frame, + STREAM_STDOUT, SandboxPolicyWire, SandboxRuntimeAdapter, SessionSnapshotWire, SignalWire, + encode_frame, read_frame, read_stream_frame, validate_resource_claims, write_frame, + write_stream_frame, }; const CONTROL_IO_TIMEOUT: Duration = Duration::from_secs(30); @@ -194,6 +196,9 @@ mod linux { let config: BoundaryConfig = serde_json::from_slice(&bytes).map_err(|error| { format!("decode boundary config {}: {error}", config_path.display()) })?; + if config.adapter != qualification.adapter { + return Err("runtime qualification does not match boundary adapter".to_string()); + } validate_config(&config)?; validate_runtime_resource_claims(&config)?; validate_running_identity( @@ -211,20 +216,38 @@ mod linux { unsafe { std::env::set_var(openshell_core::sandbox_env::USER_ENVIRONMENT, child_env); } - crate::sandbox::apply_supervisor_startup_hardening() - .map_err(|error| format!("install sandbox process prelude: {error}"))?; + if matches!(config.adapter, SandboxRuntimeAdapter::NativeLinux) { + crate::sandbox::apply_supervisor_startup_hardening() + .map_err(|error| format!("install sandbox process prelude: {error}"))?; + } if nix::unistd::getpid().as_raw() == 1 { crate::managed_children::start_orphan_reaper() .map_err(|error| format!("start sandbox orphan reaper: {error}"))?; } - let (launcher, listener) = openshell_isolation_interface::linux::workload_launcher::start() - .map_err(|error| format!("start sandbox workload launcher: {error}"))?; - let protected_control_port = match &config.listener { - BoundaryListenerConfig::TlsTcp { address, .. } => Some(address.port()), - BoundaryListenerConfig::Unix { .. } | BoundaryListenerConfig::Vsock { .. } => None, + let (launcher, network_broker) = match config.adapter { + SandboxRuntimeAdapter::NativeLinux => { + let (launcher, listener) = + openshell_isolation_interface::linux::workload_launcher::start() + .map_err(|error| format!("start sandbox workload launcher: {error}"))?; + let protected_control_port = match &config.listener { + BoundaryListenerConfig::TlsTcp { address, .. } => Some(address.port()), + BoundaryListenerConfig::Unix { .. } | BoundaryListenerConfig::Vsock { .. } => { + None + } + }; + let broker = NetworkBroker::start(listener, protected_control_port) + .map_err(|error| format!("start sandbox network broker: {error}"))?; + (launcher, broker) + } + SandboxRuntimeAdapter::Gvisor => { + let launcher = + openshell_isolation_interface::linux::workload_launcher::start_unfiltered() + .map_err(|error| format!("start gVisor workload launcher: {error}"))?; + let broker = NetworkBroker::start_explicit_proxy() + .map_err(|error| format!("start gVisor explicit proxy: {error}"))?; + (launcher, broker) + } }; - let network_broker = NetworkBroker::start(listener, protected_control_port) - .map_err(|error| format!("start sandbox network broker: {error}"))?; let process_runtime = tokio::runtime::Builder::new_multi_thread() .enable_all() .build() @@ -1229,6 +1252,38 @@ mod linux { })?; return Ok(()); } + Request::AcceptExplicitProxy => { + let broker = runtime.network_accept_context()?; + if !broker.is_explicit_proxy() { + return Err("explicit proxy requested from native network adapter".to_string()); + } + let request_id = request.request_id; + runtime.process_runtime.block_on(async move { + let target = broker.accept_explicit_proxy().await.map_err(|error| { + format!("accept sandbox explicit proxy stream: {error}") + })?; + target.set_nonblocking(true).map_err(|error| { + format!("set explicit proxy stream nonblocking: {error}") + })?; + let mut target = tokio::net::TcpStream::from_std(target) + .map_err(|error| format!("register explicit proxy stream: {error}"))?; + let mut stream = stream.into_tokio()?; + let response = encode_frame(&ResponseEnvelope { + request_id, + response: Response::ExplicitProxyConnected, + }) + .map_err(|error| format!("encode explicit proxy response: {error}"))?; + stream + .write_all(&response) + .await + .map_err(|error| format!("write explicit proxy response: {error}"))?; + tokio::io::copy_bidirectional(&mut stream, &mut target) + .await + .map(|_| ()) + .map_err(|error| format!("bridge explicit proxy stream: {error}")) + })?; + return Ok(()); + } _ => {} } let supervisor_instance_id = match &request.request { @@ -1883,7 +1938,8 @@ mod linux { | Request::TerminateBoundary | Request::AttachProcess { .. } | Request::LoopbackConnect { .. } - | Request::AcceptNetwork => guest_error( + | Request::AcceptNetwork + | Request::AcceptExplicitProxy => guest_error( BoundaryErrorKind::Invalid, "streaming request used on control path", ), @@ -2276,22 +2332,36 @@ mod linux { } // SAFETY: successful getrlimit initialized the value. let core_limit = unsafe { core_limit.assume_init() }; - let (native_architecture, kernel_release) = uname_values()?; - let audit = NativeLinuxSandboxAuditEvidence { - capabilities, - no_new_privileges, - sandbox_dumpable, - child_dumpable: true, - core_limit_zero: core_limit.rlim_cur == 0 && core_limit.rlim_max == 0, - native_architecture, - kernel_release, - seccomp: self.qualification.seccomp, - landlock_abi: self.qualification.landlock_abi, - landlock_allow_deny: self.qualification.landlock_allow_deny, - udp_dns_round_trip: self.qualification.udp_dns_round_trip, - tcp_dns_round_trip: self.qualification.tcp_dns_round_trip, - tcp_allow_round_trip: self.qualification.tcp_allow_round_trip, - tcp_deny_round_trip: self.qualification.tcp_deny_round_trip, + let audit = match self.config.adapter { + SandboxRuntimeAdapter::NativeLinux => { + let (native_architecture, kernel_release) = uname_values()?; + OpenShellSandboxAdapterAudit::NativeLinux(NativeLinuxSandboxAuditEvidence { + capabilities, + no_new_privileges, + sandbox_dumpable, + child_dumpable: true, + core_limit_zero: core_limit.rlim_cur == 0 && core_limit.rlim_max == 0, + native_architecture, + kernel_release, + seccomp: self.qualification.seccomp, + landlock_abi: self.qualification.landlock_abi, + landlock_allow_deny: self.qualification.landlock_allow_deny, + udp_dns_round_trip: self.qualification.udp_dns_round_trip, + tcp_dns_round_trip: self.qualification.tcp_dns_round_trip, + tcp_allow_round_trip: self.qualification.tcp_allow_round_trip, + tcp_deny_round_trip: self.qualification.tcp_deny_round_trip, + }) + } + SandboxRuntimeAdapter::Gvisor => { + OpenShellSandboxAdapterAudit::Gvisor(GvisorSandboxAuditEvidence { + sentry_detected: self.qualification.gvisor_sentry_detected, + capabilities, + sandbox_dumpable, + core_limit_zero: core_limit.rlim_cur == 0 && core_limit.rlim_max == 0, + workload_launcher_healthy: self.workload_launcher.is_alive(), + explicit_proxy_healthy: self.network_broker.is_explicit_proxy(), + }) + } }; // The boundary reports mechanism evidence; the authenticated host // backend validates it before constructing a ConfirmedBoundary. @@ -3649,6 +3719,7 @@ mod linux { resource_claims: std::collections::BTreeMap::new(), resource_claim_files: std::collections::BTreeMap::new(), workload_identity: test_workload_identity(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: test_outer_fence(), child_env: std::collections::HashMap::new(), }; @@ -3800,6 +3871,7 @@ mod linux { resource_claims: std::collections::BTreeMap::new(), resource_claim_files: std::collections::BTreeMap::new(), workload_identity: test_workload_identity(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: test_outer_fence(), child_env: std::collections::HashMap::new(), }, @@ -4220,6 +4292,8 @@ mod linux { fn test_runtime_qualification() -> crate::RuntimeQualification { crate::RuntimeQualification { + adapter: SandboxRuntimeAdapter::default(), + gvisor_sentry_detected: false, seccomp: openshell_sandbox_backend::boundary_protocol::SeccompEvidence { new_listener: true, notification_round_trip: true, @@ -4323,6 +4397,7 @@ mod linux { resource_claims: std::collections::BTreeMap::new(), resource_claim_files: std::collections::BTreeMap::new(), workload_identity: test_workload_identity(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: test_outer_fence(), child_env: std::collections::HashMap::new(), }; @@ -4358,6 +4433,7 @@ mod linux { pod_uid_path, )]), workload_identity: test_workload_identity(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: test_outer_fence(), child_env: std::collections::HashMap::new(), }; @@ -4398,6 +4474,7 @@ mod linux { resource_claims: std::collections::BTreeMap::new(), resource_claim_files: std::collections::BTreeMap::new(), workload_identity: test_workload_identity(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: test_outer_fence(), child_env: std::collections::HashMap::new(), }, @@ -4573,6 +4650,7 @@ mod linux { resource_claims: std::collections::BTreeMap::new(), resource_claim_files: std::collections::BTreeMap::new(), workload_identity: test_workload_identity(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: test_outer_fence(), child_env: std::collections::HashMap::new(), }, @@ -4602,7 +4680,11 @@ mod linux { boundary.attach(policy.clone()), Response::Attached { .. } )); - assert!(matches!(boundary.confirm(), Response::Confirmed { .. })); + let confirmation = boundary.confirm(); + assert!( + matches!(confirmation, Response::Confirmed { .. }), + "unexpected confirmation response: {confirmation:?}" + ); let start = || { boundary.start_agent( "sandbox-reconnect".to_string(), @@ -4675,7 +4757,11 @@ mod linux { boundary.attach(policy.clone()), Response::Attached { .. } )); - assert!(matches!(boundary.confirm(), Response::Confirmed { .. })); + let confirmation = boundary.confirm(); + assert!( + matches!(confirmation, Response::Confirmed { .. }), + "unexpected confirmation response: {confirmation:?}" + ); assert_eq!( start(), Response::Started { @@ -4846,6 +4932,7 @@ mod linux { resource_claims: std::collections::BTreeMap::new(), resource_claim_files: std::collections::BTreeMap::new(), workload_identity: test_workload_identity(), + adapter: SandboxRuntimeAdapter::default(), outer_fence: test_outer_fence(), child_env: std::collections::HashMap::new(), }, diff --git a/crates/openshell-sandbox/src/delegated.rs b/crates/openshell-sandbox/src/delegated.rs index e294f81ac0..487a65eac6 100644 --- a/crates/openshell-sandbox/src/delegated.rs +++ b/crates/openshell-sandbox/src/delegated.rs @@ -74,7 +74,9 @@ pub async fn spawn_workload( .ok() .and_then(|json| serde_json::from_str(&json).ok()) .unwrap_or_default(); - user_environment.retain(|key, _value| !crate::process::is_proxy_env_var(key)); + if launcher.uses_native_linux_isolation() { + user_environment.retain(|key, _value| !crate::process::is_proxy_env_var(key)); + } let loopback_connector: Arc = Arc::new( crate::boundary_io::LocalLoopbackConnector::new(Some(boundary_runtime.clone())), ); diff --git a/crates/openshell-sandbox/src/lib.rs b/crates/openshell-sandbox/src/lib.rs index 4166d8d2c1..aa4c7c616a 100644 --- a/crates/openshell-sandbox/src/lib.rs +++ b/crates/openshell-sandbox/src/lib.rs @@ -34,6 +34,8 @@ pub mod sandbox; reason = "qualification preserves independently exercised security results" )] pub struct RuntimeQualification { + pub adapter: openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter, + pub gvisor_sentry_detected: bool, pub seccomp: openshell_sandbox_backend::boundary_protocol::SeccompEvidence, pub landlock_abi: u32, pub landlock_allow_deny: bool, diff --git a/crates/openshell-sandbox/src/main.rs b/crates/openshell-sandbox/src/main.rs index dbdc8733ec..6e55ceeee6 100644 --- a/crates/openshell-sandbox/src/main.rs +++ b/crates/openshell-sandbox/src/main.rs @@ -8,7 +8,7 @@ use std::mem::size_of; use std::path::Path; use clap::Parser; -use miette::{IntoDiagnostic, Result}; +use miette::{IntoDiagnostic, Result, WrapErr as _}; #[cfg(target_os = "linux")] use openshell_ocsf::OcsfShorthandLayer; #[cfg(target_os = "linux")] @@ -239,6 +239,8 @@ fn qualify_runtime() -> Result<(openshell_sandbox::RuntimeQualification, Qualifi wait_killable_recv: notification.wait_killable_recv, }; let qualification = openshell_sandbox::RuntimeQualification { + adapter: openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::NativeLinux, + gvisor_sentry_detected: false, seccomp: openshell_sandbox_backend::boundary_protocol::SeccompEvidence { new_listener: notification.notification_round_trip(), notification_round_trip: notification.notification_round_trip(), @@ -1847,7 +1849,50 @@ fn run_boundary(bootstrap: &Path, log_level: &str) -> Result<()> { .with_filter(console_filter), ) .try_init(); - let (qualification, _) = qualify_runtime()?; + let config_bytes = std::fs::read(bootstrap) + .into_diagnostic() + .wrap_err_with(|| format!("read boundary config {}", bootstrap.display()))?; + let config: openshell_sandbox_backend::boundary_protocol::BoundaryConfig = + serde_json::from_slice(&config_bytes) + .into_diagnostic() + .wrap_err("decode boundary config for runtime adapter selection")?; + let qualification = match config.adapter { + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::NativeLinux => { + qualify_runtime()?.0 + } + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::Gvisor => { + let version = std::fs::read_to_string("/proc/version") + .into_diagnostic() + .wrap_err("read /proc/version for gVisor qualification")?; + let sentry_detected = version.to_ascii_lowercase().contains("gvisor"); + if !sentry_detected { + return Err(miette::miette!( + "gVisor runtime adapter selected but the gVisor sentry was not detected" + )); + } + openshell_sandbox::RuntimeQualification { + adapter: config.adapter, + gvisor_sentry_detected: true, + seccomp: openshell_sandbox_backend::boundary_protocol::SeccompEvidence { + new_listener: false, + notification_round_trip: false, + id_validation: false, + addfd_send: false, + retained_socket_operation: false, + proc_fd_identity: false, + task_memory_read: false, + task_memory_write: false, + cancellation: false, + }, + landlock_abi: 0, + landlock_allow_deny: false, + udp_dns_round_trip: false, + tcp_dns_round_trip: false, + tcp_allow_round_trip: false, + tcp_deny_round_trip: false, + } + } + }; openshell_sandbox::run(bootstrap, qualification) } diff --git a/crates/openshell-sandbox/src/network_broker.rs b/crates/openshell-sandbox/src/network_broker.rs index 6b01cc0967..b8a93bc4af 100644 --- a/crates/openshell-sandbox/src/network_broker.rs +++ b/crates/openshell-sandbox/src/network_broker.rs @@ -198,10 +198,12 @@ struct NotificationQueues { /// Live broker handle retained by the sandbox boundary. #[derive(Clone)] pub struct NetworkBroker { - _accept_monitor: Arc, + _accept_monitor: Option>, pending: Arc>>, pending_dns: Arc>>, + pending_proxy: Option>>>, dns_address: SocketAddr, + proxy_address: Option, healthy: Arc, } @@ -298,14 +300,68 @@ impl NetworkBroker { }) .map_err(|error| io::Error::other(format!("start network broker: {error}")))?; Ok(Self { - _accept_monitor: accept_monitor, + _accept_monitor: Some(accept_monitor), pending: Arc::new(tokio::sync::Mutex::new(pending_rx)), pending_dns: Arc::new(tokio::sync::Mutex::new(pending_dns_rx)), + pending_proxy: None, dns_address, + proxy_address: None, healthy, }) } + /// Start the workload-local HTTP/CONNECT listener used by the gVisor + /// adapter. Accepted byte streams are reverse-tunnelled to the existing + /// supervisor proxy over authenticated Sandbox Protocol connections. + pub(crate) fn start_explicit_proxy() -> io::Result { + Self::start_explicit_proxy_at(SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128)) + } + + fn start_explicit_proxy_at(address: SocketAddr) -> io::Result { + let listener = TcpListener::bind(address)?; + let proxy_address = listener.local_addr()?; + let (_pending_tx, pending_rx) = mpsc::channel(1); + let (_dns_tx, pending_dns_rx) = mpsc::channel(1); + let (proxy_tx, proxy_rx) = mpsc::channel(OPEN_QUEUE_CAPACITY); + let healthy = Arc::new(AtomicBool::new(true)); + let broker_healthy = healthy.clone(); + std::thread::Builder::new() + .name("openshell-explicit-proxy".to_string()) + .spawn(move || { + for accepted in listener.incoming() { + match accepted { + Ok(stream) => { + let _ = stream.set_nodelay(true); + if proxy_tx.blocking_send(stream).is_err() { + break; + } + } + Err(error) if error.kind() == io::ErrorKind::Interrupted => {} + Err(error) => { + tracing::error!(%error, "sandbox explicit proxy listener failed"); + break; + } + } + } + broker_healthy.store(false, Ordering::Release); + }) + .map_err(|error| io::Error::other(format!("start explicit proxy listener: {error}")))?; + Ok(Self { + _accept_monitor: None, + pending: Arc::new(tokio::sync::Mutex::new(pending_rx)), + pending_dns: Arc::new(tokio::sync::Mutex::new(pending_dns_rx)), + pending_proxy: Some(Arc::new(tokio::sync::Mutex::new(proxy_rx))), + dns_address: SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 0), + proxy_address: Some(proxy_address), + healthy, + }) + } + + #[cfg(test)] + fn start_explicit_proxy_for_test() -> io::Result { + Self::start_explicit_proxy_at(SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 0)) + } + pub(crate) async fn accept(&self) -> io::Result { self.pending .lock() @@ -324,13 +380,30 @@ impl NetworkBroker { .ok_or_else(|| io::Error::new(io::ErrorKind::BrokenPipe, "DNS broker queue closed")) } + pub(crate) async fn accept_explicit_proxy(&self) -> io::Result { + let pending = self.pending_proxy.as_ref().ok_or_else(|| { + io::Error::new( + io::ErrorKind::Unsupported, + "network broker is not in explicit-proxy mode", + ) + })?; + pending + .lock() + .await + .recv() + .await + .ok_or_else(|| io::Error::new(io::ErrorKind::BrokenPipe, "explicit proxy queue closed")) + } + #[cfg(test)] pub(crate) fn dns_address(&self) -> SocketAddr { self.dns_address } pub(crate) fn confirm_healthy(&self) -> io::Result<()> { - if self.healthy.load(Ordering::Acquire) && self.dns_address.port() != 0 { + if self.healthy.load(Ordering::Acquire) + && (self.dns_address.port() != 0 || self.proxy_address.is_some()) + { Ok(()) } else { Err(io::Error::new( @@ -339,6 +412,16 @@ impl NetworkBroker { )) } } + + #[must_use] + pub(crate) const fn is_explicit_proxy(&self) -> bool { + self.proxy_address.is_some() + } + + #[cfg(test)] + fn explicit_proxy_address(&self) -> Option { + self.proxy_address + } } fn start_dns_relay( @@ -1754,6 +1837,30 @@ mod tests { use std::io::{Read as _, Write as _}; use std::os::unix::net::{UnixListener, UnixStream}; + #[tokio::test] + async fn explicit_proxy_accepts_raw_client_streams() { + let broker = + NetworkBroker::start_explicit_proxy_for_test().expect("start explicit proxy listener"); + let address = broker + .explicit_proxy_address() + .expect("explicit proxy address"); + let client = tokio::task::spawn_blocking(move || { + let mut stream = TcpStream::connect(address).expect("connect explicit proxy"); + stream + .write_all(b"CONNECT example.com:443 HTTP/1.1\r\n\r\n") + .expect("write proxy request"); + }); + let mut accepted = broker + .accept_explicit_proxy() + .await + .expect("accept explicit proxy stream"); + let mut request = [0_u8; 44]; + let length = accepted.read(&mut request).expect("read proxy request"); + assert!(request[..length].starts_with(b"CONNECT example.com:443")); + client.await.expect("proxy client task"); + broker.confirm_healthy().expect("healthy explicit proxy"); + } + #[test] fn relay_rejects_descriptor_replaced_after_policy_decision() { let metadata = SocketMetadata { diff --git a/crates/openshell-sandbox/src/process.rs b/crates/openshell-sandbox/src/process.rs index 56d4f244ad..588c1d2036 100644 --- a/crates/openshell-sandbox/src/process.rs +++ b/crates/openshell-sandbox/src/process.rs @@ -578,7 +578,9 @@ impl ProcessHandle { cmd.current_dir(dir); } - strip_proxy_env(&mut cmd); + if launcher.uses_native_linux_isolation() { + strip_proxy_env(&mut cmd); + } // Set TLS trust store env vars so sandbox processes trust the ephemeral CA if let Some((ca_cert_path, combined_bundle_path)) = ca_paths { @@ -591,20 +593,31 @@ impl ProcessHandle { // process where the tracing subscriber is functional. The child's // pre_exec context cannot reliably emit structured logs. #[cfg(target_os = "linux")] - sandbox::linux::log_sandbox_readiness(policy, workspace.root()); + if launcher.uses_native_linux_isolation() { + sandbox::linux::log_sandbox_readiness(policy, workspace.root()); + } // Prepare the Landlock ruleset as the workload UID. Inaccessible paths // are already unavailable to the child and remain omitted. #[cfg(target_os = "linux")] let runtime_read_only = ca_runtime_read_only_paths(ca_paths); - let prepared_sandbox = prepare_child_sandbox(policy, workspace.root(), &runtime_read_only) - .map_err(|err| miette::miette!("Failed to prepare sandbox: {err}"))?; + let prepared_sandbox = if launcher.uses_native_linux_isolation() { + prepare_child_sandbox(policy, workspace.root(), &runtime_read_only) + .map_err(|err| miette::miette!("Failed to prepare sandbox: {err}"))? + } else { + None + }; #[cfg(target_os = "linux")] - let mut child_hardening = - openshell_isolation_interface::linux::child_seccomp::prepare(std::process::id()) - .map_err(|error| { - miette::miette!("prepare child self-protection filter: {error}") - })?; + let mut child_hardening = if launcher.uses_native_linux_isolation() { + Some( + openshell_isolation_interface::linux::child_seccomp::prepare(std::process::id()) + .map_err(|error| { + miette::miette!("prepare child self-protection filter: {error}") + })?, + ) + } else { + None + }; // Set up process group for signal handling (non-interactive mode only). // In interactive mode, we inherit the parent's process group to maintain // proper terminal control for shells and interactive programs. @@ -627,14 +640,16 @@ impl ProcessHandle { return Err(std::io::Error::last_os_error()); } - harden_child_process().map_err(|err| std::io::Error::other(err.to_string()))?; - // Phase 2 (as unprivileged user): Enforce the prepared // Landlock ruleset via restrict_self() + apply seccomp. // restrict_self() does not require root. #[cfg(target_os = "linux")] - if let Some(prepared) = prepared_sandbox.take() { - sandbox::linux::enforce_capability_free(prepared, &mut child_hardening) + if let (Some(prepared), Some(child_hardening)) = + (prepared_sandbox.take(), child_hardening.as_mut()) + { + harden_child_process() + .map_err(|err| std::io::Error::other(err.to_string()))?; + sandbox::linux::enforce_capability_free(prepared, child_hardening) .map_err(|err| std::io::Error::other(err.to_string()))?; } diff --git a/crates/openshell-sandbox/src/pty.rs b/crates/openshell-sandbox/src/pty.rs index d887342306..0c77a52861 100644 --- a/crates/openshell-sandbox/src/pty.rs +++ b/crates/openshell-sandbox/src/pty.rs @@ -57,8 +57,9 @@ pub fn install_pre_exec( _workdir: Option, slave_fd: RawFd, #[cfg(target_os = "linux")] prepared: Option, - #[cfg(target_os = "linux")] - child_hardening: openshell_isolation_interface::linux::child_seccomp::ChildHardeningProgram, + #[cfg(target_os = "linux")] child_hardening: Option< + openshell_isolation_interface::linux::child_seccomp::ChildHardeningProgram, + >, ) -> anyhow::Result<()> { #[cfg(target_os = "linux")] let mut prepared = prepared; @@ -75,7 +76,7 @@ pub fn install_pre_exec( #[cfg(target_os = "linux")] prepared.take(), #[cfg(target_os = "linux")] - &mut child_hardening, + child_hardening.as_mut(), ) }); } @@ -92,8 +93,9 @@ pub fn install_pre_exec_no_pty( policy: SandboxPolicy, _workdir: Option, #[cfg(target_os = "linux")] prepared: Option, - #[cfg(target_os = "linux")] - child_hardening: openshell_isolation_interface::linux::child_seccomp::ChildHardeningProgram, + #[cfg(target_os = "linux")] child_hardening: Option< + openshell_isolation_interface::linux::child_seccomp::ChildHardeningProgram, + >, ) -> anyhow::Result<()> { #[cfg(target_os = "linux")] let mut prepared = prepared; @@ -111,7 +113,7 @@ pub fn install_pre_exec_no_pty( #[cfg(target_os = "linux")] prepared.take(), #[cfg(target_os = "linux")] - &mut child_hardening, + child_hardening.as_mut(), ) }); } @@ -121,14 +123,14 @@ pub fn install_pre_exec_no_pty( fn enter_sandbox( policy: &SandboxPolicy, #[cfg(target_os = "linux")] prepared: Option, - #[cfg(target_os = "linux")] - child_hardening: &mut openshell_isolation_interface::linux::child_seccomp::ChildHardeningProgram, + #[cfg(target_os = "linux")] child_hardening: Option< + &mut openshell_isolation_interface::linux::child_seccomp::ChildHardeningProgram, + >, ) -> std::io::Result<()> { - crate::process::harden_child_process() - .map_err(|error| std::io::Error::other(error.to_string()))?; - #[cfg(target_os = "linux")] - if let Some(prepared) = prepared { + if let (Some(prepared), Some(child_hardening)) = (prepared, child_hardening) { + crate::process::harden_child_process() + .map_err(|error| std::io::Error::other(error.to_string()))?; crate::sandbox::linux::enforce_capability_free(prepared, child_hardening) .map_err(|error| std::io::Error::other(error.to_string()))?; } diff --git a/crates/openshell-supervisor-network/src/proxy.rs b/crates/openshell-supervisor-network/src/proxy.rs index 46c8f69cea..c6e7fbe656 100644 --- a/crates/openshell-supervisor-network/src/proxy.rs +++ b/crates/openshell-supervisor-network/src/proxy.rs @@ -32,7 +32,8 @@ use openshell_core::provider_credentials::{ProviderCredentialSnapshot, ProviderC use openshell_core::secrets::{self, SecretResolver, rewrite_header_line_checked}; use openshell_isolation_interface::contract::{ BinaryIdentity as ContractBinaryIdentity, BoundaryDuplexStream, MediationTiming, - NetworkMediationSource, PendingTcpOpen, ResolveError, TcpOpenDecision, TcpOpenDenial, + NetworkMediationMode, NetworkMediationSource, PendingTcpOpen, ResolveError, TcpOpenDecision, + TcpOpenDenial, }; use openshell_ocsf::{ ActionId, ActivityId, DispositionId, Endpoint, HttpActivityBuilder, HttpRequest, HttpResponse, @@ -348,13 +349,28 @@ impl ProxyHandle { } } - let mut network_accepts = network_mediation_source.as_ref().map(|source| { - let accepts = FuturesUnordered::new(); - for _ in 0..MEDIATION_ACCEPT_WINDOW { - let source = source.clone(); - accepts.push(async move { source.accept_tcp().await }.boxed()); - } - accepts + let mediation_mode = network_mediation_source + .as_ref() + .map(|source| source.mode()); + let mut network_accepts = network_mediation_source.as_ref().and_then(|source| { + (source.mode() == NetworkMediationMode::TransparentTcp).then(|| { + let accepts = FuturesUnordered::new(); + for _ in 0..MEDIATION_ACCEPT_WINDOW { + let source = source.clone(); + accepts.push(async move { source.accept_tcp().await }.boxed()); + } + accepts + }) + }); + let mut explicit_accepts = network_mediation_source.as_ref().and_then(|source| { + (source.mode() == NetworkMediationMode::ExplicitProxy).then(|| { + let accepts = FuturesUnordered::new(); + for _ in 0..MEDIATION_ACCEPT_WINDOW { + let source = source.clone(); + accepts.push(async move { source.accept_explicit_proxy().await }.boxed()); + } + accepts + }) }); // Transparent opens require policy evaluation and destination // validation before the sandbox may complete connect(2). Keep @@ -368,40 +384,52 @@ impl ProxyHandle { let mut consecutive_unknown_errors: u32 = 0; loop { let accepted = if let Some(source) = network_mediation_source.as_ref() { - let accepts = network_accepts - .as_mut() - .expect("mediation source has an accept window"); - tokio::select! { - pending = accepts.next() => { - let pending = pending.expect("accept window is never empty"); - let source = source.clone(); - accepts.push(async move { source.accept_tcp().await }.boxed()); - match pending { - Ok(connection) => { - let tx = preauthorized_tx.clone(); - let dns_store = policy_dns_store.clone(); - let opa = opa_engine.clone(); - let backend_gateway = *backend_host_gateway; - let trusted_gateway = *trusted_host_gateway; - tokio::spawn(async move { - if let Some(connection) = preauthorize_transparent_open( - connection, - dns_store.as_ref(), - &opa, - backend_gateway, - trusted_gateway, - ) - .await - { - let _ = tx.send(connection).await; - } - }); - continue; + if mediation_mode == Some(NetworkMediationMode::ExplicitProxy) { + let accepts = explicit_accepts + .as_mut() + .expect("explicit mediation source has an accept window"); + let pending = accepts.next().await.expect("accept window is never empty"); + let source = source.clone(); + accepts.push(async move { source.accept_explicit_proxy().await }.boxed()); + pending + .map(|stream| (stream, None, None, None)) + .map_err(ProxyAcceptError::Source) + } else { + let accepts = network_accepts + .as_mut() + .expect("transparent mediation source has an accept window"); + tokio::select! { + pending = accepts.next() => { + let pending = pending.expect("accept window is never empty"); + let source = source.clone(); + accepts.push(async move { source.accept_tcp().await }.boxed()); + match pending { + Ok(connection) => { + let tx = preauthorized_tx.clone(); + let dns_store = policy_dns_store.clone(); + let opa = opa_engine.clone(); + let backend_gateway = *backend_host_gateway; + let trusted_gateway = *trusted_host_gateway; + tokio::spawn(async move { + if let Some(connection) = preauthorize_transparent_open( + connection, + dns_store.as_ref(), + &opa, + backend_gateway, + trusted_gateway, + ) + .await + { + let _ = tx.send(connection).await; + } + }); + continue; + } + Err(error) => Err(ProxyAcceptError::Source(error)), } - Err(error) => Err(ProxyAcceptError::Source(error)), } + Some(connection) = preauthorized_rx.recv() => Ok(connection), } - Some(connection) = preauthorized_rx.recv() => Ok(connection), } } else { let listener = listener diff --git a/crates/openshell-supervisor-network/src/run.rs b/crates/openshell-supervisor-network/src/run.rs index 024185d1ac..a4019bf477 100644 --- a/crates/openshell-supervisor-network/src/run.rs +++ b/crates/openshell-supervisor-network/src/run.rs @@ -38,7 +38,7 @@ use crate::opa::OpaEngine; use crate::policy_local::PolicyLocalContext; use crate::proxy::ProxyHandle; use openshell_core::endpoint_status::EndpointObservationSender; -use openshell_isolation_interface::contract::NetworkMediationSource; +use openshell_isolation_interface::contract::{NetworkMediationMode, NetworkMediationSource}; #[cfg(target_os = "linux")] pub struct TransparentRuntimeSetup { @@ -436,7 +436,10 @@ pub async fn run_networking( (None, None) }; - let mediated_policy_dns = if let Some(source) = network_mediation_source.clone() { + let mediated_policy_dns = if let Some(source) = network_mediation_source + .clone() + .filter(|source| source.mode() == NetworkMediationMode::TransparentTcp) + { let engine = opa_engine .cloned() .ok_or_else(|| miette::miette!("Mediated DNS requires an OPA engine"))?; diff --git a/crates/openshell-supervisor/src/lib.rs b/crates/openshell-supervisor/src/lib.rs index e05da07067..fd3efedbd5 100644 --- a/crates/openshell-supervisor/src/lib.rs +++ b/crates/openshell-supervisor/src/lib.rs @@ -462,6 +462,7 @@ pub async fn run_network_proxy( } let extension_credentials = openshell_extension_core::ExtensionCredentialStore::new(); + let (mut policy, opa_engine, _, _, _, initial_agent_proposals_enabled, _) = load_policy( None, None, @@ -608,6 +609,20 @@ pub async fn run_sandbox( // and the policy poll loop that rotates them stay the same objects. let extension_credentials = openshell_extension_core::ExtensionCredentialStore::new(); + let selected_runtime_adapter = serde_json::from_slice::< + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeDescriptor, + >(&backend_descriptor.payload) + .map_err(|error| miette::miette!("decode sandbox runtime descriptor: {error}"))? + .adapter; + let local_policy_identity = match selected_runtime_adapter { + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::NativeLinux => { + LocalPolicyIdentity::Required + } + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::Gvisor => { + LocalPolicyIdentity::EndpointOnly + } + }; + // Load policy and initialize OPA engine let openshell_endpoint_for_proxy = openshell_endpoint.clone(); let sandbox_name_for_agg = sandbox.clone(); @@ -626,7 +641,7 @@ pub async fn run_sandbox( policy_rules, policy_data, &extension_credentials, - LocalPolicyIdentity::Required, + local_policy_identity, ) .await?; @@ -806,8 +821,11 @@ pub async fn run_sandbox( info!(backend = %admitted_backend_name, "Isolation boundary attached"); let remote_boundary = (bound, admitted_backend_name, ca_file_paths); - let transparent_tcp_capable = true; - let transparent_tcp_substrate_ready = true; + let transparent_tcp_capable = matches!( + selected_runtime_adapter, + openshell_sandbox_backend::boundary_protocol::SandboxRuntimeAdapter::NativeLinux + ); + let transparent_tcp_substrate_ready = transparent_tcp_capable; // The denial channel is owned by the orchestrator: the proxy (in the // networking leaf) and the bypass monitor (in the process leaf) both // produce DenialEvents that the denial aggregator (orchestrator-side) diff --git a/docs/reference/sandbox-compute-drivers.mdx b/docs/reference/sandbox-compute-drivers.mdx index d24ff67ef4..f52e8bf143 100644 --- a/docs/reference/sandbox-compute-drivers.mdx +++ b/docs/reference/sandbox-compute-drivers.mdx @@ -463,6 +463,31 @@ the agent. One namespace-wide, empty-egress `NetworkPolicy` is the mandatory outer fence for all OpenShell workload Pods. It permits supervisor Pods to reach sandbox listeners; TLS and JWT identity enforce the exact pairing. +Set the effective Kubernetes `runtimeClassName` to `gvisor` to select the +gVisor adapter inside the same OpenShell sandbox backend: + +```shell +openshell sandbox create \ + --driver-config-json '{"kubernetes":{"pod":{"runtime_class_name":"gvisor"}}}' \ + --name gvisor-agent +``` + +The gVisor adapter omits the Pod seccomp profile and custom unprivileged-port +sysctl because GKE Sandbox rejects those settings. It replaces nested Landlock +and seccomp-notify setup with gVisor sentry qualification. The mandatory empty- +egress `NetworkPolicy` remains the direct-egress fence. OpenShell injects +`HTTP_PROXY` and `HTTPS_PROXY` for the workload, accepts proxy traffic on +`127.0.0.1:3128`, and reverse-tunnels it over the authenticated sandbox channel +to the existing supervisor proxy. + +This adapter provides endpoint-only network policy. It does not provide +per-binary network attribution, transparent TCP interception, Landlock path +allowlists, or OpenShell's nested child seccomp filter. Applications must honor +the HTTP proxy environment and use HTTP or CONNECT-compatible transports; +direct DNS and non-proxy TCP remain blocked by the workload `NetworkPolicy`. +The adapter fails confirmation unless it detects gVisor, observes a healthy +proxy tunnel, and verifies the zero-rule Kubernetes egress fence. + The Kubernetes driver creates namespaced `agents.x-k8s.io` `Sandbox` resources from the Kubernetes SIG Apps [agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox) project. It detects the served Sandbox API at runtime, caches the selected API version for the gateway process, and uses `v1beta1` when available before falling back to `v1alpha1`, so supported Agent Sandbox installations work without version-specific operator configuration. The Agent Sandbox controller turns those resources into sandbox pods and related storage. Stop patches the existing resource rather than deleting it. For `v1beta1`, diff --git a/skills/debug-openshell-cluster/SKILL.md b/skills/debug-openshell-cluster/SKILL.md index a6b54ec677..6c28fb0ac5 100644 --- a/skills/debug-openshell-cluster/SKILL.md +++ b/skills/debug-openshell-cluster/SKILL.md @@ -625,8 +625,10 @@ supervisor Pods to reach sandbox TLS listeners. The driver then creates a per-sandbox Service, split immutable bootstrap Secrets, and a gated supervisor Pod before releasing either Pod. The supervisor Pod runs `/openshell-supervisor`. Both Pods use the -same resolved non-root identity, request no capabilities, drop `ALL`, disable -privilege escalation, and use `RuntimeDefault` seccomp. The supervisor reaches +same resolved non-root identity, request no capabilities, drop `ALL`, and disable +privilege escalation. Native workload Pods use `RuntimeDefault` seccomp. An +effective `runtimeClassName: gvisor` selects the gVisor adapter and deliberately +omits the Pod seccomp profile and custom sysctl that GKE Sandbox rejects. The supervisor reaches the sandbox over per-sandbox TLS with server-certificate verification plus bootstrap-token client authentication, and owns gateway policy, provider credentials, DNS, and mediated upstream connections. @@ -654,6 +656,14 @@ a required unprivileged seccomp, task-memory, or Landlock operation. Do not add capabilities, gateway egress, or credentials to the workload Pod as a workaround. +For a gVisor workload, confirmation instead requires `/proc/version` to report +gVisor, a healthy workload-local proxy on `127.0.0.1:3128`, and the observed +zero-rule workload egress fence. Verify the workload environment contains +`HTTP_PROXY` and `HTTPS_PROXY` with that loopback address. Direct DNS and raw +TCP are expected to fail: the adapter supports HTTP/CONNECT-aware applications +and endpoint-only policy, not native transparent interception or per-binary +network rules. + #### Corporate upstream proxy When the deployment routes sandbox egress through a corporate HTTP forward