feat(sandbox): add a Sail Sailbox VM sandbox provider

[psinghal20]

Summary

Adds sail, a built-in VM sandbox provider for Sail Research Sailboxes (type = "sail", mode = "vm"), alongside modal_vm and e2b. Each Sailbox boots from Sail's devbox image, which ships Docker and Compose v2 and runs as root. That means the existing docker-in-VM flows work unchanged through VmSandbox: gateway docker-compose, agent docker run, deploy_sandbox, image builds and artifact collection. The gateway routes it to _deploy_via_vm with no provider-specific branch.

[sandbox.providers.sail.config]
api_key = "secret:sail_api_key"   # required; env:SAIL_API_KEY for local dev
app = "agent-env"                 # optional: the Sail App every Sailbox belongs to
min_size = "s"                    # optional: smallest size to pick (s|m|l)
auto_sleep = false                # optional: default false (never sleep)
# auto_sleep_min_idle_seconds = 600   # optional, 1-3600; turns autosleep on
# runtime_threads = 16                # optional, sets SAIL_RUNTIME_THREADS (1-256)
inject_model_key = true           # optional: default true, Sail injects the agent's model key

Select it with --sandbox sail or [sandbox] default = "sail".

What's in it

• sail/_sdk.py: imports the SDK lazily (import agent_env doesn't load it) and installs the API key, described in the call-outs below.
• sail/sandbox.py (SailSandbox):
  • Exec: separate stdout and stderr, exact bytes, async-iterable so collect_artifacts streams. Output is queued with bounded backpressure, and output nobody reads is drained by wait(). A leading sudo is dropped, a lost host or transport gives exit -1 so exec_script retries, and other stream errors are raised rather than reported as success.
  • Docker readiness: polls docker info and starts dockerd once if it isn't running. Compose v2 is required.
  • Host file writes: go through Sail's native filesystem API instead of base64 chunks.
  • Downloads under an allowlist: before an image or object download, the signed-URL hosts are added to the box's egress policy. This happens under a per-Sailbox lock with a fresh read of the applied policy, and is skipped when the cached policy already admits the hosts. It fails closed on an unknown policy and refuses to exceed Sail's 128 entries.
• sail/provider.py (SailSandboxProvider):
  • Sizing: picks the smallest size covering the CPU request (s/m/l = 1/4/8 vCPU). Memory and disk become ceilings, rounded up into that size's range. A request no size fits is refused before provisioning.
  • Lifetime: timeout becomes the box's max_lifetime_seconds.
  • Ports: public *.sail.box listeners. Create waits until every exposed port is routed.
  • Egress policies: allow-all → {}; an allowlist → {"allowlist": hosts + IPv4 CIDRs}. IPv6 entries and more than 128 entries are refused before provisioning. EGRESS_HOSTS = ("*.sail.box",).
  • Reconnect: get_sandbox restores the tunnel URLs from listeners and the applied egress policy. An unrepresentable policy becomes None (fail closed).
  • Create failures: a failed status, or a failure during setup or port routing, terminates the box.
  • create_container: reuses the inherited login → pull → run, then deletes /root/.docker/config.json.
• Registration: _BUILTIN_SANDBOX_PROVIDERS, the package exports, the CLI --sandbox help, the AGENTS.md provider list, sail~=0.12.8 in pyproject.toml (the lock adds sail and cloudpickle), and THIRD_PARTY_NOTICES.md.

The agent's model key never enters the Sailbox

Sail's credential injection adds the key to the agent's model requests as they leave the Sailbox. The Sailbox disk (which Sail checkpoints, see below) and its memory hold only a placeholder. This lives entirely in sail/model_key.py and the Sail provider; no core code changes.

• Secret named by the key. create_sandbox reads LITELLM_API_KEY and LITELLM_BASE_URL from the agent's env. It stores the key as a Sail secret named AGENTENV_LITELLM_<sha256(key)[:32]>, so one key maps to one secret and different keys never share one.
• Saved policy applied at create. A saved egress policy is attached when the Sailbox is created, so it's in force before the agent starts. Its rules set authorization: Bearer ${secrets.…} and x-api-key: ${secrets.…} on requests to the endpoint's host. A restricted allowlist also gets that host.
• Placeholder only inside. Every command and file write from that SailSandbox replaces the key's value with sail-injected-model-key, so docker run -e and docker exec -e (agent, judge, unit-test verifier) carry only the placeholder.
• Containers trust Sail's CA. Sail terminates TLS for rule hosts with its own CA, which is in the VM's trust store but not in containers'. A docker shim at /usr/local/bin/docker gives every container it runs or creates the VM's CA bundle, plus SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS and CURL_CA_BUNDLE.
• Lifecycle.
  • Widening the allowlist for downloads replaces the saved policy and keeps the rules; inline documents can't name secrets.
  • Terminate deletes the policy, then the secret. If another Sailbox's policy still names the secret, Sail refuses (SecretInUseError) and we leave it in place.
  • A failed or cancelled create deletes the policy and the secret, and a failed policy swap deletes the replacement.
  • Reconnect restores the injection from the applied policy. A handle reconnected in the process that deployed the agent recovers the agent's key from process memory (never persisted). In another process, it recovers the configured [model] key, if that is the injected one. Either way, the reconnected handle keeps scrubbing the key.
• Limits.
  • The endpoint must be HTTPS; an http one is refused before anything is created, unless inject_model_key = false.
  • Only the agent's own key is scrubbed. A different key sent later into the same box (for example a verifier configured with another key) is not.
  • The shim covers docker run / docker create on the VM. Containers started by an agent's own Docker-in-Docker don't get the CA.
  • Gateway, deploy_sandbox and other create_vm boxes carry no model key and don't inject.

Call-outs for reviewers

• The Sail API key is written to os.environ. This is the first core provider to do that. The Sail Python SDK has no client that takes a key explicitly; it reads SAIL_API_KEY once, when it builds its process-wide client. _sdk.connect:

  • sets the configured key under a lock;
  • makes the one synchronous App.find call that builds the client;
  • restores the previous value straight away, about 0.4s later.

  A different key in the same process raises ConfigError (one Sail key per process). An operator's own SAIL_API_KEY is overridden only for that build, then restored. The key is never passed to workloads, exec env, CONTAINER_ENV, REQUEST_HEADERS, logs or repr. A local-sandbox subprocess started during that window could inherit it, but the local provider already runs as the same user with the full host environment, so this adds no exposure. SAIL_API_KEY and SAIL_RUNTIME_THREADS are added to the env-var guard's not-configuration set: agent-env only writes them for the SDK and never reads them as configuration.

• Sail checkpoints every Sailbox automatically. A box with autosleep off was checkpointed about 2.5 minutes after creation with nothing on our side triggering it, and there is no setting to turn this off. The model key is kept out (above), and registry credentials are deleted after create_container's pull. Other values deployments put in container env still land on disk and in checkpoints:

  • object-store credentials when the S3 store has share_credentials = true: frozen host AWS credentials, so keep this off on Sail or use a scoped, short-lived role;
  • MCP server extra_env_vars;
  • an external env-state database URL.

  Retention, whether checkpoints include memory, and deletion on terminate are open questions for Sail.

• A cancelled create is cleaned up. If the caller is cancelled while a create is in flight (for example by ChainedSandboxProvider's 180s deadline), the box it produces is terminated: three attempts with backoff, then an error log naming the box. Otherwise it would leak a billed Sailbox.

• Attribution is not on the Sailbox. Sail has no labels or tags. Attribution goes in the box name (ae-<random>-<slugged values in key order>, ≤128 chars) and in a structured agent_env.sail_sandbox_started log event keyed by sailbox_id, the same pattern as Modal's modal_sandbox_started. Per-box cost comes from Sail's spend API joined on sailbox_id.

• Autosleep is off by default (AutoSleep.never()), so idle MCP servers and gateways don't sleep or come back cold. Config can turn it on.

• Resource requests round up. Requests are ceilings on Sail (billing is for observed usage), so CPU rounds up to a size and memory and disk round up into the size's range. This raises a cap, not the bill.

• SAIL_MODE is not supported. Core must not read a stage variable, so the provider uses Sail's production endpoint.

• Lazy SDK import. _sdk.connect imports sail inside the function, the same as E2B's provider does, so the Rust core loads only when Sail is used.

Testing

• Unit: make unit-test gives 6338 passed, 13 skipped, with the SDK faked and nothing offline touching the network. New tests are sail_sandbox_provider_test.py, sail_sandbox_test.py and sail_model_key_test.py (secret naming, the rules, the shim's argument rewriting, create, scrubbing, policy replacement, cleanup order and reconnect). sail is also added to the network-policy stamping test, the egress-host floor, config interpolation, the capability gate and the gateway VM-path routing test.
• Integration (int_test_slow):
  • New sail_sandbox_smoke_test.py: Docker in the box, nginx served through a *.sail.box URL, an exact binary write round-trip, reconnect restoring ports and policy, and an allowlist enforced for containers. A third test checks model-key injection end to end: a container's request reaches a header-echo endpoint with the real key, while the container's env and docker inspect show only the placeholder, and the secret is gone after terminate.
  • sail is added to the sandbox_provider parametrisations in gateway_test.py and task_steps_test.py.
  • With no Sail config, these skip with agentenv-capability-missing: remote_sandbox.
• Run live against Sail (sail==0.12.8, a local key, on the final code):
  • smoke tests: 3 passed;
  • agent-env run hello --sandbox sail: passed;
  • gateway_test.py::test_gateway_with_multi_mcp_server[sail]: passed in about 1m40s on the Sailbox, covering two MCP servers, a universe load, MCP tool calls and /step;
  • no Sailboxes were left running afterwards.
• Not run live: task_steps_test[sail] (needs a model endpoint), autosleep staying off through a long idle period, and how promptly a box expires at its max lifetime.

🤖 Generated with Claude Code

Confidence Score: 4/5

The PR does not appear safe to merge until concurrent launches cannot delete each other’s model-key secret.

Findings

1.  Failed launch deletes shared key ▶
2.  Old model keys remain ▶

Fix with agent prompt

### Issue 1
src/agent_env/providers/sandbox_providers/sail/provider.py:267-270
When two agent launches use the same `LITELLM_API_KEY`, they share a Sail secret. If one launch fails while the other is still creating its policy, this cleanup can delete the shared secret. The second agent may fail to start or lose access to its model. Keep the secret until no launch is creating or using a policy that needs it.

### Issue 2
src/agent_env/providers/sandbox_providers/sail/model_key.py:84-85
`for_env` adds each injected key to `_injected_keys`, but ending a box or failing to create one does not remove it. A long-running worker keeps old agent keys in memory and adds another entry for each distinct key. Remove entries when they are no longer needed for reconnects.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR adds Sail Sailboxes as a built-in VM sandbox provider for agents, gateways, and other Docker-based work. It also lets Sail inject an agent’s model key at the HTTPS endpoint, keeping the real key out of commands and files sent to the box.

• Sail now runs Docker-based work in its own VM sandboxes.
• Sail adds an agent’s model key to HTTPS requests without storing it in the VM.
• Gateway trajectory reads retry when a transport failure cuts off an exec.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Create agent box] --> B[Set shared model-key secret]
  B --> C[Create saved egress policy]
  C --> D[Create Sailbox]
  C --> E[Failed launch cleanup]
  E --> F[Delete policy, then shared secret]

Loading

_{Reviews (5) · Last reviewed commit: "fix(sail): address greptile review findi..." · Reviewed by Greptile}

[greptile-apps]

Failed launch deletes shared key

When two agent launches use the same LITELLM_API_KEY, they share a Sail secret. If one launch fails while the other is still creating its policy, this cleanup can delete the shared secret. The second agent may fail to start or lose access to its model. Keep the secret until no launch is creating or using a policy that needs it.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/agent_env/providers/sandbox_providers/sail/provider.py
Line: 267-270

Comment:
**Failed launch deletes shared key**

When two agent launches use the same `LITELLM_API_KEY`, they share a Sail secret. If one launch fails while the other is still creating its policy, this cleanup can delete the shared secret. The second agent may fail to start or lose access to its model. Keep the secret until no launch is creating or using a policy that needs it.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Old model keys remain

for_env adds each injected key to _injected_keys, but ending a box or failing to create one does not remove it. A long-running worker keeps old agent keys in memory and adds another entry for each distinct key. Remove entries when they are no longer needed for reconnects.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/agent_env/providers/sandbox_providers/sail/model_key.py
Line: 84-85

Comment:
**Old model keys remain**

`for_env` adds each injected key to `_injected_keys`, but ending a box or failing to create one does not remove it. A long-running worker keeps old agent keys in memory and adds another entry for each distinct key. Remove entries when they are no longer needed for reconnects.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.