feat(images): read image documents with no tarball, and pull them by name

[earakely-scale]

What

A DockerImageArtifact no longer needs an image tarball. One with no tar_gz_object_url names a registry reference in image_name, and a sandbox pulls it.

This PR makes core read such documents and pull them on VM sandboxes. Nothing writes one yet. Released readers can't parse a tarball-less document (the field was a required str), so the hub, worker and modal-proxy have to run this release before any producer ships.

How

• One predicate. DockerImageArtifact.load_problem() says why no sandbox can get an image, or None when one can. An image with a tar.gz is loaded from it. One with none is pulled by image_name, which must spell out its registry (names_registry: ghcr.io/x/y, host:5000/x, localhost/x). A bare img:v1 would silently resolve on Docker Hub, so it's refused.
• The VM loader. VmSandbox.load_docker_images refuses an unloadable image before anything runs. It loads tarballs exactly as before: the same script and the same docker images check, now pinned byte for byte by a test. It pulls the rest through pull_images. That's the docker login (for each registry the image store holds credentials for) plus docker pull with the amd64 fallback that create_container already ran. create_container now calls the same method, with the same commands.
• E2B and Sail. Their overrides move from load_docker_images to _load_tarballs. They still widen a restrictive egress policy for signed tarball downloads only. A pull goes through the policy as it is, so a restrictive policy must allow the registry itself. Registry pulls also fetch layers from hosts that vary by registry, so allowlisting the registry host alone wouldn't be enough.
• Preflight. It checks load_problem() where a VM loads the image: a gateway deploy off Modal's container gateway, an agent on a provider that puts it on a VM (not local or Modal), and a VM provider's infra. Container paths run an image by name whether or not it has a tar.gz, and Modal's container gateway swaps out service-db images the image store doesn't hold. Its file:// check no longer misreads a missing tar.gz, and an agent document the store can't read is left to the default-agent report rather than raising.
• Smaller fixes. put_tar refuses an empty tar_gz_object_url (it used to pass). load() refuses an image with no tar.gz. put_tar keeps its tar_gz_s3_url= / build_context_s3_url= keywords.

Modal container mode and the local provider's containers already used image_name alone, so they're unchanged.

Compatibility

• Tarball documents serialize exactly as before; the golden wire format is unchanged.
• A tarball-less document writes both spellings of the key as null.
• Hot path (load_docker_images, create_container): this needs the bump-version release, the bot pin, and the hg4 prod-universe smoke after rollout.

Testing

• Unit: 6,810 passed. New tests cover:
  • reading and round-tripping documents with no tarball;
  • load_problem and names_registry cases;
  • the tarball load script, pinned byte for byte;
  • pull-only loads (one login per registry, deduplicated pulls);
  • a mix of tarballs and pulls;
  • refusal before anything loads;
  • E2B and Sail leaving the policy alone for pulls;
  • preflight on every provider, and an unreadable default agent.
• Integration (local provider): docker_image_artifact_test, server_env_local_test and multi_env_local_test all passed (5).
• Live, with documents written only to a local store:

case	provider	result
agent image with no tarball, pulled by tag from a private ECR	beta_scale	deployed in 48 s, agent card served
the same image pulled by digest (repo@sha256:…)	beta_scale	deployed in 48 s
a tag that doesn't exist	beta_scale	Docker's "not found" error after 29 s; VM closed
an image name with no registry	beta_scale	refused with the new message; VM closed
one VM: a tarball image + a private ECR pull + a public pull	beta_scale	all three present after 21 s
agent image with no tarball	modal (container)	deployed in 24 s
public pull through the VM loader	local	loaded
regression: a tarball-backed agent	beta_scale	deployed in 54 s

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge based on the reviewed changes and resolved findings.

Findings

1.  Failed deployment stays open ▶

Fix with agent prompt

### Issue 1
src/agent_env/providers/sandbox_providers/sandbox.py:undefined-319
If one image pull fails quickly while another stalls, `pull_images` waits for the stalled pull before reporting the failure. `exec_script` sets no time limit on `docker pull`, so the deployment can remain stuck. Bound each pull while preserving cleanup of remote commands.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Docker image documents can now point to a registry without carrying a tarball, and VM sandboxes can pull those images by name. Bundle preflight checks whether each provider can load the images it needs.

• Docker image documents can name a registry without storing a tarball.
• VM sandboxes load archived images or pull them from their registries.
• Bundle preflight checks whether each provider can get its images.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Image artifact] --> B{Has tarball?}
  B -->|Yes| C[Load tarball on VM]
  B -->|No| D{Names a registry?}
  D -->|Yes| E[Log in if needed and pull]
  D -->|No| F[Refuse VM load]

Loading

_{Reviews (6) · Last reviewed commit: "fix(images): cancel the other pulls when..." · Reviewed by Greptile}

[earakely-scale]

Hot-path verification on beta_scale (head c30b984). Every VM deploy goes through load_docker_images, so these runs exercise the unchanged tarball path in its production shape, alongside the new pull path. All runs read the dev registry and wrote only to a local store; no model calls.

The task deploys a registry gateway env (nine MCP servers plus the gateway and service-db, all from tarballs), loads its universe, and deploys a tarball-backed agent. An env_outcome_verifier then lists the gateway's tools and calls a tool that answers from the loaded data.

case	result
one run	passed; env 126 s, load 41 s, agent 51 s (released core's runs of the same env: env 116–129 s, agent 52–57 s)
three runs at once	3/3 passed; env 123–131 s; 6 sandboxes torn down
one gateway VM loading tarballs (gateway, service-db) and pulling a server image with none	passed; env 93 s; the pulled server answered from the loaded data
one VM, three pulls, one a missing tag	the error surfaced after 17 s, once the other pulls finished; both other images present; VM terminated; no unclosed-transport or never-retrieved warnings
SIGTERM 60 s into the env deploy	exit 143, run marked cancelled. The gateway VM isn't torn down: a gateway deploy cancelled mid-way isn't recorded where teardown looks, so the VM runs until its TTL. That's a known issue, not changed by this PR

The failed-pull row was run at c30b984, which let every pull finish before raising. 5fea87d went back to cancelling the other pulls on the first failure, so a stalled pull can't hold the failure back. A cancelled exec's leftovers are the provider's to clean up: on beta_scale the exec's connection stays open (unclosed-transport warnings), and on Sail its output readers do; both are being fixed in those providers. A pull on a VM the failed deploy owns ends when the deploy closes the VM.

[greptile-apps]

Failed deployment stays open

If one image pull fails quickly while another stalls, pull_images waits for the stalled pull before reporting the failure. exec_script sets no time limit on docker pull, so the deployment can remain stuck. Bound each pull while preserving cleanup of remote commands.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/agent_env/providers/sandbox_providers/sandbox.py
Line: 319

Comment:
**Failed deployment stays open**

If one image pull fails quickly while another stalls, `pull_images` waits for the stalled pull before reporting the failure. `exec_script` sets no time limit on `docker pull`, so the deployment can remain stuck. Bound each pull while preserving cleanup of remote commands.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[earakely-scale]

Agreed. In 5fea87d the first failure cancels the other pulls again, so a stalled one can't hold it back; a test pins it with pulls that never finish. A per-pull remote timeout isn't portable: the local provider runs docker pull on a macOS host, which has no timeout. A stalled pull with nothing failing still hangs a deploy, as the existing tarball curl and create_container pull already do.

[greptile-apps]

That addresses the reported failure mode. Cancelling pending pull tasks on the first exception prevents a stalled concurrent pull from delaying failure propagation, while awaiting the cancelled tasks preserves cleanup of the local asyncio work.

I agree that a shell-level per-pull timeout would not be portable here, and an all-pulls-stalled case is consistent with the existing unbounded docker pull and tarball-download behavior rather than a regression introduced by this change. No further change is needed for this finding.