Skip to content

refactor(sandbox)!: default to Alpine and drop community image resolution - #3236

Closed
akram wants to merge 7 commits into
NVIDIA:mainfrom
akram:refactor/sandbox-alpine-default
Closed

akram wants to merge 7 commits into
NVIDIA:mainfrom
akram:refactor/sandbox-alpine-default

Conversation

@akram

@akram akram commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

First code increment toward #3116 (retire community images, default to Alpine): change the default sandbox image to a generic official Alpine image, remove the built-in community catalog resolution, make the container drivers work on a USER-less base image, and point the deployment configs at Alpine. Also adds OpenShift in-cluster build tooling. Upstream Dockerfiles and explicit image-reference behavior are preserved.

Related Issue

Part of #3116. Depends on #2750 for proxy-mode networking on a bare image (see Follow-up).

Changes

  • feat(sandbox): default_sandbox_image()docker.io/library/alpine:3.22, inherited by all compute drivers.
  • refactor(cli)!: remove DEFAULT_COMMUNITY_REGISTRY, resolve_community_image, OPENSHELL_COMMUNITY_REGISTRY; explicit --from OCI references pass through. BREAKING.
  • refactor(policy): drop the community-only /app path from the default policy.
  • feat(driver): Docker/Podman assign a numeric non-root identity (1000) for images without an OCI USER (like K8s/VM), instead of rejecting with OCI USER is required.
  • feat(deploy): Helm values, standalone K8s manifest, gateway.toml, and dev task scripts default to Alpine.
  • build(docker): multi-stage Dockerfile.{supervisor,gateway}.multistage reproducing the upstream Nix build inside OpenShift/Buildah.

Testing

  • Both images built in-cluster on OpenShift (amd64); boot verified.
  • CreateSandbox with no image → docker.io/library/alpine:3.22.
  • k8s driver (agent-sandbox controller): default pod starts; init + supervisor run on bare Alpine via the synthesized numeric identity.
  • podman driver, real SAW VM: our gateway in a KubeVirt SAW VM (driver=podman) starts an Alpine sandbox — supervisor boots (no OCI USER is required), interoperates with existing supervisors. Fails only later on proxy-mode networking (see below).
  • cargo test green for modified crates.

Follow-up

@copy-pr-bot

copy-pr-bot Bot commented Sep 9, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@akram akram closed this Sep 9, 2026
@akram akram reopened this Sep 9, 2026
@akram
akram marked this pull request as draft September 9, 2026 10:29
@akram
akram force-pushed the refactor/sandbox-alpine-default branch from 1344ed3 to e3e2878 Compare September 10, 2026 16:21
@akram akram changed the title refactor(docker)!: retire community images, build Alpine supervisor and Debian gateway in OpenShell build(docker): add OpenShift in-cluster build variants for supervisor and gateway images Sep 10, 2026
@akram
akram force-pushed the refactor/sandbox-alpine-default branch 6 times, most recently from ceb6dcc to 813092f Compare September 10, 2026 20:19
@akram akram changed the title build(docker): add OpenShift in-cluster build variants for supervisor and gateway images refactor(sandbox)!: default to Alpine and drop community image resolution Sep 11, 2026
@akram
akram force-pushed the refactor/sandbox-alpine-default branch 3 times, most recently from d355c19 to 78d68c0 Compare September 15, 2026 19:10
… and gateway

Add multi-stage Dockerfiles that build the OpenShell supervisor and gateway
images entirely inside an OpenShift/Buildah cluster, for environments without
the upstream Nix CI pipeline that stages prebuilt binaries under
deploy/docker/.build/prebuilt-binaries.

Both reproduce the exact upstream artifacts by running the project's own Nix
devShells in a builder stage, then assembling a runtime stage identical to the
existing Dockerfile.supervisor / Dockerfile.gateway:

- Dockerfile.supervisor.multistage: builds the static musl openshell-sandbox
  binary via the musl devShell; runtime is alpine:3.22 with nftables/iptables
  and COPY --chmod=0555.
- Dockerfile.gateway.multistage: builds openshell-gateway via the glibc-2-28
  devShell, normalizes the ELF interpreter with patchelf and asserts z3 is
  statically embedded; runtime is distroless cc-debian13.

Each builder collapses build and cleanup into a single RUN so the Nix store
never enters the committed layer, keeping the intermediate commit small and
within the node's ephemeral-storage budget. The upstream Dockerfiles and CI
binary pipeline are unchanged.

Signed-off-by: Akram
Signed-off-by: Akram <akram.benaissi@gmail.com>
default_sandbox_image() now returns docker.io/library/alpine:3.22, a generic
version-qualified official image, so a fresh install no longer depends on the
community sandbox image catalog. All compute drivers (docker, podman,
kubernetes, vm) inherit this fallback.

Part of NVIDIA#3116.

Signed-off-by: Akram
Signed-off-by: Akram <akram.benaissi@gmail.com>
The restrictive default policy granted read-only access to /app, a directory
that only existed in the community base image. A generic Alpine default has no
/app, so remove it. Landlock best-effort already ignores absent paths; this
just stops advertising a community-specific layout in the default.

Part of NVIDIA#3116.

Signed-off-by: Akram
Signed-off-by: Akram <akram.benaissi@gmail.com>
…ox image

Update the shared gateway default_image, Helm chart values, the standalone
Kubernetes manifest, and the dev gateway task scripts to use
docker.io/library/alpine:3.22 instead of the community base image, consistent
with default_sandbox_image(). GPU e2e image-build base is left unchanged (CUDA
needs a glibc base).

Part of NVIDIA#3116.

Signed-off-by: Akram
Signed-off-by: Akram <akram.benaissi@gmail.com>
Remove DEFAULT_COMMUNITY_REGISTRY, resolve_community_image, and the
OPENSHELL_COMMUNITY_REGISTRY override. Bare --from values are no longer
expanded into the OpenShell Community registry; the CLI and TUI now pass
explicit OCI image references through to the gateway unchanged. The
openshell-core image module is reduced to default_sandbox_image().

BREAKING CHANGE: community sandbox shorthand names and OPENSHELL_COMMUNITY_REGISTRY
are no longer supported; pass a full OCI image reference to --from.

Part of NVIDIA#3116.

Signed-off-by: Akram <akram.benaissi@gmail.com>
With the default sandbox image now Alpine, images that declare no OCI USER
must start instead of being rejected. When the image declares no USER and
the policy requests none, the Podman and Docker drivers now supply a numeric
non-root identity (DEFAULT_SANDBOX_UID/GID = 1000) instead of rejecting,
matching the numeric-identity behavior of the Kubernetes and VM drivers. The
supervisor's resolved-identity path runs the sandbox as a synthesized
non-root account without the account existing in the image. Images that
declare a USER keep the OCI resolution path unchanged.

Part of NVIDIA#3116.

Signed-off-by: Akram <akram.benaissi@gmail.com>
Replace the remaining OpenShell Community sandbox image references (RPM/Helm
docs, SDK doc examples, and test fixtures) with the Alpine default, and update
the resolve_from test that asserted community-name expansion — bare values are
now passed through unchanged as explicit OCI references.

The VM runtime image pin (driver-vm/runtime/pins.env) and the GPU e2e build
base (tasks/scripts/e2e-gpu-build-images.sh) reference a community image for
their own runtime/build needs and are intentionally left for separate
follow-ups.

Part of NVIDIA#3116.

Signed-off-by: Akram <akram.benaissi@gmail.com>
@akram
akram force-pushed the refactor/sandbox-alpine-default branch from 78d68c0 to 243dc1e Compare September 16, 2026 16:19
@akram

akram commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favor of #3386.

This branch's history became hard to follow — it carried the now-superseded netns/rtnetlink work (#3281/#3282/#3285, closed) and was force-pushed several times while main moved onto the RFC-0012 architecture (#2942). #3386 is a clean re-derivation of the #3116 work (retire community images + default to Alpine, incl. USER-less numeric identity) on top of current main, with a fresh description.

@akram akram closed this Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant