Skip to content

Docker build stages and helper scripts - #1280

Merged
david-tingdahl-nvidia merged 12 commits into
mainfrom
dtingdahl/refactor/docker-stages
Sep 28, 2026
Merged

david-tingdahl-nvidia merged 12 commits into
mainfrom
dtingdahl/refactor/docker-stages

Conversation

@david-tingdahl-nvidia

@david-tingdahl-nvidia david-tingdahl-nvidia commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

Re-organize the Arena images builds using docker build stages and dedicated helper scripts for installing.
User-facing docker scripts remain the same as before

Why this change?

  • Makes the Dockerfile easier to read, maintain and extend
  • Avoid pip install download on source code changes (all .toml deps are now installed in the arena-deps stage)
  • Previous Dockerfile installed Arena dependencies and compiled cuRobo after copying Arena source, triggering lengthy rebuilds.

The perceived speedups are moderate for dev not using cuRobo. Main advantage is the structured docker build with existing patterns to follow for future modifications of the dockerfile.

What changes?

  • Install dependencies before copying Arena source. Read project dependencies and the full dev extra from pyproject.toml, and move installation commands into focused setup scripts.
  • Build the cuRobo wheel in a separate stage based on Isaac Lab. Install it through a read-only mount so the CUDA build toolchain and wheel archive stay out of the final images.
  • Add build_docker.sh for builds without launching a container. Share it between the launcher and NGC publisher. Preserve the default developer image and -c shortcut; replace INSTALL_CUROBO with dev and dev-curobo targets.
  • Ignore pytest caches and Emacs temporary files. Create the runtime user's home directory and remove the unused duplicate pytest alias.

CI still tests the published images. Building branch images before tests and adding remote-cache integration remain separate work.

Usage example

Replace the old feature argument in direct Docker builds.

Before:

docker build -f docker/Dockerfile.isaaclab_arena \
  --build-arg INSTALL_CUROBO=true -t isaaclab_arena:curobo .

After:

docker build -f docker/Dockerfile.isaaclab_arena \
  --target dev-curobo -t isaaclab_arena:curobo .

The existing ./docker/run_docker.sh -c and ./docker/push_to_ngc.sh -c -p commands remain supported and select the cuRobo developer image. Without -c, both scripts default to the developer image without cuRobo.

Add runtime and developer targets with optional prebuilt cuRobo wheels. Keep dependency installation and cuRobo compilation cached across Arena source edits, and share build/setup scripts.

Validate all four targets locally. Record the AGILE standing and GUI preview failures, both reproduced on the original image; committing with these baseline failures is approved. CI integration remains deferred.

Signed-off-by: David Tingdahl <dtingdahl@nvidia.com>
Signed-off-by: David Tingdahl <dtingdahl@nvidia.com>
Signed-off-by: David Tingdahl <dtingdahl@nvidia.com>
Share the build helper between local and NGC builds. Select dev-curobo with -c, preserve explicit output tags, and document the current stage layout and pending validation.

Signed-off-by: David Tingdahl <dtingdahl@nvidia.com>
@david-tingdahl-nvidia david-tingdahl-nvidia changed the title Split Arena Docker builds into reusable dependency stages Docker build optimization and refactor Sep 17, 2026
@david-tingdahl-nvidia david-tingdahl-nvidia changed the title Docker build optimization and refactor Docker build optimization Sep 17, 2026
@greptile-apps

greptile-apps Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

The PR is not safe to merge until the Dockerfile frontend mismatch is fixed, because all image targets depend on the unparseable runtime stage.

Findings

  1. P1 Unsupported Dockerfile Instruction ▶
  2. P2 GPU Validation Is Missing ▶

Summary

This PR reorganizes the Arena container into cached dependency, runtime, developer, and optional cuRobo stages and centralizes image building in a shared script.

  • Moves dependency installation ahead of Arena source copies to preserve build cache reuse.
  • Builds cuRobo in an isolated stage and installs only its wheel and runtime dependencies.
  • Adds named runtime/developer targets and updates local and NGC build launchers.
  • Contains a Dockerfile frontend mismatch that currently prevents the new stages from building.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Isaac Sim base] --> B[sim-base]
    B --> C[isaaclab]
    C --> D[arena-deps]
    D --> E[runtime]
    E --> F[dev]
    C --> G[curobo-builder]
    G -. mounted wheel .-> H[runtime-curobo]
    E --> H
    H --> I[dev-curobo]
    F --> J[default]
Loading

Reviews (1) · Last reviewed commit: "undo some changes"

Comment thread docker/Dockerfile.isaaclab_arena
Comment thread docker/setup/install-curobo-runtime-deps.sh Outdated
Comment thread docker/setup/export_requirements.py Outdated
Comment thread docker/Dockerfile.isaaclab_arena Outdated
Comment thread docker/Dockerfile.isaaclab_arena Outdated
Comment thread docker/build_docker.sh Outdated
Comment thread docker/build_docker.sh Outdated
@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

Restructures the Arena Dockerfile into named stages so dependency installation and cuRobo compilation no longer rebuild on Arena source edits, and factors the inline RUN blocks into docker/setup/*.sh scripts shared via bind mounts. The core idea — install deps before copying source, and build the cuRobo wheel in an isolated stage installed through a mount — is the right fix and makes the file much easier to read. My concerns are about the extra surface bolted on around it: two image targets nothing builds, and a second place where "dev vs runtime" dependencies get decided.

Design, Boundaries & Scope

The runtime / runtime-curobo targets are the part I'd push back on. Because the dev extra's user tools (jupyter, streamlit, streamlit-ace, simready-search) are installed in arena-deps, and pytest is a plain runtime dependency, runtime ends up differing from dev by only debugpy, pre-commit and gh — while doubling the target matrix, the target→tag mapping (now in three scripts), and the stages that can rot. Nothing in CI or push_to_ngc.sh builds them, so they're unverified from day one. The caching win the PR is after comes entirely from the stage reordering, which stands on its own.

Related: docker/setup/export_requirements.py re-derives the dependency split from pyproject.toml using a hardcoded contributor_packages = {"debugpy"} set. That's a second source of truth — a contributor-only tool added to the dev extra later will quietly ship in the runtime image.

Findings

🟡 Warning: docker/setup/export_requirements.py:27 — hardcoded contributor allowlist duplicates the dependency classification that belongs in pyproject.toml.
🟡 Warning: docker/Dockerfile.isaaclab_arena:60 — runtime / runtime-curobo add build-matrix surface for ~3 packages and are never built by CI or the NGC publisher.
🔵 Improvement: docker/Dockerfile.isaaclab_arena:113 — /usr/local/share/arena/curobo-build.json has no reader anywhere in the repo.
🔵 Improvement: docker/build_docker.sh:37 — the target→tag mapping is repeated in build_docker.sh, run_docker.sh and push_to_ngc.sh.
🔵 Improvement: docker/build_docker.sh:23 — -l is never passed by either caller.

Two things that don't map to a diff line:

  • The PR description's last bullet mentions guarding notebook examples against starting a second simulator and adding a real GPU IK check, but the final commit ("undo some changes") dropped those — no notebook or isaaclab_arena_curobo/tests files are in the diff. Worth trimming the description so it matches what's being merged.
  • While you're reorganising the shell setup: docker/setup/entrypoint.sh:56 appends a pytest alias to /etc/aliasess.bashrc (typo — nothing reads that file), and configure-shell.sh already puts that alias in /etc/bash.bashrc. Good moment to delete the line.

Test Coverage

No unit tests are expected for Docker plumbing, but it's worth being explicit about the gap: CI runs its test phases inside the published :latest / :curobo NGC images, so nothing in this PR's build is exercised before merge — a broken stage would first surface in the post-merge build_and_push_*_post_merge jobs. The PR acknowledges branch-image builds are follow-up work. On top of that, runtime and runtime-curobo aren't built by any script or workflow, so they have no coverage at all, before or after merge.

Verdict

Minor fixes needed

Share target tags across build, launch, and publish commands. Remove the unused log-file option and ineffective duplicate pytest alias. Explain the cuRobo build record as human-readable provenance.

Signed-off-by: David Tingdahl <dtingdahl@nvidia.com>
Signed-off-by: David Tingdahl <dtingdahl@nvidia.com>
Signed-off-by: David Tingdahl <dtingdahl@nvidia.com>
@david-tingdahl-nvidia david-tingdahl-nvidia changed the title Docker build optimization Docker build stages and helper scripts Sep 18, 2026

@alexmillane alexmillane left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! Thank you for cleaning that up!

Comment thread docker/setup/build-curobo-wheel.sh
@david-tingdahl-nvidia
david-tingdahl-nvidia merged commit da8ee04 into main Sep 28, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants