Skip to content

[Backport release/3.0.0] Bring over the remaining cloner change (#8045) - #8073

Merged
StafaH merged 1 commit into
isaac-sim:release/3.0.0from
StafaH:codex/backport-missing-pr-8045-release-3.0
Sep 27, 2026
Merged

StafaH merged 1 commit into
isaac-sim:release/3.0.0from
StafaH:codex/backport-missing-pr-8045-release-3.0

Conversation

@StafaH

@StafaH StafaH commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Backport #8045 to release/3.0.0, including its cloner implementation, tests, documentation, and package changelog fragments. The release branch otherwise contains the merged develop PRs from the recent catch-up window. #8059 is already covered by merged backport #8069 and is not included here.

The source cherry-pick applies cleanly except for docs/source/how-to/cloning.rst. The resolution keeps the release branch's valid replicate_physics=False guidance alongside the new cloner API guide. This PR provides a release-based alternative to the bot's draft #8072, with the later #8071 and #8069 release commits included in its base.

Direct ClonePlan callers and custom clone contexts must follow the migration guidance included with #8045. Normal InteractiveScene and clone_plan_from_env_0 signatures remain unchanged.

Validation

  • 44 core cloner tests passed.
  • 11 Newton cloner tests and 4 subtests passed.
  • uv run isaaclab -f passed with the changelog base set to release/3.0.0.
  • git diff --check passed.
  • Full documentation build passed with the dev extra (Sphinx 8.2.3, warnings treated as errors).

Type of change

  • Breaking change to direct cloner APIs, with migration documentation.
  • Release backport.

Checklist

  • Read the contribution guidelines.
  • Ran formatting and lint checks.
  • Included documentation, tests, and changelog fragments from the source PR.
  • Original contributor already listed.

This PR targets the release branch; no additional backport is needed.

)

Replace the asset-by-environment matrix with reusable asset definitions
and explicit world compositions. Repeated memberships create distinct
instances without importing the same geometry again.

```python
plan = cloner.make_clone_plan(
    (banana_cfg, franka_cfg),
    ((0, 1), (0, 1, 1), (0, 0, 1), (1,)),
    16,
)
plan.asset_cfgs
plan.env_template
plan.positions
plan.topology.world_prototypes        # [0,1, 0,1,1, 0,0,1, 1]
plan.topology.world_prototype_starts   # [0,0,2,5,8,9]; shared world -1 comes first
plan.topology.world_prototype_layout  # [0,0,0,0, 1,1,1,1, 2,2,2,2, 3,3,3,3]
```

- Newton imports each asset once, composes each selected world prototype
once, and batches replication. Clone contexts consume world indices
directly, without reconstructing dense masks.
- Consumers read the plan and compose `cloner.path`/`cloner.query`
operations. They no longer read USD clone-context state. Clone contexts
execute replication; they do not own naming, placement, or runtime
resources.
- Path utilities return source/template tables and ancestry; each clone
backend owns its copy selection. USD and both OVRTX paths preserve
independent child overrides. OVRTX shares clone and partition
orchestration between its two native APIs.
- NumPy handles planning and initialization queries.
`cloner.to_warp(plan.topology, device)` explicitly materializes numeric
arrays for batched, graph-capturable runtime queries. No implicit
transfers or mirrored cfg objects.

Compared with develop `d728f0335`: **577 fewer package-production lines
and 679 fewer test lines; 1,468 fewer lines overall**, including
documentation and release notes.

Normal `InteractiveScene` and `clone_plan_from_env_0` signatures remain
unchanged.

Direct plan callers use `ClonePlan(topology, asset_cfgs,
env_template=..., positions=...)`. `ReplicateSession` takes
`world_prototypes` and `weights` instead of `valid_set`; sequential
selection produces contiguous weighted groups.

`path` and `query` are stateless function namespaces in `clone_plan.py`,
exported from `isaaclab.cloner`. Import the namespaces from the package;
`cloner.path.*` and `cloner.query.*` calls remain unchanged.

Path filters live in `cloner.path` and take the host plan. Numeric
queries take `plan.topology` and integer IDs, individually or in
batches. All three numeric queries return flat world indices and
boundaries shaped `[num_queries, num_worlds + 2]`. Warp callers supply
resident IDs and preallocated outputs.

Custom clone contexts receive `replicate(plan, asset_prototype_ids)`;
context instances are constructed by the dispatcher from the simulation.
Authored sources come from
`cloner.path.get_asset_prototype_paths(plan)`.
`get_world_prototype_asset_templates(plan)` returns templates and
prototype boundaries; `include_world_indices=True` additionally returns
world IDs and their per-prototype boundaries. These tables replace
`get_instance_paths`; the two boundary arrays group different
relationships. Neither table depends on a USD context. Removed joined
path mappings, `iter_subtree_copies`, and the
`under`/`relativize`/`split` wrappers. Compose plan indexing with
`match`, `relative_to`, `rebase`, and `get_parent_indices`; the API
reference documents these primitives. Raw backend replication APIs
retain their path-based inputs.

Matched warm runs of `2c2c7bb55` against develop `d728f0335` on RTX 5090
**GPU 0**: 4096 environments, seed 42, identical dependencies, no
interactive visualizer. Newton uses MJWarp. Each checkout/workload had
one excluded primer, then two fresh warm launches with the PR/develop
order reversed for the second pair. Values are means of those two
samples.

| Workload | Physics → renderer | Startup PR / develop (s) | Runtime PR
/ develop (ms/step) | Aggregate FPS PR / develop |
| --- | --- | ---: | ---: | ---: |
| Cartpole | Newton → none | 5.06 / 5.02 | 1.246 / 1.280 | 3,287,586 /
3,199,427 |
| Humanoid | Newton → none | 6.95 / 6.93 | 8.028 / 8.033 | 510,212 /
509,877 |
| Franka Lift (varied objects) | Newton → none | 9.16 / 11.71 | 15.465 /
15.902 | 264,855 / 257,581 |
| Rough Anymal-D | Newton → none | 10.66 / 10.73 | 29.491 / 29.346 |
138,889 / 139,577 |
| Kuka Allegro (cube) | Newton → none | 10.92 / 10.91 | 18.800 / 18.504
| 217,874 / 221,360 |
| Kuka Allegro (varied objects) | Newton → none | 11.38 / 15.04 | 18.100
/ 17.828 | 226,292 / 229,748 |
| Cartpole Camera | Newton → Newton | 6.49 / 6.50 | 5.012 / 5.057 |
817,279 / 810,016 |
| Kuka Camera (cube) | Newton → Newton | 13.47 / 13.40 | 29.159 / 28.731
| 140,472 / 142,564 |
| Kuka Camera (varied objects) | Newton → Newton | 14.17 / 18.53 |
28.207 / 28.076 | 145,213 / 145,890 |
| Cartpole | OVPhysX → none | 11.06 / 10.89 | 3.693 / 3.796 | 1,108,994
/ 1,078,998 |

- **Heterogeneous manipulation startup improved by 21.8–24.3%.** Scene
creation accounts for approximately 2.6 s of the Franka saving, 3.5 s
for Kuka, and 4.2 s for Kuka Camera. Newton composes each selected world
prototype once and batches contiguous copies. Homogeneous scenes already
had a batched path.
- **Controls are approximately neutral, not universally faster.** The
largest startup cost was OVPhysX Cartpole: +0.17 s (+1.6%), with
overlapping sample ranges. Runtime changes were small and mixed: 1.6%
slower to 2.7% faster. This sweep does not establish a general runtime
improvement.
- Sequential heterogeneous assignment now groups worlds contiguously by
prototype rather than round-robin. Variant counts match, but per-world
variant order and random-action contact trajectories can differ.
- Native body/shape counts matched; every Newton case used one backend.
Camera output was finite and nonconstant. Process peak RSS differed by
at most 15 MB (<1%). Savings persisted when including reset and the 20
warmup steps.

Startup includes imports, cfg resolution, application launch, and
completed environment construction; it excludes explicit reset, warmup,
and teardown. Runtime measures a GPU-synchronized 100-step random-action
loop after 20 warmup steps. FPS is aggregate environment-steps/s, not
policy-training throughput. These are warm-cache results, not cold
startup measurements.

- Latest follow-up: **85 tests and 4 subtests passed** across the core
and Newton cloner suites. Extended existing tests reproduced the
spawnless-child and root-transform regressions before the fix and passed
afterward. A two-link Newton reproduction retained one intact revolute
articulation.
- **338 tests and 4 subtests passed**: core/Newton/OVPhysX/OVRTX
cloning, NumPy/Warp queries and CUDA graph replay, material binding,
deformable/MPM geometry, segmentation, frame views, scene lifecycle, and
separate Kit core/PhysX cloner suites. OVPhysX CPU and GPU tests ran in
separate processes.
- The existing material-initialization regression fails with the
previous method and passes with the fix; its simulation fixture supplies
only the plan, not clone contexts.
- Repository-wide ownership audit and an existing architecture test
prevent consumer access to clone-context instances.
- `uv run isaaclab -f`, changelog validation, and `git diff --check`
passed. Documentation builds in CI.

- Breaking API change for direct plan callers and custom clone contexts.
- Documentation update.

- [x] Read the contribution guidelines
- [x] Ran formatting and lint checks
- [x] Updated documentation and migration notes
- [x] Extended and consolidated existing tests
- [x] Added a changelog fragment for each touched package
- [x] Contributor already listed

- [x] <!-- backport-active-release --> Backport to the active release
branch

(cherry picked from commit 83622a5)
@StafaH
StafaH requested a review from a team September 27, 2026 02:02
@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels Sep 27, 2026
@greptile-apps

greptile-apps Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 3/5

[High risk] Refactors the core cloning topology and API across the simulation framework.

The PR should not merge until OVRTX handles plans without positions and deformable discovery excludes unused prototypes.

Findings

  1. P1 Optional positions break OVRTX ▶
  2. P1 Unused deformables enter visualization ▶

Summary

The backport replaces clone-plan source/mask rows with reusable asset and world-prototype topology, and migrates scene construction, backend replication, consumers, tests, and documentation.

  • Optional placement is not handled by OVRTX cloning.
  • Deformable discovery does not exclude unused world prototypes.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    C[Asset configurations] --> P[Clone plan and world topology]
    P --> S[InteractiveScene authoring]
    S --> B[USD and native clone backends]
    P --> D[Deformable discovery]
    B --> R[Physics and rendering]
    D --> R
Loading

Reviews (1) · Last reviewed commit: "[Cloner] Represent worlds with reusable ..."

)
logger.info("Written primvars:omni:scenePartition to %d environments", num_envs)
xforms = np.tile(np.eye(4, dtype=np.float64), (num_envs, 1, 1))
xforms[:, 3, :3] = plan.positions

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Optional positions break OVRTX

If a clone plan is created without positions, as make_clone_plan permits, this assignment puts None into a numeric transform array. OVRTX scene initialization then fails instead of preserving the authored environment placement.

Knowledge Base Used: Physics backend extensions

Comment on lines +295 to +296
prototype_ids = np.unique(plan.topology.world_prototypes[starts[1] :])
source_paths = [authored[i] for i in prototype_ids if authored[i] is not None and authored[i] not in exclude_paths]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Unused deformables enter visualization

When a world prototype is defined but no destination world selects it, these lines still discover its deformable source. Expansion has no destination for that source and retains it as a shared entry, so OVPhysX or Newton visualization can import geometry that the plan places in no world.

Knowledge Base Used:

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The backport consistently migrates cloner consumers to PrototypeWorldTopology, but the OVRTX renderer does not honor the new optional ClonePlan.positions contract.

  • Design and architecture: The topology and host-side naming/placement split is propagated across USD, PhysX, OVPhysX, Newton, OVRTX, sensors, and scene-data consumers. The remaining inconsistency is that OVRTX still assumes every plan contains explicit positions.
  • API: ClonePlan.positions is documented as optional, where None preserves authored placement. OVRTX no longer validates this narrower requirement and therefore exposes an accidental failure instead of supporting the contract or reporting a clear precondition.
  • Implementation: In OVRTXRenderer._clone_sources, assigning plan.positions directly into the translation slice fails when positions is None. Skip the environment-transform write in that case, or restore explicit validation with a clear error before cloning.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

)
logger.info("Written primvars:omni:scenePartition to %d environments", num_envs)
xforms = np.tile(np.eye(4, dtype=np.float64), (num_envs, 1, 1))
xforms[:, 3, :3] = plan.positions

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Api — OVRTX cannot accept plans without positions

ClonePlan.positions is documented here as optional, with None preserving authored placement, and UsdReplicateContext.replicate guards on it. prepare_stage no longer rejects such plans, so xforms[:, 3, :3] = plan.positions raises an unrelated TypeError instead of the previous explicit error. Either skip the environment-transform write when positions is None, matching the USD context, or restore an explicit validation message.

@StafaH

StafaH commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 27, 2026
@StafaH
StafaH merged commit 6794412 into isaac-sim:release/3.0.0 Sep 27, 2026
54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants