[Backport release/3.0.0] Bring over the remaining cloner change (#8045) - #8073
Conversation
) Replace the asset-by-environment matrix with reusable asset definitions and explicit world compositions. Repeated memberships create distinct instances without importing the same geometry again. ```python plan = cloner.make_clone_plan( (banana_cfg, franka_cfg), ((0, 1), (0, 1, 1), (0, 0, 1), (1,)), 16, ) plan.asset_cfgs plan.env_template plan.positions plan.topology.world_prototypes # [0,1, 0,1,1, 0,0,1, 1] plan.topology.world_prototype_starts # [0,0,2,5,8,9]; shared world -1 comes first plan.topology.world_prototype_layout # [0,0,0,0, 1,1,1,1, 2,2,2,2, 3,3,3,3] ``` - Newton imports each asset once, composes each selected world prototype once, and batches replication. Clone contexts consume world indices directly, without reconstructing dense masks. - Consumers read the plan and compose `cloner.path`/`cloner.query` operations. They no longer read USD clone-context state. Clone contexts execute replication; they do not own naming, placement, or runtime resources. - Path utilities return source/template tables and ancestry; each clone backend owns its copy selection. USD and both OVRTX paths preserve independent child overrides. OVRTX shares clone and partition orchestration between its two native APIs. - NumPy handles planning and initialization queries. `cloner.to_warp(plan.topology, device)` explicitly materializes numeric arrays for batched, graph-capturable runtime queries. No implicit transfers or mirrored cfg objects. Compared with develop `d728f0335`: **577 fewer package-production lines and 679 fewer test lines; 1,468 fewer lines overall**, including documentation and release notes. Normal `InteractiveScene` and `clone_plan_from_env_0` signatures remain unchanged. Direct plan callers use `ClonePlan(topology, asset_cfgs, env_template=..., positions=...)`. `ReplicateSession` takes `world_prototypes` and `weights` instead of `valid_set`; sequential selection produces contiguous weighted groups. `path` and `query` are stateless function namespaces in `clone_plan.py`, exported from `isaaclab.cloner`. Import the namespaces from the package; `cloner.path.*` and `cloner.query.*` calls remain unchanged. Path filters live in `cloner.path` and take the host plan. Numeric queries take `plan.topology` and integer IDs, individually or in batches. All three numeric queries return flat world indices and boundaries shaped `[num_queries, num_worlds + 2]`. Warp callers supply resident IDs and preallocated outputs. Custom clone contexts receive `replicate(plan, asset_prototype_ids)`; context instances are constructed by the dispatcher from the simulation. Authored sources come from `cloner.path.get_asset_prototype_paths(plan)`. `get_world_prototype_asset_templates(plan)` returns templates and prototype boundaries; `include_world_indices=True` additionally returns world IDs and their per-prototype boundaries. These tables replace `get_instance_paths`; the two boundary arrays group different relationships. Neither table depends on a USD context. Removed joined path mappings, `iter_subtree_copies`, and the `under`/`relativize`/`split` wrappers. Compose plan indexing with `match`, `relative_to`, `rebase`, and `get_parent_indices`; the API reference documents these primitives. Raw backend replication APIs retain their path-based inputs. Matched warm runs of `2c2c7bb55` against develop `d728f0335` on RTX 5090 **GPU 0**: 4096 environments, seed 42, identical dependencies, no interactive visualizer. Newton uses MJWarp. Each checkout/workload had one excluded primer, then two fresh warm launches with the PR/develop order reversed for the second pair. Values are means of those two samples. | Workload | Physics → renderer | Startup PR / develop (s) | Runtime PR / develop (ms/step) | Aggregate FPS PR / develop | | --- | --- | ---: | ---: | ---: | | Cartpole | Newton → none | 5.06 / 5.02 | 1.246 / 1.280 | 3,287,586 / 3,199,427 | | Humanoid | Newton → none | 6.95 / 6.93 | 8.028 / 8.033 | 510,212 / 509,877 | | Franka Lift (varied objects) | Newton → none | 9.16 / 11.71 | 15.465 / 15.902 | 264,855 / 257,581 | | Rough Anymal-D | Newton → none | 10.66 / 10.73 | 29.491 / 29.346 | 138,889 / 139,577 | | Kuka Allegro (cube) | Newton → none | 10.92 / 10.91 | 18.800 / 18.504 | 217,874 / 221,360 | | Kuka Allegro (varied objects) | Newton → none | 11.38 / 15.04 | 18.100 / 17.828 | 226,292 / 229,748 | | Cartpole Camera | Newton → Newton | 6.49 / 6.50 | 5.012 / 5.057 | 817,279 / 810,016 | | Kuka Camera (cube) | Newton → Newton | 13.47 / 13.40 | 29.159 / 28.731 | 140,472 / 142,564 | | Kuka Camera (varied objects) | Newton → Newton | 14.17 / 18.53 | 28.207 / 28.076 | 145,213 / 145,890 | | Cartpole | OVPhysX → none | 11.06 / 10.89 | 3.693 / 3.796 | 1,108,994 / 1,078,998 | - **Heterogeneous manipulation startup improved by 21.8–24.3%.** Scene creation accounts for approximately 2.6 s of the Franka saving, 3.5 s for Kuka, and 4.2 s for Kuka Camera. Newton composes each selected world prototype once and batches contiguous copies. Homogeneous scenes already had a batched path. - **Controls are approximately neutral, not universally faster.** The largest startup cost was OVPhysX Cartpole: +0.17 s (+1.6%), with overlapping sample ranges. Runtime changes were small and mixed: 1.6% slower to 2.7% faster. This sweep does not establish a general runtime improvement. - Sequential heterogeneous assignment now groups worlds contiguously by prototype rather than round-robin. Variant counts match, but per-world variant order and random-action contact trajectories can differ. - Native body/shape counts matched; every Newton case used one backend. Camera output was finite and nonconstant. Process peak RSS differed by at most 15 MB (<1%). Savings persisted when including reset and the 20 warmup steps. Startup includes imports, cfg resolution, application launch, and completed environment construction; it excludes explicit reset, warmup, and teardown. Runtime measures a GPU-synchronized 100-step random-action loop after 20 warmup steps. FPS is aggregate environment-steps/s, not policy-training throughput. These are warm-cache results, not cold startup measurements. - Latest follow-up: **85 tests and 4 subtests passed** across the core and Newton cloner suites. Extended existing tests reproduced the spawnless-child and root-transform regressions before the fix and passed afterward. A two-link Newton reproduction retained one intact revolute articulation. - **338 tests and 4 subtests passed**: core/Newton/OVPhysX/OVRTX cloning, NumPy/Warp queries and CUDA graph replay, material binding, deformable/MPM geometry, segmentation, frame views, scene lifecycle, and separate Kit core/PhysX cloner suites. OVPhysX CPU and GPU tests ran in separate processes. - The existing material-initialization regression fails with the previous method and passes with the fix; its simulation fixture supplies only the plan, not clone contexts. - Repository-wide ownership audit and an existing architecture test prevent consumer access to clone-context instances. - `uv run isaaclab -f`, changelog validation, and `git diff --check` passed. Documentation builds in CI. - Breaking API change for direct plan callers and custom clone contexts. - Documentation update. - [x] Read the contribution guidelines - [x] Ran formatting and lint checks - [x] Updated documentation and migration notes - [x] Extended and consolidated existing tests - [x] Added a changelog fragment for each touched package - [x] Contributor already listed - [x] <!-- backport-active-release --> Backport to the active release branch (cherry picked from commit 83622a5)
|
| ) | ||
| logger.info("Written primvars:omni:scenePartition to %d environments", num_envs) | ||
| xforms = np.tile(np.eye(4, dtype=np.float64), (num_envs, 1, 1)) | ||
| xforms[:, 3, :3] = plan.positions |
There was a problem hiding this comment.
Optional positions break OVRTX
If a clone plan is created without positions, as make_clone_plan permits, this assignment puts None into a numeric transform array. OVRTX scene initialization then fails instead of preserving the authored environment placement.
Knowledge Base Used: Physics backend extensions
| prototype_ids = np.unique(plan.topology.world_prototypes[starts[1] :]) | ||
| source_paths = [authored[i] for i in prototype_ids if authored[i] is not None and authored[i] not in exclude_paths] |
There was a problem hiding this comment.
Unused deformables enter visualization
When a world prototype is defined but no destination world selects it, these lines still discover its deformable source. Expansion has no destination for that source and retains it as a shared entry, so OVPhysX or Newton visualization can import geometry that the plan places in no world.
Knowledge Base Used:
There was a problem hiding this comment.
Isaac Lab Review Bot
The backport consistently migrates cloner consumers to PrototypeWorldTopology, but the OVRTX renderer does not honor the new optional ClonePlan.positions contract.
- Design and architecture: The topology and host-side naming/placement split is propagated across USD, PhysX, OVPhysX, Newton, OVRTX, sensors, and scene-data consumers. The remaining inconsistency is that OVRTX still assumes every plan contains explicit positions.
- API:
ClonePlan.positionsis documented as optional, whereNonepreserves authored placement. OVRTX no longer validates this narrower requirement and therefore exposes an accidental failure instead of supporting the contract or reporting a clear precondition. - Implementation: In
OVRTXRenderer._clone_sources, assigningplan.positionsdirectly into the translation slice fails when positions isNone. Skip the environment-transform write in that case, or restore explicit validation with a clear error before cloning.
Minor fixes needed. Posted 1 actionable finding inline.
Automated review; human maintainers own approval decisions.
| ) | ||
| logger.info("Written primvars:omni:scenePartition to %d environments", num_envs) | ||
| xforms = np.tile(np.eye(4, dtype=np.float64), (num_envs, 1, 1)) | ||
| xforms[:, 3, :3] = plan.positions |
There was a problem hiding this comment.
🟡 Warning · Api — OVRTX cannot accept plans without positions
ClonePlan.positions is documented here as optional, with None preserving authored placement, and UsdReplicateContext.replicate guards on it. prepare_stage no longer rejects such plans, so xforms[:, 3, :3] = plan.positions raises an unrelated TypeError instead of the previous explicit error. Either skip the environment-transform write when positions is None, matching the USD context, or restore an explicit validation message.
|
run-ci |
Description
Backport #8045 to
release/3.0.0, including its cloner implementation, tests, documentation, and package changelog fragments. The release branch otherwise contains the mergeddevelopPRs from the recent catch-up window. #8059 is already covered by merged backport #8069 and is not included here.The source cherry-pick applies cleanly except for
docs/source/how-to/cloning.rst. The resolution keeps the release branch's validreplicate_physics=Falseguidance alongside the new cloner API guide. This PR provides a release-based alternative to the bot's draft #8072, with the later #8071 and #8069 release commits included in its base.Direct
ClonePlancallers and custom clone contexts must follow the migration guidance included with #8045. NormalInteractiveSceneandclone_plan_from_env_0signatures remain unchanged.Validation
uv run isaaclab -fpassed with the changelog base set torelease/3.0.0.git diff --checkpassed.devextra (Sphinx 8.2.3, warnings treated as errors).Type of change
Checklist
This PR targets the release branch; no additional backport is needed.