Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@
- Use modern Python type hints, including `X | None` instead of `Optional[X]`.
- Use `snake_case` for methods, functions, and CLI arguments.
- Keep related public symbols discoverable through consistent prefixes.
- Keep joint-wrench sensor coverage separate from articulation control-joint selection. Reuse cached
body bindings without changing the shared view's joint filters or creating a second view for sensing.
- For external wrenches, follow the asset API's `is_global` boolean and `_b`/`_w` buffer naming. Keep
frame conversion decisions in `WrenchComposer` and track pending contributions with plain booleans;
do not introduce frame enums, content bitmasks, or a classification layer.
- Name wrench reads `get_forces_and_torques`, matching the existing add/set methods; avoid a separate
"submission" API or compatibility alias for the unreleased `resolve_submission` method.
- Use concrete types for public interfaces where practical.
- Use Google-style docstrings for public APIs.
- Document SI units for public physical quantities in docstrings using inline `[unit]` notation (e.g. `Particle positions [m], shape [N, 3]`); use `[m or rad, depending on joint type]` where applicable, and skip non-physical fields (indices, counts, flags).
Expand Down Expand Up @@ -43,6 +50,8 @@
- Find and extend the closest existing test before creating a new test file or test case.
- Add a test only when it covers a distinct behavior, regression, boundary, or failure mode that existing tests do not cover clearly.
- Test observable behavior and public contracts, not implementation details.
- Validate joint-wrench frames with the same physical fixture and analytic load expectations across backends.
Do not derive the expected wrench by repeating the production transformation on the backend's raw output.
- Use hard-coded values only when they are the intended contract or a small, independently verified example; otherwise derive the expected result from a separate, simple reference calculation.
- Keep tests focused and remove or consolidate redundant coverage instead of growing overlapping test suites.
- Do not add debug output to production Warp kernels. Use temporary standalone reproductions and remove debug output before committing.
Expand Down
40 changes: 40 additions & 0 deletions docs/source/api/lab/isaaclab.managers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,46 @@ Manager Base
Observation Manager
-------------------

Observation delay
~~~~~~~~~~~~~~~~~

Configure observation delay directly on the term:

.. code-block:: python

from isaaclab.envs import mdp
from isaaclab.managers import ObservationTermCfg

joint_pos = ObservationTermCfg(func=mdp.joint_pos_rel, delay_min_lag=1, delay_max_lag=3)

Each environment samples its lag uniformly from the inclusive bounds at initialization and reset.
By default, ``delay_hold_prob=1.0`` keeps that lag for the episode, matching the reset-based lag policy of
:class:`~isaaclab.actuators.DelayedPDActuator`. Equal bounds give a constant delay; both zero disable delay.
Both use :class:`~isaaclab.utils.buffers.DelayBuffer`; observation lag counts recorded observation
samples, while actuator lag counts physics steps.

Set ``delay_hold_prob`` below 1.0 to vary latency within an episode:

.. code-block:: python

joint_pos = ObservationTermCfg(func=mdp.joint_pos_rel, delay_min_lag=1, delay_max_lag=3, delay_hold_prob=0.8)

On each recorded sample, each environment retains its lag with probability 0.8 and otherwise draws a new
one. Zero redraws on every sample. Holding the lag keeps the latency constant while frames continue to
advance; it does not freeze the returned frame. Reset always redraws the selected environments' lags.

The processing order is observation function, modifiers, noise, clipping, scaling, delay, then history.
The ``delay_min_lag`` and ``delay_max_lag`` field names and delay placement follow
`mjlab's observation configuration <https://mujocolab.github.io/mjlab/main/source/observations.html>`_.
``delay_hold_prob`` follows mjlab's lag-retention semantics, with a default of 1.0 to preserve Isaac Lab's
reset-based delay policy.

``compute(update_history=True)`` records a sample in both delay and observation history buffers.
``compute()`` and ``compute_group()`` read without advancing either buffer or resampling lag.
For a delay buffer with no recorded sample after initialization or reset, the current input is returned
without recording it. Once recording starts, delays exceeding the available history return the oldest
sample. Partial resets invalidate only the selected environments' histories.

.. autoclass:: ObservationManager
:members:
:inherited-members:
Expand Down
7 changes: 7 additions & 0 deletions docs/source/api/lab/isaaclab.sim.spawners.rst
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,7 @@ From Files

UrdfFileCfg
UsdFileCfg
MeshFileCfg
GroundPlaneCfg

.. autofunction:: spawn_from_urdf
Expand All @@ -269,6 +270,12 @@ From Files
:members:
:exclude-members: __init__, func

.. autofunction:: spawn_from_mesh

.. autoclass:: MeshFileCfg
:members:
:exclude-members: __init__, func

.. autofunction:: spawn_ground_plane

.. autoclass:: GroundPlaneCfg
Expand Down
10 changes: 8 additions & 2 deletions docs/source/concepts/sensors/joint_wrench_sensor.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,21 @@ The ``incoming_joint_frame`` convention expresses the wrench in the child-side j
child-side joint anchor. This matches the placement of a six-axis force/torque sensor mounted at the
joint. Backend implementations convert their native solver output to this common convention.

PhysX's ``get_link_incoming_joint_force()`` already returns the wrench in the child-side joint frame,
referenced at its anchor, so the PhysX sensor exposes those components directly. Applying the USD
``localPos1`` and ``localRot1`` again would shift and rotate the wrench twice.

Configure the sensor
--------------------

Set :attr:`~sensors.JointWrenchSensorCfg.prim_path` to the articulation root. Reported body coverage
depends on the physics backend:

* PhysX and OVPhysX report every articulation link, including the root link.
* Newton reports the child link of each non-free, non-fixed joint. It therefore excludes the root
link and any links connected through free or fixed joints.
* Newton reports the child link of each non-free joint in the articulation tree, including fixed
connections between bodies such as a welded wrist sensor or tool flange. Free joints, fixed joints
to the world, and loop-closing constraints are excluded. The reported wrench at a weld includes
the loads transmitted by its child subtree, even though the joint has no degrees of freedom.

Use :attr:`~sensors.JointWrenchSensor.body_names` or
:meth:`~sensors.JointWrenchSensor.find_bodies` instead of assuming that different backends expose
Expand Down
63 changes: 48 additions & 15 deletions docs/source/developer-tools/scene_data_providers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,19 @@ The system has three layers:
1. :class:`~isaaclab.scene_data.SceneDataBackend`: a small interface implemented by each physics
manager. It exposes the backend's transform array directly as one of the
:class:`~isaaclab.scene_data.SceneDataFormat` Warp structs, plus the per-transform prim paths
and total count. There is no per-frame "update" call; the property accessors return live
views into the underlying tensor each time they're read.
and total count. Producers increment ``transforms_version`` after native state writes or buffer swaps;
SDP calls ``get_transforms(output_format)`` before reading the version, since resolving the pointer
can itself detect a swap. The default implementation returns the existing ``transforms`` property.
The version never resets, so independent readers cannot hide changes from one another.

- :attr:`SceneDataBackend.transforms`: current transforms as a Warp struct (one of
- :attr:`SceneDataBackend.transforms`: the native data as a Warp struct (one of
:class:`SceneDataFormat.Vec3_Quat`, :class:`SceneDataFormat.Transform`,
:class:`SceneDataFormat.Matrix44`, :class:`SceneDataFormat.Vec3_Matrix33`).
- :attr:`SceneDataBackend.transforms_version`: monotonic version of the native transforms.
- :attr:`SceneDataBackend.transform_count`: number of transforms.
- :attr:`SceneDataBackend.transform_paths`: list of USD prim paths, one per transform.
- :attr:`SceneDataBackend.native_transform_formats`: formats published without conversion.
PhysX publishes either packed poses or Fabric matrices and refreshes only the requested representation.
- :attr:`SceneDataBackend.points`: flattened deformable nodal positions as
:class:`SceneDataFormat.Points` (optional; rigid-only backends return an empty buffer).
- :attr:`SceneDataBackend.point_count`: total number of geometry points.
Expand All @@ -48,11 +53,10 @@ The system has three layers:
2. :class:`~isaaclab.scene_data.SceneDataProvider`: wraps a backend and offers format conversion
plus index re-mapping.

- :meth:`SceneDataProvider.get_transforms`: writes the backend's transforms into a
consumer-provided :class:`SceneDataFormat` struct, optionally converting format
(e.g. ``Vec3_Quat`` to ``Transform``) and applying an index mapping. When the backend
format matches the output format and no mapping is provided, the result is a zero-copy
passthrough.
- :meth:`SceneDataProvider.get_transforms`: binds native arrays when format and ordering match,
or SDP-owned buffers converted once per producer version and destination layout. These shared
arrays are read-only, including when they replace preallocated output fields. Pass
``allow_passthrough=False`` to write directly into caller-owned arrays instead.
- :meth:`SceneDataProvider.create_mapping`: builds a remap array from the backend's prim
paths to a consumer's desired ordering. Used when a renderer or visualizer wants
transforms indexed by its own body list rather than by the physics view order.
Expand Down Expand Up @@ -86,10 +90,12 @@ When PhysX is the active physics backend, the provider reads transforms directly
The transforms are returned as :class:`SceneDataFormat.Transform` (Warp ``transformf`` array),
so consumers that want this format get them zero-copy.

Newton-native consumers (Newton visualizer, Rerun, Viser, Newton Warp renderer, OVRTX renderer)
also need a Newton ``Model``/``State`` to render against. To provide that,
:class:`~isaaclab_newton.physics.NewtonManager` builds a **shadow Newton model** from the USD
stage on first access and updates its ``body_q`` from the PhysX backend each render frame.
Newton-native consumers (Newton visualizer, Rerun, Viser, Newton Warp renderer) also need a
Newton ``Model``/``State``. Their declared cloning contexts construct that representation from
the shared clone plan before initialization. Its rigid ``body_q`` binds to SDP's requested
``Transform`` array; no intermediate per-frame copy into a second state buffer is required.
OVRTX requests ``TransposedMatrix44d`` directly from SDP, including destination ordering and
static scale in the same conversion. It no longer reads Newton state for rigid transforms.
When the scene has PhysX or OVPhysX deformables, the shadow model also allocates
``particle_q`` render slots for soft/cloth meshes, syncs simulation nodal positions through
:meth:`SceneDataProvider.get_points` with ``allow_passthrough=False`` into a separate
Expand All @@ -99,8 +105,26 @@ barycentric sim-to-visual remap so Newton Warp and OVRTX render the paired visua
than tet simulation topology. The shadow deformable registry exposes render-slot offsets and
``particles_per_body`` counts for OVRTX point bindings.

This is hidden behind :meth:`NewtonManager.get_model` / :meth:`NewtonManager.get_state`, so
renderers don't need to know which physics backend is active.
The deformable and cable geometry bridge remains separate from this rigid-transform path.
OVRTX still uses Newton geometry metadata for those features.

PhysX owns its native Fabric refresh and publishes the resulting matrices through SDP without
fetching packed poses. ``isaaclab_physx.renderers.fabric.FabricBackend`` owns the shared native stage
and hierarchy handles. Its identity is the stage and device, not the SDP source or attribute type.
``SimulationContext`` declares ``fabric_cfg`` when Kit is available. After physics initializes, Kit,
Isaac RTX, and explicit Fabric synchronization obtain the same resource through
``get_or_create_backend(sim.fabric_cfg)``. Transform bindings are state on that resource, not a
separate backend. Consumers pass the simulation's SDP to ``update_transforms(provider)``; for foreign
physics it converts directly into Fabric local matrices, then propagates the GPU hierarchy.
Core ``RenderContext`` owns no Fabric bindings.
It binds rigid destinations as Fabric-only reset-stack roots because
physics publishes absolute poses, including for nested bodies. Visual descendants still inherit
their body's transform; authored USD is unchanged. Native source indices and world scales are
bound once. Fabric's selection reuse API reports scene-wide structural changes; the resource refreshes
array views without repeating path matching or scale capture. Otherwise GPU propagation
reuses the hierarchy topology. Clean requests never acquire writable Fabric arrays.
Renderers do not select a physics-specific synchronization path.
``FabricMatrix44`` contains only matrix storage, not bindings or native engine handles.

Newton backend
--------------
Expand All @@ -109,11 +133,20 @@ When Newton is the active physics backend, the backend wraps the Newton model's
directly. No shadow model or per-frame sync is needed: Newton already owns the authoritative
model and state, and the provider exposes that state as :class:`SceneDataFormat.Transform`.

Native reads reconcile pending authored state writes once. A new physics publication does not
itself request forward kinematics. Kit/RTX requests current Fabric transforms through SDP
before rendering, without issuing an additional physics ``forward()``. Headless viewport
capture requests these transforms on demand rather than on every visualizer step.

Externally replayed CUDA graphs do not call Python write hooks. After writes have been captured,
Newton conservatively republishes transforms when read so an unannounced replay cannot leave
rendering stale. Those reads do not benefit from clean-publication caching.

Data requirements
------------------

Visualizers and renderers declare what they need from the scene data path. This is resolved at
simulation-context construction time and is what triggers the shadow-model build for PhysX:
consumer construction time, before the shared clone plan is built:

.. list-table::
:header-rows: 1
Expand Down
8 changes: 5 additions & 3 deletions docs/source/features/multi_gpu.rst
Original file line number Diff line number Diff line change
Expand Up @@ -153,9 +153,11 @@ Run each benchmark with the same launcher options used by ``train_multigpu``:
--task Isaac-Cartpole --num_envs 4096 --max_iterations 100

``training_multigpu`` supports RSL-RL, RL-Games, and skrl with PyTorch. It does
not support skrl with JAX or SB3. It also rejects ``--video``,
``--capture_env_sensors``, and ``--check_success``, which do not produce a
meaningful aggregate result across ranks.
not support skrl with JAX or SB3. It also rejects ``--video`` and
``--capture_env_sensors``, which do not produce a meaningful aggregate result
across ranks. ``--check_success`` is supported with RSL-RL and RL-Games: the
success metric is averaged over the environments of all ranks, so every rank
stops at the same iteration.

For multi-node benchmarks, pass the same ``--nnodes``, ``--node_rank``, and
rendezvous options described in :ref:`multi-node-training` on every node.
Expand Down
4 changes: 2 additions & 2 deletions scripts/benchmarks/benchmark_newton_raycast.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ class RaycastBenchSceneCfg(InteractiveSceneCfg):
prim_path="{ENV_REGEX_NS}/SensorBody",
spawn=sim_utils.CuboidCfg(
size=(0.1, 0.1, 0.1),
rigid_props=sim_utils.RigidBodyPropertiesCfg(),
mass_props=sim_utils.MassPropertiesCfg(mass=1.0),
rigid_props=sim_utils.UsdPhysicsRigidBodyCfg(),
mass_props=sim_utils.MassCfg(mass=1.0),
),
init_state=RigidObjectCfg.InitialStateCfg(pos=(0.0, 0.0, 1.0)),
)
Expand Down
18 changes: 9 additions & 9 deletions scripts/benchmarks/benchmark_view_comparison.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,9 @@ def benchmark_usd_or_fabric(view_type: str, num_iterations: int) -> dict[str, fl
object_cfg = sim_utils.ConeCfg(
radius=0.15,
height=0.5,
rigid_props=sim_utils.RigidBodyPropertiesCfg(),
mass_props=sim_utils.MassPropertiesCfg(mass=1.0),
collision_props=sim_utils.CollisionPropertiesCfg(),
rigid_props=sim_utils.UsdPhysicsRigidBodyCfg(),
mass_props=sim_utils.MassCfg(mass=1.0),
collision_props=sim_utils.UsdPhysicsCollisionCfg(),
visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 1.0, 0.0)),
)
for i in range(args_cli.num_envs):
Expand Down Expand Up @@ -151,9 +151,9 @@ class _SceneCfg(InteractiveSceneCfg):
prim_path="{ENV_REGEX_NS}/Cube",
spawn=sim_utils.CuboidCfg(
size=(0.2, 0.2, 0.2),
rigid_props=sim_utils.RigidBodyPropertiesCfg(),
mass_props=sim_utils.MassPropertiesCfg(mass=1.0),
collision_props=sim_utils.CollisionPropertiesCfg(),
rigid_props=sim_utils.UsdPhysicsRigidBodyCfg(),
mass_props=sim_utils.MassCfg(mass=1.0),
collision_props=sim_utils.UsdPhysicsCollisionCfg(),
),
init_state=RigidObjectCfg.InitialStateCfg(pos=(0.0, 0.0, 1.0)),
)
Expand Down Expand Up @@ -207,9 +207,9 @@ def benchmark_physx(num_iterations: int) -> dict[str, float]:
object_cfg = sim_utils.ConeCfg(
radius=0.15,
height=0.5,
rigid_props=sim_utils.RigidBodyPropertiesCfg(),
mass_props=sim_utils.MassPropertiesCfg(mass=1.0),
collision_props=sim_utils.CollisionPropertiesCfg(),
rigid_props=sim_utils.UsdPhysicsRigidBodyCfg(),
mass_props=sim_utils.MassCfg(mass=1.0),
collision_props=sim_utils.UsdPhysicsCollisionCfg(),
visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 1.0, 0.0)),
)
for i in range(args_cli.num_envs):
Expand Down
6 changes: 3 additions & 3 deletions scripts/benchmarks/benchmark_xform_prim_view.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,9 @@ class _NewtonSceneCfg(InteractiveSceneCfg):
prim_path="{ENV_REGEX_NS}/Object",
spawn=sim_utils.CuboidCfg(
size=(0.2, 0.2, 0.2),
rigid_props=sim_utils.RigidBodyPropertiesCfg(),
mass_props=sim_utils.MassPropertiesCfg(mass=1.0),
collision_props=sim_utils.CollisionPropertiesCfg(),
rigid_props=sim_utils.UsdPhysicsRigidBodyCfg(),
mass_props=sim_utils.MassCfg(mass=1.0),
collision_props=sim_utils.UsdPhysicsCollisionCfg(),
),
init_state=RigidObjectCfg.InitialStateCfg(pos=(0.0, 0.0, 1.0)),
)
Expand Down
2 changes: 2 additions & 0 deletions scripts/benchmarks/test/test_early_stop.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ def stop_logging_writer(self):
class _FakeRunner:
def __init__(self, has_writer: bool = True):
self.logger = _FakeLogger(has_writer=has_writer)
self.device = "cpu"
self.current_learning_iteration = 7
self.saved: list[str] = []

Expand Down Expand Up @@ -96,6 +97,7 @@ class _FakeAlgo:
def __init__(self, horizon_length: int | None = None, config_horizon: int | None = 16, epoch_num: int = 0):
self.max_epochs = 999
self.epoch_num = epoch_num
self.ppo_device = "cpu"
if horizon_length is not None:
self.horizon_length = horizon_length
self.config = {"horizon_length": config_horizon} if config_horizon is not None else {}
Expand Down
Loading
Loading