[pull] develop from isaac-sim:develop

docs/source/api/lab/isaaclab.sensors.rst

@@ -46,6 +46,29 @@
 Sensor Base
 -----------
 
+.. rubric:: Extending batch updates
+
+Sensors opt into eager batch updates by overriding
+:attr:`SensorBase.supports_batch_update`, which defaults to ``False``.
+Eager scene updates advance sensors in order, then batch the remaining buffer refreshes for
+sensors that opted in. Lazy data access remains per sensor.
+
+For standalone use, :meth:`SensorBase.update_batch` performs the complete eager update for a
+sequence of batch-capable sensors. It calls each sensor's ``update()`` once in the supplied order,
+then refreshes pending buffers together. Each call advances sensor clocks by ``dt``, so use it
+in place of separate ``sensor.update(dt)`` calls. An unsupported sensor raises ``ValueError``
+before any input is updated; update unsupported sensors individually.
+
+An implementation can override the static ``_update_buffers_batch_impl(sensors)`` hook to perform
+shared work. Sensors inheriting the same hook are grouped together, including instances of
+different subclasses. The hook must respect each sensor's outdated-environment mask and fill its
+buffers; ``SensorBase`` handles the update timestamps after the hook succeeds. The default hook
+calls each sensor's individual buffer implementation.
+
+Camera subclasses that override the individual buffer-update hooks retain individual updates by
+default. They must explicitly opt into batching and ensure their batch implementation preserves
+the custom behavior.
+
 .. autoclass:: SensorBase
     :members:
 

docs/source/concepts/renderers.rst

@@ -428,6 +428,36 @@ For the RTX renderer (requires Isaac Sim):
 For RTX renderer settings, see
 :doc:`/source/how-to/configure_rendering`.
 
+.. _renderer-camera-batching:
+
+Batching camera renders
+-----------------------
+
+:meth:`~isaaclab.renderers.BaseRenderer.render_batch` accepts a sequence of render-data objects
+owned by the same renderer. Prepare the camera poses, intrinsics, and shared scene state before
+rendering, then read each camera's output. An empty sequence performs no rendering.
+
+.. code-block:: python
+
+   renderer.render_batch([first_render_data, second_render_data])
+   renderer.read_output(first_render_data, first_camera_data)
+   renderer.read_output(second_render_data, second_camera_data)
+
+:meth:`~isaaclab.renderers.BaseRenderer.render` continues to accept a single render-data object.
+The default ``render_batch()`` implementation calls ``render()`` for each entry, so existing
+custom renderers and single-camera callers need no changes. OVRTX overrides ``render_batch()``
+to submit the requested camera products in one native renderer step.
+
+With eager sensor updates (``scene.cfg.lazy_sensor_update=False``), ``scene.update()`` advances
+sensor clocks in scene order and collects batch-capable sensors. After the loop, the camera
+batch implementation prepares the remaining due captures for submission.
+:meth:`~isaaclab.renderers.RenderContext.render_into_cameras` groups these requests by renderer
+instance, renders each group, and reads its outputs. The context does not retain a pending-camera
+queue. Each camera retains its own update period and reset state.
+
+With lazy sensor updates, reading a camera's ``data`` refreshes only that camera's sensor buffers
+and capture timestamps. It does not refresh peer camera sensors sharing the renderer.
+
 Core concepts
 -------------
 

docs/source/concepts/sensors/index.rst

@@ -22,6 +22,12 @@ sensor derives from :class:`~isaaclab.sensors.SensorBase` and follows the same l
   debug visualization is requested.
 * :meth:`~isaaclab.sensors.SensorBase.reset` clears per-environment timestamps and internal state.
 
+With eager sensor updates (``scene.cfg.lazy_sensor_update=False``), ``scene.update(dt)`` refreshes
+sensor data before returning. It advances sensor clocks in scene order, updating ordinary sensors
+individually and collecting sensors that support batch updates. After the loop, it refreshes the
+collected sensors' pending buffers together. Each sensor retains its own update period and reset
+state. Camera sensors use this mechanism to :ref:`batch captures by renderer <renderer-camera-batching>`.
+
 Sensor data is exposed through :class:`~isaaclab.utils.warp.ProxyArray` buffers, including camera
 outputs. Use the ``torch`` property for a cached zero-copy Torch view or ``warp`` for the underlying
 Warp array.

scripts/benchmarks/nsys_trace.json

@@ -89,6 +89,7 @@
         "color": "0xFFC107",
         "module": "isaaclab.renderers.render_context",
         "functions": [
+            {"function": "RenderContext.render_into_cameras", "color": "0xFFC107"},
             {"function": "RenderContext.render_into_camera", "color": "0xFFC107"},
             {"function": "RenderContext.update_scene_state", "color": "0xFFD54F"},
             {"function": "RenderContext.ensure_initialize", "color": "0xFFE082"},
@@ -266,6 +267,7 @@
             "OVRTXRenderer.update_geometries",
             "OVRTXRenderer.update_camera",
             "OVRTXRenderer.render",
+            "OVRTXRenderer.render_batch",
             "OVRTXRenderer.read_output",
             {"module": "isaaclab_ov.renderers.ovrtx_usd", "function": "create_scene_partition_attributes"},
             {"module": "isaaclab_ov.renderers.ovrtx_usd", "function": "export_stage_to_string"}

source/isaaclab/changelog.d/fix-ovphysx-joint-wrench-frame.skip

@@ -0,0 +1 @@
+Moved the shared joint-wrench physics test into the test tree and checked registration by every backend.

source/isaaclab/changelog.d/fix-task-space-body-offset-jacobian.rst

@@ -0,0 +1,8 @@
+Fixed
+^^^^^
+
+* Fixed the body-offset Jacobian correction in :class:`~isaaclab.envs.mdp.actions.DifferentialInverseKinematicsAction`
+  and :class:`~isaaclab.envs.mdp.actions.OperationalSpaceControllerAction`. The offset is now rotated into the root
+  frame by the body orientation before shifting the translational rows, and the angular rows are no longer rotated by
+  the offset rotation, matching the offset frame's pose and velocity. Tasks that set ``body_offset`` (for example, the
+  Franka IK tasks) now receive the Jacobian of the offset frame instead of an approximation.

source/isaaclab/changelog.d/render-batch.minor.rst

@@ -0,0 +1,10 @@
+Added
+^^^^^
+
+* Added ``BaseRenderer.render_batch()`` with a default loop over the existing single-camera
+  ``render()`` interface, allowing renderers to optimize multiple camera captures.
+* Added ``SensorBase.supports_batch_update`` to opt sensors into eager scene batching and
+  ``SensorBase.update_batch()`` for standalone updates of batch-capable sensors.
+* Added ``RenderContext.render_into_cameras()`` to group prepared captures by renderer and
+  batch due cameras during eager scene updates. Preserved per-camera lazy reads, update
+  periods, and reset state.

source/isaaclab/isaaclab/benchmark/stepping.py

@@ -65,24 +65,24 @@ def profile_renderers(
     originals = []
     try:
         for _, renderer in render_context._renderer_entries:
-            render = renderer.render
-            original = vars(renderer).get("render", missing)
+            render = renderer.render_batch
+            original = vars(renderer).get("render_batch", missing)
 
             @wraps(render)
             def timed_render(render_data: Any, _render=render) -> None:
                 with wp.ScopedTimer(RENDER_PROFILE_SCOPE, dict=scope_timings, print=False, synchronize=True):
                     return _render(render_data)
 
-            renderer.render = timed_render
+            renderer.render_batch = timed_render
             originals.append((renderer, original))
 
         yield timings
     finally:
         for renderer, original in reversed(originals):
             if original is missing:
-                del renderer.render
+                del renderer.render_batch
             else:
-                renderer.render = original
+                renderer.render_batch = original
 
 
 @contextmanager

source/isaaclab/isaaclab/envs/mdp/actions/task_space_actions.py

@@ -254,18 +254,14 @@ def _compute_frame_jacobian(self):
         self._jacobian_b[:] = self.jacobian_b
         # account for the offset
         if self.cfg.body_offset is not None:
-            # Modify the jacobian to account for the offset
-            # -- translational part
-            # v_link = v_ee + w_ee x r_link_ee = v_J_ee * q + w_J_ee * q x r_link_ee
-            #        = (v_J_ee + w_J_ee x r_link_ee ) * q
-            #        = (v_J_ee - r_link_ee_[x] @ w_J_ee) * q
-            self._jacobian_b[:, 0:3, :] += torch.bmm(
-                -math_utils.skew_symmetric_matrix(self._offset_pos), self._jacobian_b[:, 3:, :]
+            # Express the lever arm in root axes; a rigid offset leaves angular velocity unchanged.
+            body_quat_b = math_utils.quat_mul(
+                math_utils.quat_inv(self._asset.data.root_quat_w.torch),
+                self._asset.data.body_quat_w.torch[:, self._body_idx],
             )
-            # -- rotational part
-            # w_link = R_link_ee @ w_ee
-            self._jacobian_b[:, 3:, :] = torch.bmm(
-                math_utils.matrix_from_quat(self._offset_rot), self._jacobian_b[:, 3:, :]
+            offset_pos_b = math_utils.quat_apply(body_quat_b, self._offset_pos)
+            self._jacobian_b[:, 0:3, :] += torch.bmm(
+                -math_utils.skew_symmetric_matrix(offset_pos_b), self._jacobian_b[:, 3:, :]
             )
 
         return self._jacobian_b
@@ -673,19 +669,15 @@ def _compute_ee_jacobian(self):
 
         # account for the offset
         if self.cfg.body_offset is not None:
-            # Modify the jacobian to account for the offset
-            # -- translational part
-            # v_link = v_ee + w_ee x r_link_ee = v_J_ee * q + w_J_ee * q x r_link_ee
-            #        = (v_J_ee + w_J_ee x r_link_ee ) * q
-            #        = (v_J_ee - r_link_ee_[x] @ w_J_ee) * q
+            # Express the lever arm in root axes; a rigid offset leaves angular velocity unchanged.
+            body_quat_b = math_utils.quat_mul(
+                math_utils.quat_inv(self._asset.data.root_quat_w.torch),
+                self._asset.data.body_quat_w.torch[:, self._ee_body_idx],
+            )
+            offset_pos_b = math_utils.quat_apply(body_quat_b, self._offset_pos)
             self._jacobian_b[:, 0:3, :] += torch.bmm(
-                -math_utils.skew_symmetric_matrix(self._offset_pos), self._jacobian_b[:, 3:, :]
-            )  # type: ignore
-            # -- rotational part
-            # w_link = R_link_ee @ w_ee
-            self._jacobian_b[:, 3:, :] = torch.bmm(
-                math_utils.matrix_from_quat(self._offset_rot), self._jacobian_b[:, 3:, :]
-            )  # type: ignore
+                -math_utils.skew_symmetric_matrix(offset_pos_b), self._jacobian_b[:, 3:, :]
+            )
 
     def _compute_ee_pose(self):
         """Computes the pose of the ee frame in root frame."""

source/isaaclab/isaaclab/renderers/base_renderer.py

@@ -15,7 +15,7 @@
 from .output_contract import RenderBufferKind, RenderBufferSpec
 
 if TYPE_CHECKING:
-    from collections.abc import Callable
+    from collections.abc import Callable, Sequence
 
     import torch
     import warp as wp
@@ -183,6 +183,20 @@ def render(self, render_data: Any) -> None:
         """
         pass
 
+    def render_batch(self, render_data: Sequence[Any]) -> None:
+        """Render a collection of cameras into their bound output buffers.
+
+        All camera poses and shared scene state must be prepared before calling this method.
+        An empty sequence is a no-op. Each object must belong to this renderer and appear once.
+        The default implementation calls :meth:`render` for each camera; subclasses may override
+        this method to submit all cameras together.
+
+        Args:
+            render_data: Renderer-specific objects from :meth:`create_render_data`.
+        """
+        for data in render_data:
+            self.render(data)
+
     @abstractmethod
     def read_output(self, render_data: Any, camera_data: CameraData) -> None:
         """Read rendered outputs from the renderer into the camera data container.

source/isaaclab/isaaclab/renderers/render_context.py

@@ -9,6 +9,7 @@
 
 import logging
 import warnings
+from collections.abc import Sequence
 from typing import TYPE_CHECKING, Any
 
 import torch
@@ -334,10 +335,37 @@ def render_into_camera(
         camera_data: CameraData,
         physics_step_count: int,
     ) -> None:
-        """Sync scene state, render, and read outputs into ``camera_data``."""
+        """Sync scene state and capture one camera through :meth:`render_into_cameras`."""
+        self.render_into_cameras([(renderer, render_data, camera_data)], physics_step_count)
+
+    def render_into_cameras(
+        self,
+        requests: Sequence[tuple[BaseRenderer, Any, CameraData]],
+        physics_step_count: int,
+    ) -> None:
+        """Render prepared cameras in batches grouped by renderer instance.
+
+        Camera poses must be updated before this call. Requests are used only for this
+        submission; the context does not retain cameras or manage sensor timing.
+
+        Args:
+            requests: Tuples of renderer, renderer-specific render data, and output camera data.
+                An empty sequence performs no work.
+            physics_step_count: Current physics step for shared scene synchronization.
+        """
+        if not requests:
+            return
+
         self.update_scene_state(physics_step_count)
-        renderer.render(render_data)
-        renderer.read_output(render_data, camera_data)
+
+        groups: dict[int, tuple[BaseRenderer, list[tuple[Any, CameraData]]]] = {}
+        for renderer, render_data, camera_data in requests:
+            groups.setdefault(id(renderer), (renderer, []))[1].append((render_data, camera_data))
+
+        for renderer, cameras in groups.values():
+            renderer.render_batch([render_data for render_data, _ in cameras])
+            for render_data, camera_data in cameras:
+                renderer.read_output(render_data, camera_data)
 
     def reset_stage_prepare_flag(self) -> None:
         """Allow :meth:`ensure_prepare_stage` to run ``prepare_stage`` again (e.g. a new USD stage)."""

source/isaaclab/isaaclab/scene/interactive_scene.py

@@ -516,8 +516,19 @@ def update(self, dt: float) -> None:
         for surface_gripper in self._surface_grippers.values():
             surface_gripper.update(dt)
         # -- sensors
+        force_recompute = not self.cfg.lazy_sensor_update
+        batched_sensors: list[SensorBase] = []
         for sensor in self._sensors.values():
-            sensor.update(dt, force_recompute=not self.cfg.lazy_sensor_update)
+            if force_recompute and sensor.supports_batch_update:
+                # Defer buffer refresh until all sensors have advanced.
+                sensor.update(dt, force_recompute=False)
+                batched_sensors.append(sensor)
+            else:
+                sensor.update(dt, force_recompute=force_recompute)
+
+        # Process batched sensors only during eager updates.
+        if batched_sensors:
+            SensorBase._process_batch(batched_sensors)
 
     """
     Operations: Scene State.

source/isaaclab/isaaclab/sensors/camera/camera.py

@@ -8,7 +8,7 @@
 import logging
 import sys
 from collections.abc import Sequence
-from typing import TYPE_CHECKING, Any, Literal
+from typing import TYPE_CHECKING, Any, Literal, cast
 
 import numpy as np
 import torch
@@ -329,6 +329,18 @@ def __str__(self) -> str:
     def num_instances(self) -> int:
         return self._view.count
 
+    @property
+    def supports_batch_update(self) -> bool:
+        """Whether captures can use the shared camera batch implementation.
+
+        Custom scalar capture hooks retain their individual update path. Subclasses may opt
+        in explicitly when they also provide a compatible ``_update_buffers_batch_impl``.
+        """
+        return (
+            type(self)._update_buffers_impl is Camera._update_buffers_impl
+            and type(self)._update_outdated_buffers is SensorBase._update_outdated_buffers
+        )
+
     @property
     def data(self) -> CameraData:
         # update sensors if needed
@@ -703,15 +715,18 @@ def _initialize_impl(self):
         # Create internal buffers (includes intrinsic matrix and pose init)
         self._create_buffers()
 
-    def _update_buffers_impl(self, env_mask: wp.array):
-        if not self._env_mask_has_any(env_mask):
-            return
-        # Increment frame count
+    def _prepare_camera(self, env_mask: wp.array) -> None:
+        """Advance capture frames and refresh requested poses before rendering."""
         if self.cfg.update_latest_camera_pose:
             self._update_poses(env_mask=env_mask, frame_op=1)
         else:
             self._update_camera_state(env_mask=env_mask, frame_op=1)
 
+    def _update_buffers_impl(self, env_mask: wp.array):
+        if not self._env_mask_has_any(env_mask):
+            return
+        self._prepare_camera(env_mask)
+
         sim_ctx = sim_utils.SimulationContext.instance()
         renderer = self._renderer
         assert renderer is not None
@@ -726,6 +741,30 @@ def _update_buffers_impl(self, env_mask: wp.array):
             renderer.render(self._render_data)
             renderer.read_output(self._render_data, self._data)
 
+    @staticmethod
+    def _update_buffers_batch_impl(sensors: Sequence[SensorBase]) -> None:
+        """Prepare due cameras and render them together through their shared context."""
+        cameras = cast(Sequence[Camera], sensors)
+        sim_ctx = sim_utils.SimulationContext.instance()
+        if sim_ctx is None:
+            for camera in cameras:
+                camera._update_buffers_impl(camera._is_outdated)
+            return
+
+        ready = []
+        for camera in cameras:
+            if not camera._env_mask_has_any(camera._is_outdated):
+                continue
+            camera._prepare_camera(camera._is_outdated)
+            ready.append(camera)
+        if not ready:
+            return
+
+        sim_ctx.render_context.render_into_cameras(
+            [(camera._renderer, camera._render_data, camera._data) for camera in ready],
+            sim_ctx.get_physics_step_count(),
+        )
+
     """
     Private Helpers
     """