[Backport release/3.0.0] Rendering, backend writers, and Newton tests (#7941, #7977, #7983, #7991, #8003)

docs/source/api/lab/isaaclab.sensors.rst

@@ -46,6 +46,29 @@
 Sensor Base
 -----------
 
+.. rubric:: Extending batch updates
+
+Sensors opt into eager batch updates by overriding
+:attr:`SensorBase.supports_batch_update`, which defaults to ``False``.
+Eager scene updates advance sensors in order, then batch the remaining buffer refreshes for
+sensors that opted in. Lazy data access remains per sensor.
+
+For standalone use, :meth:`SensorBase.update_batch` performs the complete eager update for a
+sequence of batch-capable sensors. It calls each sensor's ``update()`` once in the supplied order,
+then refreshes pending buffers together. Each call advances sensor clocks by ``dt``, so use it
+in place of separate ``sensor.update(dt)`` calls. An unsupported sensor raises ``ValueError``
+before any input is updated; update unsupported sensors individually.
+
+An implementation can override the static ``_update_buffers_batch_impl(sensors)`` hook to perform
+shared work. Sensors inheriting the same hook are grouped together, including instances of
+different subclasses. The hook must respect each sensor's outdated-environment mask and fill its
+buffers; ``SensorBase`` handles the update timestamps after the hook succeeds. The default hook
+calls each sensor's individual buffer implementation.
+
+Camera subclasses that override the individual buffer-update hooks retain individual updates by
+default. They must explicitly opt into batching and ensure their batch implementation preserves
+the custom behavior.
+
 .. autoclass:: SensorBase
     :members:
 

docs/source/concepts/renderers.rst

@@ -428,6 +428,36 @@ For the RTX renderer (requires Isaac Sim):
 For RTX renderer settings, see
 :doc:`/source/how-to/configure_rendering`.
 
+.. _renderer-camera-batching:
+
+Batching camera renders
+-----------------------
+
+:meth:`~isaaclab.renderers.BaseRenderer.render_batch` accepts a sequence of render-data objects
+owned by the same renderer. Prepare the camera poses, intrinsics, and shared scene state before
+rendering, then read each camera's output. An empty sequence performs no rendering.
+
+.. code-block:: python
+
+   renderer.render_batch([first_render_data, second_render_data])
+   renderer.read_output(first_render_data, first_camera_data)
+   renderer.read_output(second_render_data, second_camera_data)
+
+:meth:`~isaaclab.renderers.BaseRenderer.render` continues to accept a single render-data object.
+The default ``render_batch()`` implementation calls ``render()`` for each entry, so existing
+custom renderers and single-camera callers need no changes. OVRTX overrides ``render_batch()``
+to submit the requested camera products in one native renderer step.
+
+With eager sensor updates (``scene.cfg.lazy_sensor_update=False``), ``scene.update()`` advances
+sensor clocks in scene order and collects batch-capable sensors. After the loop, the camera
+batch implementation prepares the remaining due captures for submission.
+:meth:`~isaaclab.renderers.RenderContext.render_into_cameras` groups these requests by renderer
+instance, renders each group, and reads its outputs. The context does not retain a pending-camera
+queue. Each camera retains its own update period and reset state.
+
+With lazy sensor updates, reading a camera's ``data`` refreshes only that camera's sensor buffers
+and capture timestamps. It does not refresh peer camera sensors sharing the renderer.
+
 Core concepts
 -------------
 

docs/source/concepts/sensors/index.rst

@@ -22,6 +22,12 @@ sensor derives from :class:`~isaaclab.sensors.SensorBase` and follows the same l
   debug visualization is requested.
 * :meth:`~isaaclab.sensors.SensorBase.reset` clears per-environment timestamps and internal state.
 
+With eager sensor updates (``scene.cfg.lazy_sensor_update=False``), ``scene.update(dt)`` refreshes
+sensor data before returning. It advances sensor clocks in scene order, updating ordinary sensors
+individually and collecting sensors that support batch updates. After the loop, it refreshes the
+collected sensors' pending buffers together. Each sensor retains its own update period and reset
+state. Camera sensors use this mechanism to :ref:`batch captures by renderer <renderer-camera-batching>`.
+
 Sensor data is exposed through :class:`~isaaclab.utils.warp.ProxyArray` buffers, including camera
 outputs. Use the ``torch`` property for a cached zero-copy Torch view or ``warp`` for the underlying
 Warp array.

docs/source/developer-tools/scene_data_providers.rst

@@ -31,14 +31,19 @@ The system has three layers:
 1. :class:`~isaaclab.scene_data.SceneDataBackend`: a small interface implemented by each physics
    manager. It exposes the backend's transform array directly as one of the
    :class:`~isaaclab.scene_data.SceneDataFormat` Warp structs, plus the per-transform prim paths
-   and total count. There is no per-frame "update" call; the property accessors return live
-   views into the underlying tensor each time they're read.
+   and total count. Producers increment ``transforms_version`` after native state writes or buffer swaps;
+   SDP calls ``get_transforms(output_format)`` before reading the version, since resolving the pointer
+   can itself detect a swap. The default implementation returns the existing ``transforms`` property.
+   The version never resets, so independent readers cannot hide changes from one another.
 
-   - :attr:`SceneDataBackend.transforms`: current transforms as a Warp struct (one of
+   - :attr:`SceneDataBackend.transforms`: the native data as a Warp struct (one of
      :class:`SceneDataFormat.Vec3_Quat`, :class:`SceneDataFormat.Transform`,
      :class:`SceneDataFormat.Matrix44`, :class:`SceneDataFormat.Vec3_Matrix33`).
+   - :attr:`SceneDataBackend.transforms_version`: monotonic version of the native transforms.
    - :attr:`SceneDataBackend.transform_count`: number of transforms.
    - :attr:`SceneDataBackend.transform_paths`: list of USD prim paths, one per transform.
+   - :attr:`SceneDataBackend.native_transform_formats`: formats published without conversion.
+     PhysX publishes either packed poses or Fabric matrices and refreshes only the requested representation.
    - :attr:`SceneDataBackend.points`: flattened deformable nodal positions as
      :class:`SceneDataFormat.Points` (optional; rigid-only backends return an empty buffer).
    - :attr:`SceneDataBackend.point_count`: total number of geometry points.
@@ -48,11 +53,10 @@ The system has three layers:
 2. :class:`~isaaclab.scene_data.SceneDataProvider`: wraps a backend and offers format conversion
    plus index re-mapping.
 
-   - :meth:`SceneDataProvider.get_transforms`: writes the backend's transforms into a
-     consumer-provided :class:`SceneDataFormat` struct, optionally converting format
-     (e.g. ``Vec3_Quat`` to ``Transform``) and applying an index mapping. When the backend
-     format matches the output format and no mapping is provided, the result is a zero-copy
-     passthrough.
+   - :meth:`SceneDataProvider.get_transforms`: binds native arrays when format and ordering match,
+     or SDP-owned buffers converted once per producer version and destination layout. These shared
+     arrays are read-only, including when they replace preallocated output fields. Pass
+     ``allow_passthrough=False`` to write directly into caller-owned arrays instead.
    - :meth:`SceneDataProvider.create_mapping`: builds a remap array from the backend's prim
      paths to a consumer's desired ordering. Used when a renderer or visualizer wants
      transforms indexed by its own body list rather than by the physics view order.
@@ -86,10 +90,12 @@ When PhysX is the active physics backend, the provider reads transforms directly
 The transforms are returned as :class:`SceneDataFormat.Transform` (Warp ``transformf`` array),
 so consumers that want this format get them zero-copy.
 
-Newton-native consumers (Newton visualizer, Rerun, Viser, Newton Warp renderer, OVRTX renderer)
-also need a Newton ``Model``/``State`` to render against. To provide that,
-:class:`~isaaclab_newton.physics.NewtonManager` builds a **shadow Newton model** from the USD
-stage on first access and updates its ``body_q`` from the PhysX backend each render frame.
+Newton-native consumers (Newton visualizer, Rerun, Viser, Newton Warp renderer) also need a
+Newton ``Model``/``State``. Their declared cloning contexts construct that representation from
+the shared clone plan before initialization. Its rigid ``body_q`` binds to SDP's requested
+``Transform`` array; no intermediate per-frame copy into a second state buffer is required.
+OVRTX requests ``TransposedMatrix44d`` directly from SDP, including destination ordering and
+static scale in the same conversion. It no longer reads Newton state for rigid transforms.
 When the scene has PhysX or OVPhysX deformables, the shadow model also allocates
 ``particle_q`` render slots for soft/cloth meshes, syncs simulation nodal positions through
 :meth:`SceneDataProvider.get_points` with ``allow_passthrough=False`` into a separate
@@ -99,8 +105,26 @@ barycentric sim-to-visual remap so Newton Warp and OVRTX render the paired visua
 than tet simulation topology. The shadow deformable registry exposes render-slot offsets and
 ``particles_per_body`` counts for OVRTX point bindings.
 
-This is hidden behind :meth:`NewtonManager.get_model` / :meth:`NewtonManager.get_state`, so
-renderers don't need to know which physics backend is active.
+The deformable and cable geometry bridge remains separate from this rigid-transform path.
+OVRTX still uses Newton geometry metadata for those features.
+
+PhysX owns its native Fabric refresh and publishes the resulting matrices through SDP without
+fetching packed poses. ``isaaclab_physx.renderers.fabric.FabricBackend`` owns the shared native stage
+and hierarchy handles. Its identity is the stage and device, not the SDP source or attribute type.
+``SimulationContext`` declares ``fabric_cfg`` when Kit is available. After physics initializes, Kit,
+Isaac RTX, and explicit Fabric synchronization obtain the same resource through
+``get_or_create_backend(sim.fabric_cfg)``. Transform bindings are state on that resource, not a
+separate backend. Consumers pass the simulation's SDP to ``update_transforms(provider)``; for foreign
+physics it converts directly into Fabric local matrices, then propagates the GPU hierarchy.
+Core ``RenderContext`` owns no Fabric bindings.
+It binds rigid destinations as Fabric-only reset-stack roots because
+physics publishes absolute poses, including for nested bodies. Visual descendants still inherit
+their body's transform; authored USD is unchanged. Native source indices and world scales are
+bound once. Fabric's selection reuse API reports scene-wide structural changes; the resource refreshes
+array views without repeating path matching or scale capture. Otherwise GPU propagation
+reuses the hierarchy topology. Clean requests never acquire writable Fabric arrays.
+Renderers do not select a physics-specific synchronization path.
+``FabricMatrix44`` contains only matrix storage, not bindings or native engine handles.
 
 Newton backend
 --------------
@@ -109,11 +133,20 @@ When Newton is the active physics backend, the backend wraps the Newton model's
 directly. No shadow model or per-frame sync is needed: Newton already owns the authoritative
 model and state, and the provider exposes that state as :class:`SceneDataFormat.Transform`.
 
+Native reads reconcile pending authored state writes once. A new physics publication does not
+itself request forward kinematics. Kit/RTX requests current Fabric transforms through SDP
+before rendering, without issuing an additional physics ``forward()``. Headless viewport
+capture requests these transforms on demand rather than on every visualizer step.
+
+Externally replayed CUDA graphs do not call Python write hooks. After writes have been captured,
+Newton conservatively republishes transforms when read so an unannounced replay cannot leave
+rendering stale. Those reads do not benefit from clean-publication caching.
+
 Data requirements
 ------------------
 
 Visualizers and renderers declare what they need from the scene data path. This is resolved at
-simulation-context construction time and is what triggers the shadow-model build for PhysX:
+consumer construction time, before the shared clone plan is built:
 
 .. list-table::
    :header-rows: 1

scripts/benchmarks/nsys_trace.json

@@ -89,6 +89,7 @@
         "color": "0xFFC107",
         "module": "isaaclab.renderers.render_context",
         "functions": [
+            {"function": "RenderContext.render_into_cameras", "color": "0xFFC107"},
             {"function": "RenderContext.render_into_camera", "color": "0xFFC107"},
             {"function": "RenderContext.update_scene_state", "color": "0xFFD54F"},
             {"function": "RenderContext.ensure_initialize", "color": "0xFFE082"},
@@ -266,6 +267,7 @@
             "OVRTXRenderer.update_geometries",
             "OVRTXRenderer.update_camera",
             "OVRTXRenderer.render",
+            "OVRTXRenderer.render_batch",
             "OVRTXRenderer.read_output",
             {"module": "isaaclab_ov.renderers.ovrtx_usd", "function": "create_scene_partition_attributes"},
             {"module": "isaaclab_ov.renderers.ovrtx_usd", "function": "export_stage_to_string"}

source/isaaclab/changelog.d/camera-render-spec-paths.major.rst

@@ -0,0 +1,6 @@
+Removed
+^^^^^^^
+
+* **Breaking:** Removed ``CameraRenderSpec.camera_path_relative_to_env_0``. Remove this argument
+  from render-spec constructors and use the absolute paths in ``camera_prim_paths`` instead.
+  OVRTX derived cloned camera paths from the authored source camera internally.

source/isaaclab/changelog.d/core-cleanup-backend-signatures.skip

@@ -0,0 +1 @@
+Test coverage for backend signature fixes.

source/isaaclab/changelog.d/render-batch.minor.rst

@@ -0,0 +1,10 @@
+Added
+^^^^^
+
+* Added ``BaseRenderer.render_batch()`` with a default loop over the existing single-camera
+  ``render()`` interface, allowing renderers to optimize multiple camera captures.
+* Added ``SensorBase.supports_batch_update`` to opt sensors into eager scene batching and
+  ``SensorBase.update_batch()`` for standalone updates of batch-capable sensors.
+* Added ``RenderContext.render_into_cameras()`` to group prepared captures by renderer and
+  batch due cameras during eager scene updates. Preserved per-camera lazy reads, update
+  periods, and reset state.

source/isaaclab/changelog.d/sdp-transform-publication.major.rst

@@ -0,0 +1,19 @@
+Changed
+^^^^^^^
+
+* **Breaking:** Added ``transforms_version`` to scene-data backends. Custom backends must initialize
+  it to zero and increment it after native pose writes or buffer swaps. SDP reads the existing
+  ``transforms`` property through ``get_transforms(output_format)`` without resetting the version. Backends
+  publishing multiple native formats may override that method and ``native_transform_formats``.
+  ``SceneDataProvider.get_transforms`` bound shared,
+  read-only arrays by default: matching layouts aliased native data and other layouts converted
+  once per publication. Callers requiring their own writable or preallocated arrays must pass
+  ``allow_passthrough=False``; this wrote directly into the supplied arrays without a staging copy.
+* Routed rigid Fabric conversion through SDP while the Kit rendering integration owned destination binding and
+  GPU hierarchy propagation, preserving native PhysX publication and authored scale. Converted rigid destinations
+  became Fabric-only reset-stack roots so nested bodies retained their absolute physics poses.
+  Transform freshness no longer depended on the physics-step counter;
+  ``RenderContext.reset_scene_state_cadence`` remained available for geometry updates.
+  ``SimulationContext.fabric_cfg`` declared the shared native Fabric stage/device without allocating bindings.
+* Used native Warp structs for Fabric transform bindings, relying on the project-managed Warp
+  dependency selected by Isaac Lab's Kit launch configuration.

source/isaaclab/isaaclab/benchmark/stepping.py

@@ -65,24 +65,24 @@ def profile_renderers(
     originals = []
     try:
         for _, renderer in render_context._renderer_entries:
-            render = renderer.render
-            original = vars(renderer).get("render", missing)
+            render = renderer.render_batch
+            original = vars(renderer).get("render_batch", missing)
 
             @wraps(render)
             def timed_render(render_data: Any, _render=render) -> None:
                 with wp.ScopedTimer(RENDER_PROFILE_SCOPE, dict=scope_timings, print=False, synchronize=True):
                     return _render(render_data)
 
-            renderer.render = timed_render
+            renderer.render_batch = timed_render
             originals.append((renderer, original))
 
         yield timings
     finally:
         for renderer, original in reversed(originals):
             if original is missing:
-                del renderer.render
+                del renderer.render_batch
             else:
-                renderer.render = original
+                renderer.render_batch = original
 
 
 @contextmanager

source/isaaclab/isaaclab/renderers/base_renderer.py

@@ -15,7 +15,7 @@
 from .output_contract import RenderBufferKind, RenderBufferSpec
 
 if TYPE_CHECKING:
-    from collections.abc import Callable
+    from collections.abc import Callable, Sequence
 
     import torch
     import warp as wp
@@ -183,6 +183,20 @@ def render(self, render_data: Any) -> None:
         """
         pass
 
+    def render_batch(self, render_data: Sequence[Any]) -> None:
+        """Render a collection of cameras into their bound output buffers.
+
+        All camera poses and shared scene state must be prepared before calling this method.
+        An empty sequence is a no-op. Each object must belong to this renderer and appear once.
+        The default implementation calls :meth:`render` for each camera; subclasses may override
+        this method to submit all cameras together.
+
+        Args:
+            render_data: Renderer-specific objects from :meth:`create_render_data`.
+        """
+        for data in render_data:
+            self.render(data)
+
     @abstractmethod
     def read_output(self, render_data: Any, camera_data: CameraData) -> None:
         """Read rendered outputs from the renderer into the camera data container.

source/isaaclab/isaaclab/renderers/camera_render_spec.py

@@ -23,15 +23,14 @@ class CameraRenderSpec:
         cfg: Camera configuration (data types, resolution, filters, etc.).
         device: Torch device string (e.g. ``"cuda:0"``) used by GPU annotators and Warp.
         num_instances: Number of tiled camera instances (environments).
-        camera_prim_paths: Absolute USD paths for each environment's camera prim.
-        view_count: Number of camera prims (must match ``len(camera_prim_paths)``).
-        camera_path_relative_to_env_0: Camera prim path with ``/World/envs/env_0/`` prefix
-            stripped; required by OVRTX. Empty string if the first camera is not under env 0.
+        camera_prim_paths: Absolute paths of the authored camera prims. When the renderer
+            clones environments internally, this may contain only the source camera path;
+            the renderer resolves its logical per-environment paths during registration.
+        view_count: Number of logical camera instances in the sensor view.
     """
 
     cfg: CameraCfg
     device: str
     num_instances: int
     camera_prim_paths: tuple[str, ...]
     view_count: int
-    camera_path_relative_to_env_0: str