[4C/10] Route rigid rendering transforms through SDP

docs/source/developer-tools/scene_data_providers.rst

@@ -31,14 +31,19 @@ The system has three layers:
 1. :class:`~isaaclab.scene_data.SceneDataBackend`: a small interface implemented by each physics
    manager. It exposes the backend's transform array directly as one of the
    :class:`~isaaclab.scene_data.SceneDataFormat` Warp structs, plus the per-transform prim paths
-   and total count. There is no per-frame "update" call; the property accessors return live
-   views into the underlying tensor each time they're read.
+   and total count. Producers increment ``transforms_version`` after native state writes or buffer swaps;
+   SDP calls ``get_transforms(output_format)`` before reading the version, since resolving the pointer
+   can itself detect a swap. The default implementation returns the existing ``transforms`` property.
+   The version never resets, so independent readers cannot hide changes from one another.
 
-   - :attr:`SceneDataBackend.transforms`: current transforms as a Warp struct (one of
+   - :attr:`SceneDataBackend.transforms`: the native data as a Warp struct (one of
      :class:`SceneDataFormat.Vec3_Quat`, :class:`SceneDataFormat.Transform`,
      :class:`SceneDataFormat.Matrix44`, :class:`SceneDataFormat.Vec3_Matrix33`).
+   - :attr:`SceneDataBackend.transforms_version`: monotonic version of the native transforms.
    - :attr:`SceneDataBackend.transform_count`: number of transforms.
    - :attr:`SceneDataBackend.transform_paths`: list of USD prim paths, one per transform.
+   - :attr:`SceneDataBackend.native_transform_formats`: formats published without conversion.
+     PhysX publishes either packed poses or Fabric matrices and refreshes only the requested representation.
    - :attr:`SceneDataBackend.points`: flattened deformable nodal positions as
      :class:`SceneDataFormat.Points` (optional; rigid-only backends return an empty buffer).
    - :attr:`SceneDataBackend.point_count`: total number of geometry points.
@@ -48,11 +53,10 @@ The system has three layers:
 2. :class:`~isaaclab.scene_data.SceneDataProvider`: wraps a backend and offers format conversion
    plus index re-mapping.
 
-   - :meth:`SceneDataProvider.get_transforms`: writes the backend's transforms into a
-     consumer-provided :class:`SceneDataFormat` struct, optionally converting format
-     (e.g. ``Vec3_Quat`` to ``Transform``) and applying an index mapping. When the backend
-     format matches the output format and no mapping is provided, the result is a zero-copy
-     passthrough.
+   - :meth:`SceneDataProvider.get_transforms`: binds native arrays when format and ordering match,
+     or SDP-owned buffers converted once per producer version and destination layout. These shared
+     arrays are read-only, including when they replace preallocated output fields. Pass
+     ``allow_passthrough=False`` to write directly into caller-owned arrays instead.
    - :meth:`SceneDataProvider.create_mapping`: builds a remap array from the backend's prim
      paths to a consumer's desired ordering. Used when a renderer or visualizer wants
      transforms indexed by its own body list rather than by the physics view order.
@@ -86,10 +90,12 @@ When PhysX is the active physics backend, the provider reads transforms directly
 The transforms are returned as :class:`SceneDataFormat.Transform` (Warp ``transformf`` array),
 so consumers that want this format get them zero-copy.
 
-Newton-native consumers (Newton visualizer, Rerun, Viser, Newton Warp renderer, OVRTX renderer)
-also need a Newton ``Model``/``State`` to render against. To provide that,
-:class:`~isaaclab_newton.physics.NewtonManager` builds a **shadow Newton model** from the USD
-stage on first access and updates its ``body_q`` from the PhysX backend each render frame.
+Newton-native consumers (Newton visualizer, Rerun, Viser, Newton Warp renderer) also need a
+Newton ``Model``/``State``. Their declared cloning contexts construct that representation from
+the shared clone plan before initialization. Its rigid ``body_q`` binds to SDP's requested
+``Transform`` array; no intermediate per-frame copy into a second state buffer is required.
+OVRTX requests ``TransposedMatrix44d`` directly from SDP, including destination ordering and
+static scale in the same conversion. It no longer reads Newton state for rigid transforms.
 When the scene has PhysX or OVPhysX deformables, the shadow model also allocates
 ``particle_q`` render slots for soft/cloth meshes, syncs simulation nodal positions through
 :meth:`SceneDataProvider.get_points` with ``allow_passthrough=False`` into a separate
@@ -99,8 +105,26 @@ barycentric sim-to-visual remap so Newton Warp and OVRTX render the paired visua
 than tet simulation topology. The shadow deformable registry exposes render-slot offsets and
 ``particles_per_body`` counts for OVRTX point bindings.
 
-This is hidden behind :meth:`NewtonManager.get_model` / :meth:`NewtonManager.get_state`, so
-renderers don't need to know which physics backend is active.
+The deformable and cable geometry bridge remains separate from this rigid-transform path.
+OVRTX still uses Newton geometry metadata for those features.
+
+PhysX owns its native Fabric refresh and publishes the resulting matrices through SDP without
+fetching packed poses. ``isaaclab_physx.renderers.fabric.FabricBackend`` owns the shared native stage
+and hierarchy handles. Its identity is the stage and device, not the SDP source or attribute type.
+``SimulationContext`` declares ``fabric_cfg`` when Kit is available. After physics initializes, Kit,
+Isaac RTX, and explicit Fabric synchronization obtain the same resource through
+``get_or_create_backend(sim.fabric_cfg)``. Transform bindings are state on that resource, not a
+separate backend. Consumers pass the simulation's SDP to ``update_transforms(provider)``; for foreign
+physics it converts directly into Fabric local matrices, then propagates the GPU hierarchy.
+Core ``RenderContext`` owns no Fabric bindings.
+It binds rigid destinations as Fabric-only reset-stack roots because
+physics publishes absolute poses, including for nested bodies. Visual descendants still inherit
+their body's transform; authored USD is unchanged. Native source indices and world scales are
+bound once. Fabric's selection reuse API reports scene-wide structural changes; the resource refreshes
+array views without repeating path matching or scale capture. Otherwise GPU propagation
+reuses the hierarchy topology. Clean requests never acquire writable Fabric arrays.
+Renderers do not select a physics-specific synchronization path.
+``FabricMatrix44`` contains only matrix storage, not bindings or native engine handles.
 
 Newton backend
 --------------
@@ -109,11 +133,20 @@ When Newton is the active physics backend, the backend wraps the Newton model's
 directly. No shadow model or per-frame sync is needed: Newton already owns the authoritative
 model and state, and the provider exposes that state as :class:`SceneDataFormat.Transform`.
 
+Native reads reconcile pending authored state writes once. A new physics publication does not
+itself request forward kinematics. Kit/RTX requests current Fabric transforms through SDP
+before rendering, without issuing an additional physics ``forward()``. Headless viewport
+capture requests these transforms on demand rather than on every visualizer step.
+
+Externally replayed CUDA graphs do not call Python write hooks. After writes have been captured,
+Newton conservatively republishes transforms when read so an unannounced replay cannot leave
+rendering stale. Those reads do not benefit from clean-publication caching.
+
 Data requirements
 ------------------
 
 Visualizers and renderers declare what they need from the scene data path. This is resolved at
-simulation-context construction time and is what triggers the shadow-model build for PhysX:
+consumer construction time, before the shared clone plan is built:
 
 .. list-table::
    :header-rows: 1

source/isaaclab/changelog.d/sdp-transform-publication.major.rst

@@ -0,0 +1,19 @@
+Changed
+^^^^^^^
+
+* **Breaking:** Added ``transforms_version`` to scene-data backends. Custom backends must initialize
+  it to zero and increment it after native pose writes or buffer swaps. SDP reads the existing
+  ``transforms`` property through ``get_transforms(output_format)`` without resetting the version. Backends
+  publishing multiple native formats may override that method and ``native_transform_formats``.
+  ``SceneDataProvider.get_transforms`` bound shared,
+  read-only arrays by default: matching layouts aliased native data and other layouts converted
+  once per publication. Callers requiring their own writable or preallocated arrays must pass
+  ``allow_passthrough=False``; this wrote directly into the supplied arrays without a staging copy.
+* Routed rigid Fabric conversion through SDP while the Kit rendering integration owned destination binding and
+  GPU hierarchy propagation, preserving native PhysX publication and authored scale. Converted rigid destinations
+  became Fabric-only reset-stack roots so nested bodies retained their absolute physics poses.
+  Transform freshness no longer depended on the physics-step counter;
+  ``RenderContext.reset_scene_state_cadence`` remained available for geometry updates.
+  ``SimulationContext.fabric_cfg`` declared the shared native Fabric stage/device without allocating bindings.
+* Used native Warp structs for Fabric transform bindings, relying on the project-managed Warp
+  dependency selected by Isaac Lab's Kit launch configuration.

source/isaaclab/isaaclab/renderers/render_context.py

@@ -60,8 +60,8 @@ def _write_material(
 class RenderContext:
     """Orchestrate simulation-owned renderers and own flat runtime material buffers.
 
-    Renderer instances are borrowed from the simulation's backend registry. Scene state updates
-    run at most once per physics step, regardless of how many cameras share a renderer.
+    Renderer instances are borrowed from the simulation's backend registry. SDP owns transform
+    freshness, including pose writes that do not advance the physics-step counter.
     """
 
     __slots__ = (
@@ -70,7 +70,7 @@ class RenderContext:
         "_physics_initialized",
         "_prepared_renderer_ids",
         "_prepared_num_envs",
-        "_last_scene_state_step",
+        "_last_geometry_update_step",
         "_visual_materials",
         "_visual_material_batches",
         "_visual_material_batches_by_channel",
@@ -88,7 +88,7 @@ def __init__(self, backend_registry: list[tuple[BackendCfg, Any]]) -> None:
         self._physics_initialized: bool = False  # Set to True after the first PHYSICS_READY callback fires.
         self._prepared_renderer_ids: set[int] = set()
         self._prepared_num_envs: int | None = None
-        self._last_scene_state_step: int | None = None
+        self._last_geometry_update_step: int | None = None  # Physics step of the last renderer geometry update.
         self._visual_materials: list[Any] = []
         self._visual_material_batches: tuple[VisualMaterialBatch, ...] = ()
         self._visual_material_batches_by_channel: dict[str, VisualMaterialBatch] = {}
@@ -126,7 +126,7 @@ def validate_renderer_cfg(self, cfg: RendererCfg) -> None:
     def register_renderer(self, cfg: RendererCfg, renderer: BaseRenderer) -> None:
         """Include a newly registry-owned renderer in cloning and post-physics initialization."""
         self.clone_contexts.update(cfg.cloning_contexts)
-        self._last_scene_state_step = None
+        self._last_geometry_update_step = None
         if self._physics_initialized:
             renderer.initialize()
 
@@ -317,19 +317,15 @@ def ensure_prepare_stage(self, stage: Any, num_envs: int) -> None:
             self._prepared_num_envs = num_envs
 
     def update_scene_state(self, physics_step_count: int) -> None:
-        """Update scene state on all backends (at most once per step).
+        """Publish physics state and refresh renderers through SDP's producer versions.
 
-        Invokes :meth:`BaseRenderer.update_transforms` and then
-        :meth:`BaseRenderer.update_geometries` on each registered renderer.
+        Transforms follow SDP freshness; geometry updates retain their once-per-step cadence.
         """
-        if self._last_scene_state_step == physics_step_count:
-            return
-
         for _cfg, renderer in self._renderer_entries:
             renderer.update_transforms()
-            renderer.update_geometries()
-
-        self._last_scene_state_step = physics_step_count
+            if self._last_geometry_update_step != physics_step_count:
+                renderer.update_geometries()
+        self._last_geometry_update_step = physics_step_count
 
     def render_into_camera(
         self,
@@ -349,8 +345,8 @@ def reset_stage_prepare_flag(self) -> None:
         self._prepared_num_envs = None
 
     def reset_scene_state_cadence(self) -> None:
-        """Clear per-step scene state update dedupe (e.g. a long pause with no physics)."""
-        self._last_scene_state_step = None
+        """Invalidate geometry updates after resets that do not advance the physics step."""
+        self._last_geometry_update_step = None
 
     def close(self) -> None:
         """Release material writers and lifecycle bookkeeping, not registry-owned renderers.
@@ -368,7 +364,7 @@ def close(self) -> None:
         self.clone_contexts.clear()
         self._prepared_renderer_ids.clear()
         self._prepared_num_envs = None
-        self._last_scene_state_step = None
+        self._last_geometry_update_step = None
         self._physics_initialized = False
         self._visual_materials.clear()
         self._visual_material_batches = ()

source/isaaclab/isaaclab/scene/interactive_scene.py

@@ -498,8 +498,7 @@ def update(self, dt: float) -> None:
         Args:
             dt: The amount of time passed from last :meth:`update` call.
         """
-        # Scene-wide renderer scene-state sync once per step when all sensors update,
-        # so per-camera fetches do not own this concern (deduped inside RenderContext).
+        # Publish transforms before eager sensors read their Fabric-backed poses.
         if not self.cfg.lazy_sensor_update:
             self.sim.render_context.update_scene_state(self.sim.get_physics_step_count())
 

source/isaaclab/isaaclab/scene_data/__init__.pyi

@@ -3,12 +3,7 @@
 #
 # SPDX-License-Identifier: BSD-3-Clause
 
-__all__ = [
-    "REQUIRES_STAGE_AND_MODEL",
-    "SceneDataBackend",
-    "SceneDataFormat",
-    "SceneDataProvider",
-]
+__all__ = ["REQUIRES_STAGE_AND_MODEL", "SceneDataBackend", "SceneDataFormat", "SceneDataProvider"]
 
 from .scene_data_backend import SceneDataBackend, SceneDataFormat
 from .scene_data_provider import REQUIRES_STAGE_AND_MODEL, SceneDataProvider

source/isaaclab/isaaclab/scene_data/scene_data_backend.py

@@ -16,6 +16,8 @@
 
 from __future__ import annotations
 
+from typing import Any
+
 import warp as wp
 
 # Under Sphinx ``autodoc_mock_imports``, ``wp.struct`` is a ``_MockObject``
@@ -69,6 +71,20 @@ class Matrix44:
         matrices: wp.array(dtype=wp.mat44f) = None
         """Per-transform 4x4 homogeneous transform matrices [m]."""
 
+    @wp_struct
+    class TransposedMatrix44d:
+        """Double-precision row-vector transforms, as consumed by USD renderers."""
+
+        matrices: wp.array(dtype=wp.mat44d) = None
+        """World transforms [m], shape [transform_count]."""
+
+    @wp_struct
+    class FabricMatrix44:
+        """Double-precision row-vector matrices in native Fabric storage."""
+
+        matrices: wp.fabricarray(dtype=wp.mat44d) = None
+        """Transforms [m], shape [transform_count]."""
+
     @wp_struct
     class Points:
         """Flat world-space nodal or particle positions."""
@@ -78,13 +94,25 @@ class Points:
 
 
 class SceneDataBackend:
+    transforms_version: int
+    """Monotonic producer version, incremented after native writes or buffer swaps; never reset by readers."""
+
+    @property
+    def native_transform_formats(self) -> tuple[Any, ...]:
+        """Formats available without conversion, used when binding consumer destinations."""
+        return (self.transforms._cls,)
+
+    def get_transforms(self, output_format: Any) -> Any:
+        """Publish the requested native format when available, otherwise the primary format."""
+        return self.transforms
+
     @property
     def transforms(
         self,
     ) -> (
         SceneDataFormat.Vec3_Quat | SceneDataFormat.Transform | SceneDataFormat.Matrix44 | SceneDataFormat.Vec3_Matrix33
     ):
-        """Return the sim backends transforms as one of the SceneDataFormat structs."""
+        """Return native transforms without copying; pointer changes must increment ``transforms_version``."""
         raise NotImplementedError
 
     @property