Fix OVRTX scene-camera black frames from unscoped render-var lookups by matthewtrepte · Pull Request #7943 · isaac-sim/IsaacLab

matthewtrepte · 2026-09-22T06:00:30Z

Description

OVRTX 0.4 keys frame.render_vars by source name; 0.5+ keys it by the authored RenderVar prim path. The version-compat shim correctly detected this switch, but the static prim-path table it fell back to still used the pre-per-camera-scoping "/Render/Vars/" path. OVRTX 0.5+ actually authors each camera's render vars under its own scope, "/RenderCamera_/Vars/" (see render_scope_name in build_render_product_as_string, which already remaps correctly for authoring). The read side never applied that same remapping, so every frame.render_vars.get() silently returned None and the output buffer stayed at its zero-initialized (black) state.

Fixed by resolving each render-var key from the owning camera's render_scope_name (now stored on OVRTXCameraRenderData) via a new resolve_render_var_key() helper, instead of indexing the unscoped global RENDER_VAR_FRAME_KEYS table.

Verified against test_rendering_cartpole_kitless.py [ovstage-ovphysx-ovrtx_renderer-static]: before the fix it raised "render vars missing from the OVRTX frame"; after the fix it produces real (non-black) images matching the golden reference at SSIM 0.997, just outside the pixel-diff tolerance (likely minor rendering-noise drift, not a content bug).

Type of change

Bug fix (non-breaking change which fixes an issue)

Release backport

Backport this pull request to the active release branch after it merges into develop

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

Docker and GPU tests run on demand. Push the commits you want tested, then
comment run-ci on the pull request.

I have read and understood the contribution guidelines
I have run the pre-commit checks with ./isaaclab.sh --format
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump extension.toml — CI handles that)
I have added my name to the CONTRIBUTORS.md or my name already exists there

OVRTX 0.4 keys frame.render_vars by source name; 0.5+ keys it by the authored RenderVar prim path. The version-compat shim correctly detected this switch, but the static prim-path table it fell back to still used the pre-per-camera-scoping "/Render/Vars/<name>" path. OVRTX 0.5+ actually authors each camera's render vars under its own scope, "/RenderCamera_<id>/Vars/<name>" (see render_scope_name in build_render_product_as_string, which already remaps correctly for authoring). The read side never applied that same remapping, so every frame.render_vars.get() silently returned None and the output buffer stayed at its zero-initialized (black) state. Fixed by resolving each render-var key from the owning camera's render_scope_name (now stored on OVRTXCameraRenderData) via a new resolve_render_var_key() helper, instead of indexing the unscoped global RENDER_VAR_FRAME_KEYS table. Verified against test_rendering_cartpole_kitless.py [ovstage-ovphysx-ovrtx_renderer-static]: before the fix it raised "render vars missing from the OVRTX frame"; after the fix it produces real (non-black) images matching the golden reference at SSIM 0.997, just outside the pixel-diff tolerance (likely minor rendering-noise drift, not a content bug). Not caused by and unrelated to newton-physics/newton#4256 -- this code path is OVPhysX + kitless OVRTX rendering and does not touch Newton physics.

greptile-apps · 2026-09-22T06:03:10Z

Confidence Score: 4/5

The production lookup fix is internally consistent, but the PR should not merge until its OVRTX 0.5+ renderer contract tests are updated for scoped camera data.

Findings

Contract Tests Lose Scope ▶

Summary

This PR corrects OVRTX 0.5+ frame extraction by resolving each RenderVar key against its owning camera’s authored render scope instead of using the global unscoped path.

Adds per-camera render-scope state to OVRTXCameraRenderData.
Introduces version-aware, scope-aware RenderVar key resolution.
Applies scoped resolution to color, depth, segmentation, normals, motion-vector, and metadata AOVs.
Adds a changelog fragment documenting the black-frame fix.
The existing renderer contract fixtures were not updated for the new required scope state and fail on OVRTX 0.5+.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Camera setup] --> B[Author RenderVars under /RenderCamera_id/Vars]
  A --> C[Store render_scope_name]
  D[OVRTX frame] --> E[Resolve source using camera scope]
  C --> E
  E --> F[Lookup scoped RenderVar prim path]
  F --> G[Populate camera output buffers]

_{Reviews (1) · Last reviewed commit: "Fix OVRTX scene-camera black frames from..."}

greptile-apps · 2026-09-22T06:03:13Z

+    def _render_var_key(self, render_data: OVRTXCameraRenderData, source: str) -> str:
+        """Return *render_data*'s camera's ``frame.render_vars`` key for a render-var source name."""
+        return resolve_render_var_key(source, render_data.render_scope_name)


Contract Tests Lose Scope

On OVRTX 0.5+, the renderer contract tests now fail before processing a frame. Their OVRTXCameraRenderData fixture bypasses __init__ and does not define render_scope_name, but this code now accesses that attribute for every frame lookup, causing an AttributeError. The fixture and its frame keys need a camera scope so the new scoped lookup behavior is covered.

isaaclab-review-bot

Isaac Lab Review Bot

Replaces import-time render-var keys with per-camera key resolution using each camera’s authored render scope, addressing OVRTX 0.5+ black frames caused by unscoped lookups. The proposed unknown-source finding is not actionable for the changed consumers because every supplied source is a fixed, known entry from the existing render-var mapping.

Design and architecture: The camera scope is captured consistently in the legacy, ovstage, and additional-camera registration paths, while version-specific key behavior remains centralized in ovrtx_compat. The read-side remapping mirrors the render-product authoring convention; this creates a non-blocking maintenance dependency between the compatibility helper and build_render_product_as_string.
API: The existing RENDER_VAR_FRAME_KEYS symbol is retained, and the new resolve_render_var_key() interface is documented with modern typing and a Google-style docstring. A package changelog fragment documents the user-visible fix.
Implementation: The RGB, HDR, depth, albedo, normals, motion-vector, semantic, instance-segmentation, and segmentation metadata lookup paths all resolve keys through the owning camera’s scope. All changed call sites pass fixed source names already represented by render_var_prim_paths_by_source(), so the fallback for an unknown source does not establish a concrete failure in this patch.

No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.

Automated review; human maintainers own approval decisions.

_make_ovrtx_camera_render_data() built OVRTXCameraRenderData via __new__, bypassing __init__, so it never got the new render_scope_name attribute added for the black-frame fix -- three pre-existing tests crashed with AttributeError. Give the helper a render_scope_name parameter and set it. Also add a regression test that a camera with a known render_scope_name reads its own scoped render-var key (not the unscoped fallback), plus unit tests for resolve_render_var_key's OVRTX 0.4 and 0.5+ branches. Previously nothing exercised the actual per-camera scoping the fix introduces, so a regression there could pass CI while reintroducing the black-frame bug.

StafaH · 2026-09-22T14:25:29Z

run-ci

nvsekkin

The functional fix looks correct. Please also update the PR description to preserve the distinction noted inline: #7860 made Isaac Lab author per-camera scopes for every OVRTX version; the OVRTX 0.5-specific behavior is that frame.render_vars is keyed by those authored prim paths.

nvsekkin · 2026-09-22T15:57:34Z

+
+* Fixed :class:`~isaaclab_ov.renderers.ovrtx_renderer.OVRTXRenderer` producing all-black frames
+  for scene camera sensors on OVRTX 0.5+. Render-var lookups used a fixed, unscoped
+  ``"/Render/Vars/<name>"`` key, but OVRTX 0.5+ authors each camera's render vars under its own


note: per-camera scope is not specific to OVRTX-0.5. This behavior was a regression caused by a semantic merge conflict between #7860 and #7861. Please update this distinction here, in ovrtx_compat.py, the test docstring, and the PR description.

nvsekkin · 2026-09-22T15:58:03Z

+0.5 keys it by the authored RenderVar prim path. :data:`RENDER_VAR_FRAME_KEYS` gives the
+*single-camera* (unscoped) form of that path (``"/Render/Vars/LdrColor"``) for callers
+that do not have a specific camera's render scope. OVRTX 0.5+ actually authors render
+vars under a *per-camera* scope (``"/RenderCamera_<id>/Vars/LdrColor"``, see


same conclusion about OVRTX version here

nvsekkin · 2026-09-22T15:58:49Z



+def test_ovrtx_process_frame_reads_the_camera_scoped_key(monkeypatch: pytest.MonkeyPatch):
+    """OVRTX 0.5+ authors each camera's render vars under its own scope; frame reads for a


same conclusion about OVRTX version here

nvsekkin · 2026-09-22T16:06:19Z

question: are these version changes intentional? this patch does not touch any dependencies.

nvsekkin · 2026-09-22T16:07:01Z

thanks for this fix @matthewtrepte - I dropped some comments about docstrings and descriptions and I checked the backport toggle since this would have landed on release branch too.

matthewtrepte requested a review from a team September 22, 2026 06:00

matthewtrepte requested review from AntoineRichard, huidongc, marcodiiga, pbarejko and rilei-nvidia as code owners September 22, 2026 06:00

github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Sep 22, 2026

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

isaaclab-review-bot Bot reviewed Sep 22, 2026

View reviewed changes

matthewtrepte requested review from StafaH and kellyguo11 as code owners September 22, 2026 06:12

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 22, 2026

nvsekkin approved these changes Sep 22, 2026

View reviewed changes

nvsekkin reviewed Sep 22, 2026

View reviewed changes

nvsekkin mentioned this pull request Sep 22, 2026

Fix OVRTX 0.5 RenderVar lookups for per-camera scopes #7951

Closed

8 tasks

matthewtrepte closed this Sep 22, 2026

Navigation Menu

Fix OVRTX scene-camera black frames from unscoped render-var lookups - #7943

Fix OVRTX scene-camera black frames from unscoped render-var lookups#7943
matthewtrepte wants to merge 2 commits into
isaac-sim:developfrom
matthewtrepte:mtrepte/fix-ovrtx-black-frame-render-var-scope

matthewtrepte commented Sep 22, 2026 •

edited by nvsekkin

Loading

Uh oh!

greptile-apps Bot commented Sep 22, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot Sep 22, 2026

Uh oh!

isaaclab-review-bot Bot left a comment

Uh oh!

StafaH commented Sep 22, 2026

Uh oh!

nvsekkin left a comment •

edited

Loading

Uh oh!

nvsekkin Sep 22, 2026

Uh oh!

nvsekkin Sep 22, 2026

Uh oh!

nvsekkin Sep 22, 2026

Uh oh!

nvsekkin Sep 22, 2026

Uh oh!

nvsekkin commented Sep 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants



		def test_ovrtx_process_frame_reads_the_camera_scoped_key(monkeypatch: pytest.MonkeyPatch):
		"""OVRTX 0.5+ authors each camera's render vars under its own scope; frame reads for a

Conversation

matthewtrepte commented Sep 22, 2026 • edited by nvsekkin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Release backport

Screenshots

Checklist

Uh oh!

greptile-apps Bot commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 4/5

Findings

Summary

Diagram

Uh oh!

greptile-apps Bot Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

Isaac Lab Review Bot

Uh oh!

StafaH commented Sep 22, 2026

Uh oh!

nvsekkin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nvsekkin Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

nvsekkin Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

nvsekkin Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

nvsekkin Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

nvsekkin commented Sep 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

matthewtrepte commented Sep 22, 2026 •

edited by nvsekkin

Loading

greptile-apps Bot commented Sep 22, 2026 •

edited

Loading

nvsekkin left a comment •

edited

Loading