From cd89038034eb4242c4730dc141d4723de6d7d989 Mon Sep 17 00:00:00 2001 From: Mustafa H <34825877+StafaH@users.noreply.github.com> Date: Fri, 25 Sep 2026 13:54:46 -0700 Subject: [PATCH 1/6] [Cleanup] Remove the leftover isaaclab_ovphysx directory (#8033) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Description `isaaclab_ovphysx` was merged into `isaaclab_ov` in #6992, but `source/isaaclab_ovphysx/` still exists. It holds a single empty changelog fragment, `changelog.d/vidurv-regex-fragment-targeting.skip`, added afterwards by #6640 under the old path. The directory has no `config/extension.toml`, so the changelog compiler never treats it as a package and never consumes the fragment. It would stay forever and keep the old package name visible in the source tree. This PR deletes that fragment, which removes the directory: - **No code or config changes.** Nothing in the workspace, CI or tooling refers to the directory. - **References kept on purpose.** The remaining mentions of `isaaclab_ovphysx` are historical changelog entries and a note in the issue-audit skill that the package was merged, and they stay. ## Type of change - Repository cleanup (no user-facing change) ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] My changes generate no new warnings - [x] I have added a changelog fragment under `source//changelog.d/` for every touched package (do **not** edit `CHANGELOG.rst` or bump `extension.toml` — CI handles that) - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there ## Release backport - [x] Backport this pull request to the active release branch after it merges into `develop` --- .../changelog.d/vidurv-regex-fragment-targeting.skip | 0 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 source/isaaclab_ovphysx/changelog.d/vidurv-regex-fragment-targeting.skip diff --git a/source/isaaclab_ovphysx/changelog.d/vidurv-regex-fragment-targeting.skip b/source/isaaclab_ovphysx/changelog.d/vidurv-regex-fragment-targeting.skip deleted file mode 100644 index e69de29bb2d1..000000000000 From 1db0ea5fcd0e1ebf39e172f999e7a2d79c807f16 Mon Sep 17 00:00:00 2001 From: Mustafa H <34825877+StafaH@users.noreply.github.com> Date: Fri, 25 Sep 2026 13:55:14 -0700 Subject: [PATCH 2/6] [Camera] Refactor and speed up camera-based tasks [1/7] (#8036) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Description Part 1 of the camera performance series (see the series list below). This part fixes a bug that also blocked the fast image path. `sensor.data.output[...]` returns a `ProxyArray`, whose `.dtype` is the Warp dtype. `mdp.image` and the Cartpole camera observations passed it straight into `normalize_camera_image`, so every `images.dtype == torch.uint8` check failed: - colorized semantic segmentation was only cast to float and reached policies as values in `[0, 255]` instead of `[0, 1]`; - the fused uint8 normalization kernel never ran. The image terms now read `.torch`. The other camera tasks that relied on the deprecated implicit `ProxyArray`-as-tensor conversion (Cartpole showcase, stack blueprint, drone ARL) do the same. ## Series 1. #8036: Read camera outputs through `ProxyArray.torch` (bug fix) (base: develop) 2. #8037: Fused normalization for strided camera images (base: #8036) 3. #8038: Skip redundant observation copies (base: #8037) 4. #8039: Remove per-step host synchronizations (base: develop) 5. #8040: Newton `render_batch` in one sensor-graph launch (base: develop) 6. #8041: RSL-RL cuDNN benchmark, mixed precision and compile settings (base: develop) 7. #8042: Kuka Allegro camera tasks on the shared image observation (base: #8038) The series is split from #7440, which remains the combined reference branch. ## Type of change - Bug fix (non-breaking change which fixes an issue) ## Validation - New `test_colorized_segmentation_is_scaled` (`mdp.image`) fails on `develop` and passes here. - The Cartpole camera observation test's fake sensor now returns a real `ProxyArray`, like actual sensors. With it, 4 of 8 cases fail on `develop` (the same segmentation bug) and all pass here. - `test_stacked_image_mdp.py`, `test_cartpole_camera_observations.py` and `test_images.py` pass. ## Series results Measured on `Isaac-Lift-KukaAllegro-Camera`, 4096 envs, Newton physics and Newton renderer, RTX PRO 6000 Blackwell, with all 7 parts applied (reference branch: #7440). | Metric | Before | After | |---|---|---| | Env step, RGB 128, one camera | 44.4 ms | 37.3 ms (+19% throughput) | | Env step, RGB 128, two cameras | 77.3 ms | 64.9 ms (+19%) | | Env step, RGB 64 | 26.8 ms | 23.6 ms (+14%) | | Env step, depth 128 | 37.3 ms | 33.7 ms (+10%) | | RSL-RL iteration, RGB 128, one camera | 3.19 s | 2.88 s (+11%), peak memory −40% | | RSL-RL iteration, RGB 128, two cameras | 6.23 s | 5.63 s (+11%), peak memory −47% | ## Release backport - [x] Backport this pull request to the active release branch after it merges into `develop` --- .../camera-perf-01-proxyarray-dtype.rst | 6 ++++++ .../isaaclab/envs/mdp/observations.py | 2 +- .../test/envs/test_stacked_image_mdp.py | 19 ++++++++++++++++--- .../camera-perf-01-proxyarray-dtype.rst | 5 +++++ .../cartpole_camera/cartpole_camera_env.py | 4 ++-- .../contrib/drone_arl/mdp/observations.py | 2 +- .../franka/stack_ik_rel_blueprint_env_cfg.py | 2 +- .../cartpole/cartpole_direct_camera_env.py | 4 ++-- .../core/cartpole/mdp/observations.py | 2 +- .../core/test_cartpole_camera_observations.py | 4 +++- 10 files changed, 38 insertions(+), 12 deletions(-) create mode 100644 source/isaaclab/changelog.d/camera-perf-01-proxyarray-dtype.rst create mode 100644 source/isaaclab_tasks/changelog.d/camera-perf-01-proxyarray-dtype.rst diff --git a/source/isaaclab/changelog.d/camera-perf-01-proxyarray-dtype.rst b/source/isaaclab/changelog.d/camera-perf-01-proxyarray-dtype.rst new file mode 100644 index 000000000000..0dd5001b5d1a --- /dev/null +++ b/source/isaaclab/changelog.d/camera-perf-01-proxyarray-dtype.rst @@ -0,0 +1,6 @@ +Fixed +^^^^^ + +* Fixed :func:`~isaaclab.envs.mdp.observations.image` passing the sensor's ``ProxyArray`` to the + image normalization, which left colorized semantic segmentation unscaled in ``[0, 255]`` and + skipped the fused normalization kernel. diff --git a/source/isaaclab/isaaclab/envs/mdp/observations.py b/source/isaaclab/isaaclab/envs/mdp/observations.py index 3ca6b9b1d404..5c38fb32d38e 100644 --- a/source/isaaclab/isaaclab/envs/mdp/observations.py +++ b/source/isaaclab/isaaclab/envs/mdp/observations.py @@ -395,7 +395,7 @@ def image( The images produced at the last time-step """ sensor: Camera | RayCasterCamera = env.scene.sensors[sensor_cfg.name] - images = sensor.data.output[data_type] + images = sensor.data.output[data_type].torch # depth image conversion if (data_type == "distance_to_camera") and convert_perspective_to_orthogonal: images = math_utils.orthogonalize_perspective_depth(images, sensor.data.intrinsic_matrices) diff --git a/source/isaaclab/test/envs/test_stacked_image_mdp.py b/source/isaaclab/test/envs/test_stacked_image_mdp.py index 16e3554a8c47..3c7ed1cdd0f0 100644 --- a/source/isaaclab/test/envs/test_stacked_image_mdp.py +++ b/source/isaaclab/test/envs/test_stacked_image_mdp.py @@ -16,10 +16,12 @@ import pytest import torch +import warp as wp pytestmark = pytest.mark.unit from isaaclab.envs.mdp.observations import image_features, stacked_image +from isaaclab.utils.warp import ProxyArray NUM_ENVS = 4 HEIGHT = 8 @@ -195,9 +197,9 @@ def test_consecutive_rgb_calls_return_independent_storage(self): ) -def _make_image_env_with_sensor(camera_buf: torch.Tensor) -> SimpleNamespace: - """Mock env exposing ``env.scene.sensors[name].data.output[type]`` = ``camera_buf``.""" - sensor = SimpleNamespace(data=SimpleNamespace(output={"rgb": camera_buf})) +def _make_image_env_with_sensor(camera_buf: torch.Tensor, data_type: str = "rgb") -> SimpleNamespace: + """Mock env exposing ``env.scene.sensors[name].data.output[type]`` as a ProxyArray over ``camera_buf``.""" + sensor = SimpleNamespace(data=SimpleNamespace(output={data_type: ProxyArray(wp.from_torch(camera_buf))})) scene = SimpleNamespace(sensors={"tiled_camera": sensor}) return SimpleNamespace(scene=scene, num_envs=NUM_ENVS, device="cpu") @@ -225,6 +227,17 @@ def test_clone_true_returns_independent_copy(self): out = image(env, sensor_cfg=cfg, data_type="rgb", normalize=False, clone=True) assert out.data_ptr() != camera_buf.data_ptr() + def test_colorized_segmentation_is_scaled(self): + """Colorized segmentation from a sensor ProxyArray is scaled like RGB, not only cast to float.""" + from isaaclab.envs.mdp.observations import image + + camera_buf = torch.full((NUM_ENVS, HEIGHT, WIDTH, 4), 255, dtype=torch.uint8) + camera_buf[:, 0, 0] = 0 + env = _make_image_env_with_sensor(camera_buf, "semantic_segmentation") + cfg = SimpleNamespace(name="tiled_camera") + out = image(env, sensor_cfg=cfg, data_type="semantic_segmentation") + assert out.max() <= 1.0 + def test_image_features_flattens_encoder_output(): """Feature extractors return a flat observation after the environment batch dimension.""" diff --git a/source/isaaclab_tasks/changelog.d/camera-perf-01-proxyarray-dtype.rst b/source/isaaclab_tasks/changelog.d/camera-perf-01-proxyarray-dtype.rst new file mode 100644 index 000000000000..80f43edcbabc --- /dev/null +++ b/source/isaaclab_tasks/changelog.d/camera-perf-01-proxyarray-dtype.rst @@ -0,0 +1,5 @@ +Fixed +^^^^^ + +* Fixed the Cartpole camera observations reading the sensor's ``ProxyArray`` directly, which left + colorized semantic segmentation unscaled. diff --git a/source/isaaclab_tasks/isaaclab_tasks/contrib/cartpole_showcase/cartpole_camera/cartpole_camera_env.py b/source/isaaclab_tasks/isaaclab_tasks/contrib/cartpole_showcase/cartpole_camera/cartpole_camera_env.py index 5e0c21f0ba70..31c5fc8e06bd 100644 --- a/source/isaaclab_tasks/isaaclab_tasks/contrib/cartpole_showcase/cartpole_camera/cartpole_camera_env.py +++ b/source/isaaclab_tasks/isaaclab_tasks/contrib/cartpole_showcase/cartpole_camera/cartpole_camera_env.py @@ -47,12 +47,12 @@ def _get_observations(self) -> dict: # get camera data data_type = "rgb" if "rgb" in self.cfg.scene.tiled_camera.data_types else "depth" if "rgb" in self.cfg.scene.tiled_camera.data_types: - camera_data = self._tiled_camera.data.output[data_type] / 255.0 + camera_data = self._tiled_camera.data.output[data_type].torch / 255.0 # normalize the camera data for better training results mean_tensor = torch.mean(camera_data, dim=(1, 2), keepdim=True) camera_data -= mean_tensor elif "depth" in self.cfg.scene.tiled_camera.data_types: - camera_data = self._tiled_camera.data.output[data_type] + camera_data = self._tiled_camera.data.output[data_type].torch camera_data[camera_data == float("inf")] = 0 # fundamental spaces diff --git a/source/isaaclab_tasks/isaaclab_tasks/contrib/drone_arl/mdp/observations.py b/source/isaaclab_tasks/isaaclab_tasks/contrib/drone_arl/mdp/observations.py index e084eabcd013..e04358aaa2dd 100644 --- a/source/isaaclab_tasks/isaaclab_tasks/contrib/drone_arl/mdp/observations.py +++ b/source/isaaclab_tasks/isaaclab_tasks/contrib/drone_arl/mdp/observations.py @@ -189,7 +189,7 @@ def __call__(self, env: ManagerBasedEnv, sensor_cfg: SceneEntityCfg, data_type: already stored during initialization. They are included in the signature only to satisfy the observation manager's parameter validation. """ - images = self.camera_sensor.data.output[self.data_type].clone() + images = self.camera_sensor.data.output[self.data_type].torch.clone() if (self.data_type == "distance_to_camera") and self.convert_perspective_to_orthogonal: images = math_utils.orthogonalize_perspective_depth(images, self.camera_sensor.data.intrinsic_matrices) diff --git a/source/isaaclab_tasks/isaaclab_tasks/contrib/stack/config/franka/stack_ik_rel_blueprint_env_cfg.py b/source/isaaclab_tasks/isaaclab_tasks/contrib/stack/config/franka/stack_ik_rel_blueprint_env_cfg.py index c96701e23ee6..e99cbb5755ca 100644 --- a/source/isaaclab_tasks/isaaclab_tasks/contrib/stack/config/franka/stack_ik_rel_blueprint_env_cfg.py +++ b/source/isaaclab_tasks/isaaclab_tasks/contrib/stack/config/franka/stack_ik_rel_blueprint_env_cfg.py @@ -66,7 +66,7 @@ def image( sensor: Camera | RayCasterCamera = env.scene.sensors[sensor_cfg.name] # obtain the input image - images = sensor.data.output[data_type] + images = sensor.data.output[data_type].torch # depth image conversion if (data_type == "distance_to_camera") and convert_perspective_to_orthogonal: diff --git a/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/cartpole_direct_camera_env.py b/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/cartpole_direct_camera_env.py index a0aca79ccf3d..de68649c3beb 100644 --- a/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/cartpole_direct_camera_env.py +++ b/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/cartpole_direct_camera_env.py @@ -55,7 +55,7 @@ def __init__(self, cfg: CartpoleCameraEnvCfg, render_mode: str | None = None, ** def _get_observations(self) -> dict: data_type = self.cfg.scene.tiled_camera.data_types[0] - camera_data = self._tiled_camera.data.output[data_type] + camera_data = self._tiled_camera.data.output[data_type].torch rgb_like = is_rgb_like(data_type) segmentation = data_type == "semantic_segmentation" @@ -92,7 +92,7 @@ def _get_observations(self) -> dict: obs = obs.clone() if self.cfg.write_image_to_file: - save_images_to_file(self._tiled_camera.data.output[data_type] / 255.0, f"cartpole_{data_type}.png") + save_images_to_file(self._tiled_camera.data.output[data_type].torch / 255.0, f"cartpole_{data_type}.png") critic_obs = super()._get_observations()["policy"] return {"policy": obs, "critic": critic_obs} diff --git a/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/mdp/observations.py b/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/mdp/observations.py index 0168b83ed7ef..919702006598 100644 --- a/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/mdp/observations.py +++ b/source/isaaclab_tasks/isaaclab_tasks/core/cartpole/mdp/observations.py @@ -45,7 +45,7 @@ def reset(self, env_ids: Sequence[int] | None = None) -> None: def __call__(self, env: ManagerBasedRLEnv, sensor_cfg: SceneEntityCfg, data_type: str) -> torch.Tensor: camera: Camera = env.scene.sensors[sensor_cfg.name] - camera_data = camera.data.output[data_type] + camera_data = camera.data.output[data_type].torch rgb_like = is_rgb_like(data_type) segmentation = data_type == "semantic_segmentation" diff --git a/source/isaaclab_tasks/test/core/test_cartpole_camera_observations.py b/source/isaaclab_tasks/test/core/test_cartpole_camera_observations.py index 1b12beffc5b9..beaef94858dc 100644 --- a/source/isaaclab_tasks/test/core/test_cartpole_camera_observations.py +++ b/source/isaaclab_tasks/test/core/test_cartpole_camera_observations.py @@ -16,8 +16,10 @@ import pytest import torch +import warp as wp from isaaclab.managers import ObservationTermCfg, SceneEntityCfg +from isaaclab.utils.warp import ProxyArray from isaaclab_tasks.core.cartpole.mdp.observations import CameraImageStack @@ -30,7 +32,7 @@ def _observe(images: torch.Tensor, frame_stack: int, device: str) -> torch.Tensor: """Run the observation term over ``images`` using a minimal environment stub.""" - camera = SimpleNamespace(data=SimpleNamespace(output={"semantic_segmentation": images})) + camera = SimpleNamespace(data=SimpleNamespace(output={"semantic_segmentation": ProxyArray(wp.from_torch(images))})) env = SimpleNamespace( cfg=SimpleNamespace(frame_stack=frame_stack), num_envs=images.shape[0], From 5e7a960e555d4321e6c0a1ff33faa1ad6cbd0428 Mon Sep 17 00:00:00 2001 From: Xu Xin Date: Sat, 26 Sep 2026 05:49:25 +0800 Subject: [PATCH 3/6] Fixes terrain origin computing in the inverted pyramid slope terrain (#4470) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Description Height-field terrain origins are now chosen by their generators and returned with the discretized height field. `height_field_to_mesh` places that origin in the padded mesh. The inverted pyramid slope origin therefore lies on its center platform for any platform width, including widths below 1 m. The duplicate `height_field_to_mesh_v2` decorator and its fixed sampling-width parameter have been removed. The built-in random and wave terrains retain their center-area maximum behavior. Other terrains with a flat center platform return its actual height. Custom height-field generators using the decorator must now return `(height_field, origin)`, where `origin` is a three-element position in meters relative to the generated height field. ## Validation - Regression test failed on `develop` for both tested inverted pyramid configurations and passes with this change. - `uv run --frozen python -m pytest source/isaaclab/test/terrains/test_terrain_generator.py -q` — 10 passed. - `uv run python tools/changelog/cli.py check develop` — passed. - `uv run isaaclab -f` — passed. - `uv run --isolated --extra dev -- make -C docs current-docs` — passed without warnings. ## Type of change - Bug fix with a height-field generator return-contract change (migration described above and in the changelog fragment). ## Screenshots | Before | After | | ------ | ----- | | ![Before](https://github.com/user-attachments/assets/eef5309e-6422-41a3-af43-c99a582532cb) | ![After](https://github.com/user-attachments/assets/bdd742ab-9c81-4b44-9455-cb41b6760a73) | ## Release backport - [x] Backport this pull request to the active release branch after it merges into `develop` --------- Co-authored-by: xuxin <747302550@qq.com> Co-authored-by: Mustafa Haiderbhai --- CONTRIBUTORS.md | 1 + .../height-field-generator-origins.rst | 8 ++ .../terrains/height_field/hf_terrains.py | 87 ++++++++++++------- .../isaaclab/terrains/height_field/utils.py | 15 ++-- .../test/terrains/test_terrain_generator.py | 21 +++++ 5 files changed, 92 insertions(+), 40 deletions(-) create mode 100644 source/isaaclab/changelog.d/height-field-generator-origins.rst diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 152fc43d0460..9cc6e035bfba 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -218,6 +218,7 @@ Guidelines for modifications: * Xiaodi Yuan * Xinjie Yao * Xinpeng Liu +* Xin Xu * Xu Li * Yang Jin * Yanzi Zhu diff --git a/source/isaaclab/changelog.d/height-field-generator-origins.rst b/source/isaaclab/changelog.d/height-field-generator-origins.rst new file mode 100644 index 000000000000..86721a7ef7f0 --- /dev/null +++ b/source/isaaclab/changelog.d/height-field-generator-origins.rst @@ -0,0 +1,8 @@ +Fixed +^^^^^ + +* **Breaking:** Fixed height-field terrain origins to use the location chosen by each generator. Inverted pyramid + slopes now place the origin on the center platform for any platform width. Height-field generator + functions wrapped with :func:`~isaaclab.terrains.height_field.utils.height_field_to_mesh` now return + ``(height_field, origin)``; custom generators must return their local origin [m] as a three-element + array alongside the discretized height field. diff --git a/source/isaaclab/isaaclab/terrains/height_field/hf_terrains.py b/source/isaaclab/isaaclab/terrains/height_field/hf_terrains.py index 3869eae25c3f..a5cbdb5acac3 100644 --- a/source/isaaclab/isaaclab/terrains/height_field/hf_terrains.py +++ b/source/isaaclab/isaaclab/terrains/height_field/hf_terrains.py @@ -19,7 +19,9 @@ @height_field_to_mesh -def random_uniform_terrain(difficulty: float, cfg: hf_terrains_cfg.HfRandomUniformTerrainCfg) -> np.ndarray: +def random_uniform_terrain( + difficulty: float, cfg: hf_terrains_cfg.HfRandomUniformTerrainCfg +) -> tuple[np.ndarray, np.ndarray]: """Generate a terrain with height sampled uniformly from a specified range. .. image:: ../../_static/terrains/height_field/random_uniform_terrain.jpg @@ -34,9 +36,7 @@ def random_uniform_terrain(difficulty: float, cfg: hf_terrains_cfg.HfRandomUnifo cfg: The configuration for the terrain. Returns: - The height field of the terrain as a 2D numpy array with discretized heights. - The shape of the array is (width, length), where width and length are the number of points - along the x and y axis, respectively. + The discretized height field with shape (width, length) and its origin [m] with shape (3,). Raises: ValueError: When the downsampled scale is smaller than the horizontal scale. @@ -77,11 +77,14 @@ def random_uniform_terrain(difficulty: float, cfg: hf_terrains_cfg.HfRandomUnifo y_upsampled = np.linspace(0, cfg.size[1] * cfg.horizontal_scale, length_pixels) z_upsampled = func(x_upsampled, y_upsampled) # round off the interpolated heights to the nearest vertical step - return np.rint(z_upsampled).astype(np.int16) + height_field = np.rint(z_upsampled).astype(np.int16) + return height_field, _terrain_origin(height_field, cfg) @height_field_to_mesh -def pyramid_sloped_terrain(difficulty: float, cfg: hf_terrains_cfg.HfPyramidSlopedTerrainCfg) -> np.ndarray: +def pyramid_sloped_terrain( + difficulty: float, cfg: hf_terrains_cfg.HfPyramidSlopedTerrainCfg +) -> tuple[np.ndarray, np.ndarray]: """Generate a terrain with a truncated pyramid structure. The terrain is a pyramid-shaped sloped surface with a slope of :obj:`slope` that trims into a flat platform @@ -102,9 +105,7 @@ def pyramid_sloped_terrain(difficulty: float, cfg: hf_terrains_cfg.HfPyramidSlop cfg: The configuration for the terrain. Returns: - The height field of the terrain as a 2D numpy array with discretized heights. - The shape of the array is (width, length), where width and length are the number of points - along the x and y axis, respectively. + The discretized height field with shape (width, length) and its origin [m] with shape (3,). """ # resolve terrain configuration if cfg.inverted: @@ -146,11 +147,14 @@ def pyramid_sloped_terrain(difficulty: float, cfg: hf_terrains_cfg.HfPyramidSlop hf_raw = np.clip(hf_raw, min(0, z_pf), max(0, z_pf)) # round off the heights to the nearest vertical step - return np.rint(hf_raw).astype(np.int16) + height_field = np.rint(hf_raw).astype(np.int16) + return height_field, _terrain_origin(height_field, cfg, height_field[center_x, center_y]) @height_field_to_mesh -def pyramid_stairs_terrain(difficulty: float, cfg: hf_terrains_cfg.HfPyramidStairsTerrainCfg) -> np.ndarray: +def pyramid_stairs_terrain( + difficulty: float, cfg: hf_terrains_cfg.HfPyramidStairsTerrainCfg +) -> tuple[np.ndarray, np.ndarray]: """Generate a terrain with a pyramid stair pattern. The terrain is a pyramid stair pattern which trims to a flat platform at the center of the terrain. @@ -169,9 +173,7 @@ def pyramid_stairs_terrain(difficulty: float, cfg: hf_terrains_cfg.HfPyramidStai cfg: The configuration for the terrain. Returns: - The height field of the terrain as a 2D numpy array with discretized heights. - The shape of the array is (width, length), where width and length are the number of points - along the x and y axis, respectively. + The discretized height field with shape (width, length) and its origin [m] with shape (3,). """ # resolve terrain configuration step_height = cfg.step_height_range[0] + difficulty * (cfg.step_height_range[1] - cfg.step_height_range[0]) @@ -207,11 +209,14 @@ def pyramid_stairs_terrain(difficulty: float, cfg: hf_terrains_cfg.HfPyramidStai hf_raw[start_x:stop_x, start_y:stop_y] = current_step_height # round off the heights to the nearest vertical step - return np.rint(hf_raw).astype(np.int16) + height_field = np.rint(hf_raw).astype(np.int16) + return height_field, _terrain_origin(height_field, cfg, height_field[width_pixels // 2, length_pixels // 2]) @height_field_to_mesh -def discrete_obstacles_terrain(difficulty: float, cfg: hf_terrains_cfg.HfDiscreteObstaclesTerrainCfg) -> np.ndarray: +def discrete_obstacles_terrain( + difficulty: float, cfg: hf_terrains_cfg.HfDiscreteObstaclesTerrainCfg +) -> tuple[np.ndarray, np.ndarray]: """Generate a terrain with randomly generated obstacles as pillars with positive and negative heights. The terrain is a flat platform at the center of the terrain with randomly generated obstacles as pillars @@ -228,9 +233,7 @@ def discrete_obstacles_terrain(difficulty: float, cfg: hf_terrains_cfg.HfDiscret cfg: The configuration for the terrain. Returns: - The height field of the terrain as a 2D numpy array with discretized heights. - The shape of the array is (width, length), where width and length are the number of points - along the x and y axis, respectively. + The discretized height field with shape (width, length) and its origin [m] with shape (3,). """ # resolve terrain configuration obs_height = cfg.obstacle_height_range[0] + difficulty * ( @@ -286,11 +289,12 @@ def discrete_obstacles_terrain(difficulty: float, cfg: hf_terrains_cfg.HfDiscret y2 = (length_pixels + platform_width) // 2 hf_raw[x1:x2, y1:y2] = 0 # round off the heights to the nearest vertical step - return np.rint(hf_raw).astype(np.int16) + height_field = np.rint(hf_raw).astype(np.int16) + return height_field, _terrain_origin(height_field, cfg, 0) @height_field_to_mesh -def wave_terrain(difficulty: float, cfg: hf_terrains_cfg.HfWaveTerrainCfg) -> np.ndarray: +def wave_terrain(difficulty: float, cfg: hf_terrains_cfg.HfWaveTerrainCfg) -> tuple[np.ndarray, np.ndarray]: r"""Generate a terrain with a wave pattern. The terrain is a flat platform at the center of the terrain with a wave pattern. The wave pattern @@ -313,9 +317,7 @@ def wave_terrain(difficulty: float, cfg: hf_terrains_cfg.HfWaveTerrainCfg) -> np cfg: The configuration for the terrain. Returns: - The height field of the terrain as a 2D numpy array with discretized heights. - The shape of the array is (width, length), where width and length are the number of points - along the x and y axis, respectively. + The discretized height field with shape (width, length) and its origin [m] with shape (3,). Raises: ValueError: When the number of waves is non-positive. @@ -347,11 +349,14 @@ def wave_terrain(difficulty: float, cfg: hf_terrains_cfg.HfWaveTerrainCfg) -> np # add the waves hf_raw += amplitude_pixels * (np.cos(yy * wave_number) + np.sin(xx * wave_number)) # round off the heights to the nearest vertical step - return np.rint(hf_raw).astype(np.int16) + height_field = np.rint(hf_raw).astype(np.int16) + return height_field, _terrain_origin(height_field, cfg) @height_field_to_mesh -def stepping_stones_terrain(difficulty: float, cfg: hf_terrains_cfg.HfSteppingStonesTerrainCfg) -> np.ndarray: +def stepping_stones_terrain( + difficulty: float, cfg: hf_terrains_cfg.HfSteppingStonesTerrainCfg +) -> tuple[np.ndarray, np.ndarray]: """Generate a terrain with a stepping stones pattern. The terrain is a stepping stones pattern which trims to a flat platform at the center of the terrain. @@ -365,9 +370,7 @@ def stepping_stones_terrain(difficulty: float, cfg: hf_terrains_cfg.HfSteppingSt cfg: The configuration for the terrain. Returns: - The height field of the terrain as a 2D numpy array with discretized heights. - The shape of the array is (width, length), where width and length are the number of points - along the x and y axis, respectively. + The discretized height field with shape (width, length) and its origin [m] with shape (3,). """ # resolve terrain configuration stone_width = cfg.stone_width_range[1] - difficulty * (cfg.stone_width_range[1] - cfg.stone_width_range[0]) @@ -434,4 +437,28 @@ def stepping_stones_terrain(difficulty: float, cfg: hf_terrains_cfg.HfSteppingSt y2 = (length_pixels + platform_width) // 2 hf_raw[x1:x2, y1:y2] = 0 # round off the heights to the nearest vertical step - return np.rint(hf_raw).astype(np.int16) + height_field = np.rint(hf_raw).astype(np.int16) + return height_field, _terrain_origin(height_field, cfg, 0) + + +def _terrain_origin( + height_field: np.ndarray, cfg: hf_terrains_cfg.HfTerrainBaseCfg, height: int | None = None +) -> np.ndarray: + """Return the origin [m] in the generated height field's local frame.""" + center_x = height_field.shape[0] // 2 + center_y = height_field.shape[1] // 2 + if height is None: + radius = max(1, int(1.0 / cfg.horizontal_scale)) + height = np.max( + height_field[ + max(0, center_x - radius) : center_x + radius, + max(0, center_y - radius) : center_y + radius, + ] + ) + return np.array( + [ + (height_field.shape[0] - 1) * cfg.horizontal_scale / 2, + (height_field.shape[1] - 1) * cfg.horizontal_scale / 2, + height * cfg.vertical_scale, + ] + ) diff --git a/source/isaaclab/isaaclab/terrains/height_field/utils.py b/source/isaaclab/isaaclab/terrains/height_field/utils.py index 16f327fef294..57eb80181b6c 100644 --- a/source/isaaclab/isaaclab/terrains/height_field/utils.py +++ b/source/isaaclab/isaaclab/terrains/height_field/utils.py @@ -25,8 +25,8 @@ def height_field_to_mesh(func: Callable) -> Callable: Additionally, it adds a border around the terrain to avoid artifacts at the edges. Args: - func: The height field function to convert. The function should return a 2D numpy array - with the heights of the terrain. + func: The height field function to convert. It should return the height field with discretized heights + and the terrain origin [m] relative to the generated height field. Returns: The mesh function. The mesh function returns a tuple containing a list of ``trimesh`` @@ -53,7 +53,7 @@ def wrapper(difficulty: float, cfg: HfTerrainBaseCfg): terrain_size = copy.deepcopy(cfg.size) cfg.size = tuple(sub_terrain_size) # generate the height field - z_gen = func(difficulty, cfg) + z_gen, origin = func(difficulty, cfg) # handle the border for the terrain heights[border_pixels:-border_pixels, border_pixels:-border_pixels] = z_gen # set terrain size back to config @@ -64,13 +64,8 @@ def wrapper(difficulty: float, cfg: HfTerrainBaseCfg): heights, cfg.horizontal_scale, cfg.vertical_scale, cfg.slope_threshold ) mesh = trimesh.Trimesh(vertices=vertices, faces=triangles) - # compute origin - x1 = int((cfg.size[0] * 0.5 - 1) / cfg.horizontal_scale) - x2 = int((cfg.size[0] * 0.5 + 1) / cfg.horizontal_scale) - y1 = int((cfg.size[1] * 0.5 - 1) / cfg.horizontal_scale) - y2 = int((cfg.size[1] * 0.5 + 1) / cfg.horizontal_scale) - origin_z = np.max(heights[x1:x2, y1:y2]) * cfg.vertical_scale - origin = np.array([0.5 * cfg.size[0], 0.5 * cfg.size[1], origin_z]) + # place the generator's origin in the padded mesh + origin[:2] += border_pixels * cfg.horizontal_scale return [mesh], origin return wrapper diff --git a/source/isaaclab/test/terrains/test_terrain_generator.py b/source/isaaclab/test/terrains/test_terrain_generator.py index 4ac34199cf49..5dd9d1d69fb4 100644 --- a/source/isaaclab/test/terrains/test_terrain_generator.py +++ b/source/isaaclab/test/terrains/test_terrain_generator.py @@ -18,6 +18,7 @@ TerrainGeneratorCfg, ) from isaaclab.terrains.config.rough import ROUGH_TERRAINS_CFG +from isaaclab.terrains.height_field import HfInvertedPyramidSlopedTerrainCfg from isaaclab.utils.seed import configure_seed pytestmark = pytest.mark.integration @@ -83,6 +84,26 @@ def test_repeated_objects_default_object_type(): assert origin.shape == (3,) +@pytest.mark.parametrize("platform_width,border_width", [(0.5, 0.0), (1.0, 0.0), (1.5, 0.2)]) +def test_inverted_pyramid_origin_matches_platform(platform_width: float, border_width: float): + cfg = HfInvertedPyramidSlopedTerrainCfg( + size=(8.0, 8.0), + horizontal_scale=0.1, + vertical_scale=0.005, + border_width=border_width, + slope_range=(0.4, 0.4), + platform_width=platform_width, + ) + meshes, origin = cfg.function(1.0, cfg) + center_vertices = meshes[0].vertices[ + np.isclose(meshes[0].vertices[:, 0], 4.0) & np.isclose(meshes[0].vertices[:, 1], 4.0) + ] + + np.testing.assert_allclose(origin[:2], (4.0, 4.0)) + assert len(center_vertices) == 1 + assert origin[2] == pytest.approx(center_vertices[0, 2]) + + @pytest.mark.parametrize("use_global_seed", [True, False]) def test_generation_reproducibility(use_global_seed): """Generates assorted terrains and tests that the resulting mesh is reproducible. From 6bbbd27694a6aa9251750633eb3d4a585baddffb Mon Sep 17 00:00:00 2001 From: Mustafa H <34825877+StafaH@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:04:55 -0700 Subject: [PATCH 4/6] [Core] Refactor and cleanup core subpackage for 3.0 [7/N] (#8043) # Description Seventh PR in the series that splits the core cleanup in #7949 into small, reviewable pieces. The test-only consolidation originally planned next is dropped, since #8022 already pruned the core tests with the test-audit skill. The series continues with source refactors, starting with `isaaclab.utils.string`. There is no behavior change. - `resolve_matching_names_values` duplicated the matching loop of `resolve_matching_names`, including its one-to-one bookkeeping, ordering, and error messages. Both now use the cached `_resolve_matching_names_impl`, which also returns the key index of each match (to pick the values) and whether every key matched (so `strict=False` still returns the partial matches). As a side effect, `resolve_matching_names_values` is now cached too; its output lists are rebuilt per call, as before. - The `preserve_order` reordering (two nested loops building a permutation) is a stable sort by key index. - `find_unique_string_name` no longer checks the `_1` candidate twice; `find_root_prim_path_from_regex` returns early. Their docstrings drop types that duplicate the annotations. - The "Not all regular expressions are matched!" message now always lists the available strings as a list (it printed a tuple for `resolve_matching_names`). ## Verification - A randomized comparison of the `develop` and PR implementations (4,000 random target/pattern sets through both resolvers with every `preserve_order` / `strict` combination, plus `find_unique_string_name` and `find_root_prim_path_from_regex`; 8,506 cases) gives identical results and exception messages, apart from the list-vs-tuple formatting above. - `test/utils/test_string.py`, `test/utils/test_dict.py`, `test/actuators`, and `test/assets/test_articulation_iface.py` pass. - The dict-subclass value-pairing regression failed before the fix and passed after it; `test/utils/test_string.py` passes (18 tests), and `uv run isaaclab -f` passes. ## Type of change - Code cleanup (non-breaking, no functional change) ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] I have added a changelog fragment under `source//changelog.d/` for every touched package (`.skip`) - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there ## Release backport - [x] Backport this pull request to the active release branch after it merges into `develop` --- .../core-cleanup-string-matchers.skip | 1 + source/isaaclab/isaaclab/utils/string.py | 179 +++++------------- source/isaaclab/test/utils/test_string.py | 8 + 3 files changed, 61 insertions(+), 127 deletions(-) create mode 100644 source/isaaclab/changelog.d/core-cleanup-string-matchers.skip diff --git a/source/isaaclab/changelog.d/core-cleanup-string-matchers.skip b/source/isaaclab/changelog.d/core-cleanup-string-matchers.skip new file mode 100644 index 000000000000..2ada183a868a --- /dev/null +++ b/source/isaaclab/changelog.d/core-cleanup-string-matchers.skip @@ -0,0 +1 @@ +Unified the regex name matchers in isaaclab.utils.string; behavior is unchanged. diff --git a/source/isaaclab/isaaclab/utils/string.py b/source/isaaclab/isaaclab/utils/string.py index f2a1d3632891..c399310c749e 100644 --- a/source/isaaclab/isaaclab/utils/string.py +++ b/source/isaaclab/isaaclab/utils/string.py @@ -270,22 +270,20 @@ def _resolve_matching_names_impl( list_of_strings: tuple[str, ...], preserve_order: bool, raise_when_no_match: bool, -) -> tuple[tuple[int, ...], tuple[str, ...]]: - """Cached implementation of :func:`resolve_matching_names`. +) -> tuple[tuple[int, ...], tuple[str, ...], tuple[int, ...], bool]: + """Cached implementation shared by :func:`resolve_matching_names` and :func:`resolve_matching_names_values`. - All arguments are hashable so that ``functools.cache`` can store results. - Returns tuples (immutable) to protect the cached data from mutation; - the public wrapper converts these back to fresh lists for each caller. + All arguments are hashable so that ``functools.cache`` can store results. Returns immutable tuples + (matched indices, matched names, index of the key each match came from, whether every key matched) + to protect the cached data from mutation; the public wrappers convert these back to fresh lists. """ - # find matching patterns - index_list = [] - names_list = [] - key_idx_list = [] + index_list: list[int] = [] + names_list: list[str] = [] + key_idx_list: list[int] = [] # book-keeping to check that we always have a one-to-one mapping # i.e. each target string should match only one regular expression - target_strings_match_found = [None for _ in range(len(list_of_strings))] - keys_match_found = [[] for _ in range(len(keys))] - # loop over all target strings + target_strings_match_found: list[str | None] = [None] * len(list_of_strings) + keys_match_found: list[list[str]] = [[] for _ in keys] for target_index, potential_match_string in enumerate(list_of_strings): for key_index, re_key in enumerate(keys): if re.fullmatch(re_key, potential_match_string): @@ -299,38 +297,23 @@ def _resolve_matching_names_impl( names_list.append(potential_match_string) key_idx_list.append(key_index) keys_match_found[key_index].append(potential_match_string) - # reorder keys if they should be returned in order of the query keys + # matches are collected in target order; a stable sort by key groups them in query order instead if preserve_order: - reordered_index_list = [None] * len(index_list) - global_index = 0 - for key_index in range(len(keys)): - for key_idx_position, key_idx_entry in enumerate(key_idx_list): - if key_idx_entry == key_index: - reordered_index_list[key_idx_position] = global_index - global_index += 1 - # reorder index and names list - index_list_reorder = [None] * len(index_list) - names_list_reorder = [None] * len(index_list) - for idx, reorder_idx in enumerate(reordered_index_list): - index_list_reorder[reorder_idx] = index_list[idx] - names_list_reorder[reorder_idx] = names_list[idx] - # update - index_list = index_list_reorder - names_list = names_list_reorder - # check that all regular expressions are matched - if not all(keys_match_found): - if not raise_when_no_match: - return (), () + order = sorted(range(len(index_list)), key=key_idx_list.__getitem__) + index_list = [index_list[i] for i in order] + names_list = [names_list[i] for i in order] + key_idx_list = [key_idx_list[i] for i in order] + all_matched = all(keys_match_found) + if not all_matched and raise_when_no_match: # make this print nicely aligned for debugging msg = "\n" for key, value in zip(keys, keys_match_found): msg += f"\t{key}: {value}\n" - msg += f"Available strings: {list_of_strings}\n" + msg += f"Available strings: {list(list_of_strings)}\n" raise ValueError( f"Not all regular expressions are matched! Please check that the regular expressions are correct: {msg}" ) - # return immutable tuples for safe caching - return tuple(index_list), tuple(names_list) + return tuple(index_list), tuple(names_list), tuple(key_idx_list), all_matched def resolve_matching_names( @@ -377,15 +360,19 @@ def resolve_matching_names( ValueError: When not all regular expressions are matched and :attr:`raise_when_no_match` is True. """ _keys = (keys,) if isinstance(keys, str) else tuple(keys) - idx, names = _resolve_matching_names_impl(_keys, tuple(list_of_strings), preserve_order, raise_when_no_match) + idx, names, _, all_matched = _resolve_matching_names_impl( + _keys, tuple(list_of_strings), preserve_order, raise_when_no_match + ) + if not all_matched: + return [], [] return list(idx), list(names) def clear_resolve_matching_names_cache() -> None: - """Discard all cached results from :func:`resolve_matching_names`. + """Discard cached results shared by the name and name-value resolvers. Call this when the simulation scene is torn down so that cached - name-resolution entries from destroyed assets do not accumulate + entries from :func:`resolve_matching_names` and :func:`resolve_matching_names_values` do not accumulate across scene rebuilds in long-lived processes. """ _resolve_matching_names_impl.cache_clear() @@ -400,10 +387,8 @@ def resolve_matching_names_values( """Match a list of regular expressions in a dictionary against a list of strings and return the matched indices, names, and values. - Note: - Unlike :func:`resolve_matching_names`, this function is not cached. Current callers - use it during initialization only (e.g. action/actuator config resolution), so caching - would add complexity without a measurable benefit. + Regex matching results are cached, but values are read from ``data`` on every call. Use + :func:`clear_resolve_matching_names_cache` to discard the cached matching results. If the :attr:`preserve_order` is False, the ordering of the matched indices and names is the same as the order of the provided list of strings. This means that the ordering is dictated by the order of the target strings @@ -434,67 +419,12 @@ def resolve_matching_names_values( """ if not isinstance(data, dict): raise TypeError(f"Input argument `data` should be a dictionary. Received: {data}") - # find matching patterns - index_list = [] - names_list = [] - values_list = [] - key_idx_list = [] - # book-keeping to check that we always have a one-to-one mapping - # i.e. each target string should match only one regular expression - target_strings_match_found = [None for _ in range(len(list_of_strings))] - keys_match_found = [[] for _ in range(len(data))] - # loop over all target strings - for target_index, potential_match_string in enumerate(list_of_strings): - for key_index, (re_key, value) in enumerate(data.items()): - if re.fullmatch(re_key, potential_match_string): - # check if match already found - if target_strings_match_found[target_index]: - raise ValueError( - f"Multiple matches for '{potential_match_string}':" - f" '{target_strings_match_found[target_index]}' and '{re_key}'!" - ) - # add to list - target_strings_match_found[target_index] = re_key - index_list.append(target_index) - names_list.append(potential_match_string) - values_list.append(value) - key_idx_list.append(key_index) - # add for regex key - keys_match_found[key_index].append(potential_match_string) - # reorder keys if they should be returned in order of the query keys - if preserve_order: - reordered_index_list = [None] * len(index_list) - global_index = 0 - for key_index in range(len(data)): - for key_idx_position, key_idx_entry in enumerate(key_idx_list): - if key_idx_entry == key_index: - reordered_index_list[key_idx_position] = global_index - global_index += 1 - # reorder index and names list - index_list_reorder = [None] * len(index_list) - names_list_reorder = [None] * len(index_list) - values_list_reorder = [None] * len(index_list) - for idx, reorder_idx in enumerate(reordered_index_list): - index_list_reorder[reorder_idx] = index_list[idx] - names_list_reorder[reorder_idx] = names_list[idx] - values_list_reorder[reorder_idx] = values_list[idx] - # update - index_list = index_list_reorder - names_list = names_list_reorder - values_list = values_list_reorder - # check that all regular expressions are matched - if strict and not all(keys_match_found): - # make this print nicely aligned for debugging - msg = "\n" - for key, value in zip(data.keys(), keys_match_found): - msg += f"\t{key}: {value}\n" - msg += f"Available strings: {list_of_strings}\n" - # raise error - raise ValueError( - f"Not all regular expressions are matched! Please check that the regular expressions are correct: {msg}" - ) - # return - return index_list, names_list, values_list + items = tuple(data.items()) + idx, names, key_idx, _ = _resolve_matching_names_impl( + tuple(key for key, _ in items), tuple(list_of_strings), preserve_order, strict + ) + values = [value for _, value in items] + return list(idx), list(names), [values[i] for i in key_idx] def _resolve_matching_values_dense(value: dict[str, float | int] | float | int, names: list[str]) -> tuple[float, ...]: @@ -517,45 +447,40 @@ def _resolve_matching_values_dense(value: dict[str, float | int] | float | int, def find_unique_string_name(initial_name: str, is_unique_fn: Callable[[str], bool]) -> str: """Find a unique string name based on the predicate function provided. - The string is appended with "_N", where N is a natural number till the resultant string - is unique. + + The string is appended with "_N", where N is a natural number, until the resultant string is unique. + Args: - initial_name (str): The initial string name. - is_unique_fn (Callable[[str], bool]): The predicate function to validate against. + initial_name: The initial string name. + is_unique_fn: The predicate function to validate against. + Returns: - str: A unique string based on input function. + A unique string based on input function. """ if is_unique_fn(initial_name): return initial_name iterator = 1 - result = initial_name + "_" + str(iterator) - while not is_unique_fn(result): - result = initial_name + "_" + str(iterator) + while not is_unique_fn(result := f"{initial_name}_{iterator}"): iterator += 1 return result def find_root_prim_path_from_regex(prim_path_regex: str) -> tuple[str, int]: """Find the first prim above the regex pattern prim and its position. + Args: - prim_path_regex (str): full prim path including the regex pattern prim. + prim_path_regex: Full prim path including the regex pattern prim. + Returns: - Tuple[str, int]: First position is the prim path to the parent of the regex prim. - Second position represents the level of the regex prim in the USD stage tree representation. + The prim path to the parent of the regex prim and the level of the regex prim in the USD stage tree. + Both are None when the path contains no regex pattern. """ + regex_chars = set("[]*|^") prim_paths_list = str(prim_path_regex).split("/") - root_idx = None - for prim_path_idx in range(len(prim_paths_list)): - chars = set("[]*|^") - if any((c in chars) for c in prim_paths_list[prim_path_idx]): - root_idx = prim_path_idx - break - root_prim_path = None - tree_level = None - if root_idx is not None: - root_prim_path = "/".join(prim_paths_list[:root_idx]) - tree_level = root_idx - return root_prim_path, tree_level + for root_idx, prim_path in enumerate(prim_paths_list): + if regex_chars.intersection(prim_path): + return "/".join(prim_paths_list[:root_idx]), root_idx + return None, None def list_intersection(list1: list[Any], list2: list[Any] | None) -> list[Any]: diff --git a/source/isaaclab/test/utils/test_string.py b/source/isaaclab/test/utils/test_string.py index 4dbfb07b8bd0..cd454a8e043e 100644 --- a/source/isaaclab/test/utils/test_string.py +++ b/source/isaaclab/test/utils/test_string.py @@ -206,6 +206,14 @@ def test_resolve_matching_names_values_with_basic_strings(): assert index_list == [0, 1, 2, 3, 4] assert names_list == ["a", "b", "c", "d", "e"] assert values_list == [1, 2, 2, 1, 1] + + class ReverseIterationDict(dict): + def __iter__(self): + return iter(reversed(list(super().keys()))) + + data = ReverseIterationDict({"a": 1, "b": 2}) + assert string_utils.resolve_matching_names_values(data, ["a", "b"]) == ([0, 1], ["a", "b"], [1, 2]) + # test matching names with regex data = {"a|d|e|b": 1, "b|c": 2} with pytest.raises(ValueError): From ba97a1a243298eddcf2fe2777363e135f07490e5 Mon Sep 17 00:00:00 2001 From: Mustafa H <34825877+StafaH@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:45:56 -0700 Subject: [PATCH 5/6] [Tests] Split and trim the contrib environments smoke test (#8034) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Split contrib smoke tests by runtime and sample one task per family The original contrib job ran 83 environments serially in one camera-enabled Isaac Sim process. This change separates runtime requirements and intentionally runs **one representative per task package, robot directory, and runtime**. Adding variants within a family does not add smoke tests. - **Runtime routing:** `sim_launcher.scan()` selects kitless, Kit without cameras, or Kit with cameras. Kit and kitless each use two pytest workers; cameras use one process. The job has five slots and restores the RTX shader cache. - **Family grouping:** preserve the complete robot subdirectory, including OpenArm's separate unimanual and bimanual configurations. - **Representative selection:** prefer runnable rough-terrain variants to include height-scanner coverage, then the shortest task ID. Keep one visible skip for all-skipped families. There is no list of additional same-family cases. The selection contains **47 runnable environments and 4 skips**, versus 83 runnable environments and 12 skips originally. Rough-terrain representatives replace flat-terrain runs without increasing the number of tests per family. ## Validation - Registry-wide collection check confirms exactly one representative for every family/runtime, with no duplicate task IDs: 34 Kit entries (4 skipped), 11 kitless, and 6 camera entries. - The three changed representatives—bimanual OpenArm, AnymalC Direct Rough, and Digit Rough—passed local simulation smoke tests. - Formatting, lint, changelog checks, and `git diff --check` passed. No production code changed. The initial 46-case version completed CI in **6:43**, versus **24:19** on develop on the same runner type (RTX PRO 4500 Blackwell). Those timings precede the nested-robot and representative-selection changes; the final 47-case job needs a fresh CI measurement. ## Type of change - CI / test infrastructure (no user-facing change) ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] My changes generate no new warnings - [x] I have added a changelog fragment under `source//changelog.d/` for every touched package (do **not** edit `CHANGELOG.rst` or bump `extension.toml` — CI handles that) - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there ## Release backport - [x] Backport this pull request to the active release branch after it merges into `develop` --------- Co-authored-by: Octi Zhang --- .github/workflows/build.yaml | 9 +- .../ci-xdist-contrib-environments.skip | 2 + .../test/contrib/contrib_env_test_utils.py | 114 ++++++++++++++++++ .../test/contrib/test_contrib_environments.py | 76 ------------ .../contrib/test_contrib_environments_kit.py | 37 ++++++ .../test_contrib_environments_kit_cameras.py | 33 +++++ .../test_contrib_environments_kitless.py | 26 ++++ tools/test_settings.py | 12 +- 8 files changed, 230 insertions(+), 79 deletions(-) create mode 100644 source/isaaclab_tasks/changelog.d/ci-xdist-contrib-environments.skip create mode 100644 source/isaaclab_tasks/test/contrib/contrib_env_test_utils.py delete mode 100644 source/isaaclab_tasks/test/contrib/test_contrib_environments.py create mode 100644 source/isaaclab_tasks/test/contrib/test_contrib_environments_kit.py create mode 100644 source/isaaclab_tasks/test/contrib/test_contrib_environments_kit_cameras.py create mode 100644 source/isaaclab_tasks/test/contrib/test_contrib_environments_kitless.py diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml index bafbebb87671..c8baf1c9e369 100644 --- a/.github/workflows/build.yaml +++ b/.github/workflows/build.yaml @@ -800,8 +800,15 @@ jobs: isaacsim-version: ${{ needs.config.outputs.isaacsim_image_tag }} dockerfile-path: docker/Dockerfile.curobo cache-tag: cache-curobo - include-files: "test_contrib_environments.py" + include-files: >- + test_contrib_environments_kitless.py, + test_contrib_environments_kit.py, + test_contrib_environments_kit_cameras.py warp-cache: restore + ovrtx-shader-cache: restore + ovrtx-shader-cache-trees: kit + # The three files run side by side; the kit and kitless files split across two workers each. + test-jobs: "5" container-name: isaac-lab-contrib-environments-test test-record-video: diff --git a/source/isaaclab_tasks/changelog.d/ci-xdist-contrib-environments.skip b/source/isaaclab_tasks/changelog.d/ci-xdist-contrib-environments.skip new file mode 100644 index 000000000000..e42f93534650 --- /dev/null +++ b/source/isaaclab_tasks/changelog.d/ci-xdist-contrib-environments.skip @@ -0,0 +1,2 @@ +Split the contributed-environment smoke test by the runtime each environment needs. +Kept one smoke test per family, preserving nested robot directories and preferring rough-terrain coverage. diff --git a/source/isaaclab_tasks/test/contrib/contrib_env_test_utils.py b/source/isaaclab_tasks/test/contrib/contrib_env_test_utils.py new file mode 100644 index 000000000000..fa7caa535479 --- /dev/null +++ b/source/isaaclab_tasks/test/contrib/contrib_env_test_utils.py @@ -0,0 +1,114 @@ +# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md). +# All rights reserved. +# +# SPDX-License-Identifier: BSD-3-Clause + +"""Shared parametrization for the contributed-environment smoke tests. + +The smoke tests are split by the runtime each environment needs, so a test process only starts what its +environments use: ``kitless`` environments run without Isaac Sim, ``kit`` environments need Isaac Sim for +PhysX but no renderer, and ``kit_cameras`` environments also need the RTX renderer, the expensive part of +starting Isaac Sim. The runtime comes from :func:`isaaclab.app.sim_launcher.scan`, the same check +``launch_simulation`` uses to decide whether to start Isaac Sim, so a new environment lands in the right file +without being listed anywhere. + +Contributed environments are intentionally sampled once per task package, robot directory, and runtime. +Additional variants in the same family do not add smoke tests. +""" + +from collections import defaultdict +from typing import Literal + +import gymnasium as gym +import pytest + +from isaaclab.app.sim_launcher import scan + +from isaaclab_tasks.utils.parse_cfg import parse_env_cfg + +# Local imports should be imported last +from env_test_utils import setup_environment # isort: skip + +Runtime = Literal["kitless", "kit", "kit_cameras"] + +_SKIPPED_TASKS = { + "IsaacContrib-AutoMate-Assembly-Direct": "Requires CUDA support outside the standard environment test runner.", + "IsaacContrib-AutoMate-Disassembly-Direct": "Requires CUDA support outside the standard environment test runner.", +} +_SKIPPED_TASK_SUBSTRINGS = { + # Under random actions the Kamino P-ADMM solver intermittently diverges and the whole robot state + # (root pose, joint state) turns NaN mid-episode, so the run fails nondeterministically (about 1 in 12 + # seeds locally; the sibling HoldPose task stays finite). The termination terms cannot catch a NaN state. + # Re-enable once the solver instability is resolved upstream. + "DrLegs-Walk": "Kamino solver intermittently produces NaN robot state under random actions.", + "RmpFlow": "Uses SingleArticulation, which requires an update.", + "Skillgen": "Requires cuRobo-specific coverage.", + "Suction": "Requires CPU simulation.", +} +_COVERED_TASKS = [ + "IsaacContrib-Lift-Cube-Franka", # Already covered by test_environment_determinism.py +] + + +def _skip_reason(task_name: str) -> str | None: + """Return the documented reason for skipping a contributed environment.""" + if task_name in _SKIPPED_TASKS: + return _SKIPPED_TASKS[task_name] + return next((reason for substring, reason in _SKIPPED_TASK_SUBSTRINGS.items() if substring in task_name), None) + + +def task_runtime(task_name: str) -> Runtime: + """Return the runtime an environment's default configuration launches with.""" + config_scan = scan(parse_env_cfg(task_name)) + if not config_scan.needs_kit: + return "kitless" + return "kit_cameras" if config_scan.has_kit_camera else "kit" + + +def _variant_family(task_name: str) -> tuple[str, str]: + """Return the task package and robot directory an environment's configuration is defined in. + + Contributed tasks keep per-robot configurations under ``/config//``; a task without + that layout is its own robot. Preserve nested robot directories, such as OpenArm's unimanual and + bimanual configurations. + """ + entry_point = gym.spec(task_name).kwargs["env_cfg_entry_point"] + module = entry_point.partition(":")[0] if isinstance(entry_point, str) else entry_point.__module__ + parts = module.split(".") + if "config" not in parts[:-1]: + return ".".join(parts[:-1]), "" + config_index = parts.index("config") + robot = ".".join(parts[config_index + 1 : -1]) + return ".".join(parts[:config_index]), robot + + +def contrib_environment_params(runtime: Runtime) -> list: + """Return exactly one environment per task package and robot directory for ``runtime``. + + Prefer runnable rough-terrain variants, which also exercise the height scanner, then the shortest task ID. + Keep one entry for all-skipped families so their skip reason stays visible. + """ + tasks_by_family: dict[tuple[str, str, Runtime], list[str]] = defaultdict(list) + task_marks = {} + for task_param in setup_environment(multi_agent=False, tier="contrib", exclude_task_names=_COVERED_TASKS): + task_name = getattr(task_param, "values", (task_param,))[0] + task_marks[task_name] = getattr(task_param, "marks", ()) + tasks_by_family[(*_variant_family(task_name), task_runtime(task_name))].append(task_name) + + params = [] + for (_, _, family_runtime), task_names in sorted(tasks_by_family.items()): + if family_runtime != runtime: + continue + task_name = min( + task_names, key=lambda name: (_skip_reason(name) is not None, "-Rough-" not in name, len(name), name) + ) + marks = task_marks[task_name] + if (skip_reason := _skip_reason(task_name)) is not None: + marks = (*marks, pytest.mark.skip(reason=skip_reason)) + params.append(pytest.param(task_name, id=task_name, marks=marks)) + return params + + +def num_envs(task_name: str) -> int: + """Return how many environments the smoke test steps for a task.""" + return 3 if task_name == "IsaacContrib-Multitask-Manipulation" else 2 diff --git a/source/isaaclab_tasks/test/contrib/test_contrib_environments.py b/source/isaaclab_tasks/test/contrib/test_contrib_environments.py deleted file mode 100644 index 3d2b961e244d..000000000000 --- a/source/isaaclab_tasks/test/contrib/test_contrib_environments.py +++ /dev/null @@ -1,76 +0,0 @@ -# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md). -# All rights reserved. -# -# SPDX-License-Identifier: BSD-3-Clause - -"""Launch Isaac Sim Simulator first.""" - -import sys - -# Import pinocchio before AppLauncher so Isaac Lab's dependency wins over Isaac Sim's bundled copy. -if sys.platform != "win32": - import pinocchio # noqa: F401 - -from isaaclab.app import AppLauncher - -app_launcher = AppLauncher(headless=True, enable_cameras=True) -simulation_app = app_launcher.app - - -"""Rest everything follows.""" - -import pytest - -import isaaclab_tasks # noqa: F401 - -# Local imports should be imported last -from env_test_utils import _run_environments, setup_environment # isort: skip - - -_SKIPPED_TASKS = { - "IsaacContrib-AutoMate-Assembly-Direct": "Requires CUDA support outside the standard environment test runner.", - "IsaacContrib-AutoMate-Disassembly-Direct": "Requires CUDA support outside the standard environment test runner.", -} -_SKIPPED_TASK_SUBSTRINGS = { - # Under random actions the Kamino P-ADMM solver intermittently diverges and the whole robot state - # (root pose, joint state) turns NaN mid-episode, so the run fails nondeterministically (about 1 in 12 - # seeds locally; the sibling HoldPose task stays finite). The termination terms cannot catch a NaN state. - # Re-enable once the solver instability is resolved upstream. - "DrLegs-Walk": "Kamino solver intermittently produces NaN robot state under random actions.", - "RmpFlow": "Uses SingleArticulation, which requires an update.", - "Skillgen": "Requires cuRobo-specific coverage.", - "Suction": "Requires CPU simulation.", -} -_COVERED_TASKS = [ - "IsaacContrib-Lift-Cube-Franka", # Already covered by test_environment_determinism.py -] - - -def _skip_reason(task_name: str) -> str | None: - """Return the documented reason for skipping a contributed environment.""" - if task_name in _SKIPPED_TASKS: - return _SKIPPED_TASKS[task_name] - return next((reason for substring, reason in _SKIPPED_TASK_SUBSTRINGS.items() if substring in task_name), None) - - -def _contrib_environment_params() -> list: - """Return each contributed environment with its documented test marks.""" - params = [] - for task_param in setup_environment( - multi_agent=False, - tier="contrib", - exclude_task_names=_COVERED_TASKS, - ): - task_name = getattr(task_param, "values", (task_param,))[0] - marks = getattr(task_param, "marks", ()) - skip_reason = _skip_reason(task_name) - if skip_reason is not None: - marks = (*marks, pytest.mark.skip(reason=skip_reason)) - params.append(pytest.param(task_name, id=task_name, marks=marks)) - return params - - -@pytest.mark.parametrize("task_name", _contrib_environment_params()) -def test_contrib_environments(task_name): - num_envs = 3 if task_name == "IsaacContrib-Multitask-Manipulation" else 2 - _run_environments(task_name, device="cuda", num_envs=num_envs) diff --git a/source/isaaclab_tasks/test/contrib/test_contrib_environments_kit.py b/source/isaaclab_tasks/test/contrib/test_contrib_environments_kit.py new file mode 100644 index 000000000000..4af2b90c96d0 --- /dev/null +++ b/source/isaaclab_tasks/test/contrib/test_contrib_environments_kit.py @@ -0,0 +1,37 @@ +# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md). +# All rights reserved. +# +# SPDX-License-Identifier: BSD-3-Clause + +"""Smoke tests for contributed environments that need Isaac Sim for physics but render nothing. + +Isaac Sim starts without cameras here: the RTX renderer is the expensive part of its startup, and every +environment that needs it runs in ``test_contrib_environments_kit_cameras.py`` instead. +""" + +import sys + +# Import pinocchio before AppLauncher so Isaac Lab's dependency wins over Isaac Sim's bundled copy. +if sys.platform != "win32": + import pinocchio # noqa: F401 + +from isaaclab.app import AppLauncher + +app_launcher = AppLauncher(headless=True) +simulation_app = app_launcher.app + + +"""Rest everything follows.""" + +import pytest + +import isaaclab_tasks # noqa: F401 + +# Local imports should be imported last +from contrib_env_test_utils import contrib_environment_params, num_envs # isort: skip +from env_test_utils import _run_environments # isort: skip + + +@pytest.mark.parametrize("task_name", contrib_environment_params("kit")) +def test_contrib_environments_kit(task_name): + _run_environments(task_name, device="cuda", num_envs=num_envs(task_name)) diff --git a/source/isaaclab_tasks/test/contrib/test_contrib_environments_kit_cameras.py b/source/isaaclab_tasks/test/contrib/test_contrib_environments_kit_cameras.py new file mode 100644 index 000000000000..a5e290ccda88 --- /dev/null +++ b/source/isaaclab_tasks/test/contrib/test_contrib_environments_kit_cameras.py @@ -0,0 +1,33 @@ +# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md). +# All rights reserved. +# +# SPDX-License-Identifier: BSD-3-Clause + +"""Smoke tests for contributed environments that render with Isaac Sim's RTX renderer.""" + +import sys + +# Import pinocchio before AppLauncher so Isaac Lab's dependency wins over Isaac Sim's bundled copy. +if sys.platform != "win32": + import pinocchio # noqa: F401 + +from isaaclab.app import AppLauncher + +app_launcher = AppLauncher(headless=True, enable_cameras=True) +simulation_app = app_launcher.app + + +"""Rest everything follows.""" + +import pytest + +import isaaclab_tasks # noqa: F401 + +# Local imports should be imported last +from contrib_env_test_utils import contrib_environment_params, num_envs # isort: skip +from env_test_utils import _run_environments # isort: skip + + +@pytest.mark.parametrize("task_name", contrib_environment_params("kit_cameras")) +def test_contrib_environments_kit_cameras(task_name): + _run_environments(task_name, device="cuda", num_envs=num_envs(task_name)) diff --git a/source/isaaclab_tasks/test/contrib/test_contrib_environments_kitless.py b/source/isaaclab_tasks/test/contrib/test_contrib_environments_kitless.py new file mode 100644 index 000000000000..0b2d90f18aeb --- /dev/null +++ b/source/isaaclab_tasks/test/contrib/test_contrib_environments_kitless.py @@ -0,0 +1,26 @@ +# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md). +# All rights reserved. +# +# SPDX-License-Identifier: BSD-3-Clause + +"""Smoke tests for contributed environments that run without Isaac Sim.""" + +import os + +# TODO: Remove once usd-core>=26.5 is the minimum. Earlier releases can corrupt +# the heap while parsing Newton payloads concurrently, so disable USD concurrency +# before importing modules that may initialize OpenUSD. +os.environ["PXR_WORK_THREAD_LIMIT"] = "1" + +import pytest + +import isaaclab_tasks # noqa: F401 + +# Local imports should be imported last +from contrib_env_test_utils import contrib_environment_params, num_envs # isort: skip +from env_test_utils import _run_environments # isort: skip + + +@pytest.mark.parametrize("task_name", contrib_environment_params("kitless")) +def test_contrib_environments_kitless(task_name): + _run_environments(task_name, device="cuda", num_envs=num_envs(task_name)) diff --git a/tools/test_settings.py b/tools/test_settings.py index ca63b8d0aa3b..ca5fdbdd156b 100644 --- a/tools/test_settings.py +++ b/tools/test_settings.py @@ -23,7 +23,9 @@ "test_environments_isaacsim_physx.py": 10000, "test_environments_newton.py": 10000, "test_environments_ovphysx.py": 10000, - "test_contrib_environments.py": 10000, + "test_contrib_environments_kit.py": 10000, + "test_contrib_environments_kit_cameras.py": 10000, + "test_contrib_environments_kitless.py": 10000, "test_environment_determinism.py": 1000, # This test runs through many the environments for 100 steps each "test_multi_agent_environments.py": 800, # This test runs through multi-agent environments for 100 steps each "test_generate_dataset_franka_state.py": 10000, # This test runs annotation for 10 demos and generation for 1 demo @@ -83,6 +85,10 @@ PYTEST_WORKERS = { # 20 independent export round trips, ~18 min serially: the RL job's long pole. "test_leapp_export_flow.py": 4, + # Contributed-environment smoke tests: environment runs of several seconds to 2 min each. The camera file + # stays whole: its workers would each start the RTX renderer, and one environment dominates it. + "test_contrib_environments_kit.py": 2, + "test_contrib_environments_kitless.py": 2, } """Test files split across ``pytest-xdist`` workers, and how many. @@ -113,7 +119,9 @@ CUROBO_TESTS = [ *CUROBO_PLANNER_TESTS, "test_generate_dataset_skillgen.py", - "test_contrib_environments.py", + "test_contrib_environments_kit.py", + "test_contrib_environments_kit_cameras.py", + "test_contrib_environments_kitless.py", ] """A list of tests that require cuRobo installation. From f17e6a1f9650ce771e88a55bad0ad1a439ca7624 Mon Sep 17 00:00:00 2001 From: Mustafa H <34825877+StafaH@users.noreply.github.com> Date: Fri, 25 Sep 2026 15:51:14 -0700 Subject: [PATCH 6/6] Document Haptikos exoskeleton teleoperation with CloudXR (#8046) # Description Supersedes #6029 by @PKonMagos. This carries Haptikostech's Haptikos exoskeleton documentation onto current `develop` and updates the setup instructions for the Isaac Teleop and CloudXR workflow now used by Isaac Lab. The guide now covers the matching Isaac Teleop release branch, the separately obtained Haptikos C++ API required to build the plugin, CloudXR push-device configuration, and the correct launch order. No Isaac Lab runtime code changes are included. ## Type of change - Documentation update ## Release backport - [x] Backport this pull request to the active release branch after it merges into `develop` ## Validation - [x] `uv run isaaclab -f` - [x] `uv run --isolated --extra dev -- make -C docs current-docs` (warning-free) - [x] On-demand Docker CI (`run-ci`): workflow passed; GPU jobs skipped for docs-only changes Hardware-specific Haptikos operation was not exercised locally. --------- Signed-off-by: PKontrazis Co-authored-by: PKontrazis --- docs/source/features/isaac_teleop.rst | 13 ++-- docs/source/how-to/cloudxr_teleoperation.rst | 66 ++++++++++++++++++++ 2 files changed, 75 insertions(+), 4 deletions(-) diff --git a/docs/source/features/isaac_teleop.rst b/docs/source/features/isaac_teleop.rst index 4ffceebe34b5..4ac0f97aeac2 100644 --- a/docs/source/features/isaac_teleop.rst +++ b/docs/source/features/isaac_teleop.rst @@ -51,6 +51,11 @@ input modes, which determine which retargeters and control schemes are available - Isaac Teleop plugin (bundled) - Migrated from the now-deprecated ``isaac-teleop-device-plugins`` repo. Combine with an external wrist-tracking source for wrist positioning. See :ref:`manus-vive-handtracking`. + * - Haptikos Exoskeletons + - Exoskeleton hand tracking with controller wrist poses + - Isaac Teleop plugin (separate executable) + - Requires the Haptikos App, exoskeletons, and an OpenXR headset with controllers. + See :ref:`haptikos-quest-handtracking`. .. _isaac-teleop-control-schemes: @@ -86,7 +91,7 @@ starting point, then see the detailed pipeline examples below. - 28 - ``fixed_base_upper_body_ik_g1_env_cfg.py`` * - Complex dex hand (e.g. GR1T2, G1 Inspire) - - Hand tracking / Manus gloves + - Hand tracking / Manus gloves / Haptikos exoskeletons - Bimanual ``Se3AbsRetargeter`` + ``DexBiManualRetargeter`` - 36+ - ``pickplace_gr1t2_env_cfg.py`` @@ -1769,9 +1774,9 @@ There are two levels of device integration: **Isaac Teleop plugin (C++ level)** For new hardware that requires a custom driver or SDK. Plugins push data via OpenXR tensor - collections. Existing plugins include Manus gloves, OAK-D camera, controller synthetic hands, - and foot pedals. After creating the plugin, update the retargeting pipeline config to consume - data from the new plugin's source node. + collections. Existing plugins include Manus gloves, Haptikos exoskeletons, OAK-D camera, + controller synthetic hands, and foot pedals. After creating the plugin, update the retargeting + pipeline config to consume data from the new plugin's source node. See the `Plugins directory `_ for examples. diff --git a/docs/source/how-to/cloudxr_teleoperation.rst b/docs/source/how-to/cloudxr_teleoperation.rst index 1132b0a0a98f..cde5cf01491b 100644 --- a/docs/source/how-to/cloudxr_teleoperation.rst +++ b/docs/source/how-to/cloudxr_teleoperation.rst @@ -803,6 +803,72 @@ Start teleoperation Move your hands and the simulated follower will mirror the glove-tracked finger joints in real time. +.. _haptikos-quest-handtracking: + +Haptikos Exoskeletons with Quest +-------------------------------- + +The `Haptikos plugin `_ +combines controller wrist poses with exoskeleton finger tracking from the Haptikos Core App and +pushes hand joints into the OpenXR runtime. Isaac Lab receives them through Isaac Teleop's +standard hand-tracking input, so no Haptikos-specific Isaac Lab device is needed. The plugin +supports Linux and has been tested with Meta Quest headsets; other headsets with controllers may +also work. + +Build the plugin +^^^^^^^^^^^^^^^^ + +The Haptikos plugin is built from Isaac Teleop source; it is not included in Isaac Lab's +``teleop`` extra. Check out the release branch matching Isaac Lab's ``isaacteleop`` pin (currently +``1.4.x``), obtain the `Haptikos Robotics API `_, +and copy its ``HaptikosCpp_API_Shared`` directory into ``src/plugins/haptikos``. The C++ API is +required to build the tracking plugin, even if you do not use haptic feedback. Follow the +`plugin's setup instructions `_ +for Haptikos account and licensing requirements. + +.. code-block:: bash + + git clone https://github.com/NVIDIA/IsaacTeleop.git + cd IsaacTeleop + git checkout release/1.4.x + # Copy HaptikosCpp_API_Shared to src/plugins/haptikos before building. + cmake -S . -B build -DENABLE_CLANG_FORMAT_CHECK=OFF + cmake --build build --target haptikos_hands_plugin --parallel 4 + +Run Isaac Lab and the plugin +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Attach a controller to each exoskeleton using the included mount. Calibrate the exoskeleton +forward direction against the headset, then keep the Haptikos Core App, exoskeletons, and +controllers active. Use the hand-tracking task below. + +Haptikos uses an external OpenXR push device. The shipped CloudXR profiles disable push devices, +so enable them in a custom profile before launching Isaac Lab: + +.. code-block:: bash + + cp $(uv run --extra teleop,isaacsim python -c \ + "from isaaclab_teleop import CLOUDXR_JS_ENV; print(CLOUDXR_JS_ENV)") ~/haptikos.env + sed -i 's/NV_CXR_ENABLE_PUSH_DEVICES=0/NV_CXR_ENABLE_PUSH_DEVICES=1/' ~/haptikos.env + + uv run --extra teleop,isaacsim isaaclab teleop run \ + --task IsaacContrib-PickPlace-GR1T2-WaistEnabled-Abs \ + --visualizer kit --xr --cloudxr_env ~/haptikos.env + +Once CloudXR is waiting for a connection, open a separate terminal and start the plugin with +the runtime environment created by Isaac Lab: + +.. code-block:: bash + + cd /path/to/IsaacTeleop + source ~/.cloudxr/run/cloudxr.env + ./build/src/plugins/haptikos/haptikos_hands_plugin + +Connect the Quest using :ref:`the Quest/Pico connection steps `, then start +teleoperation from the headset. + +See :ref:`isaac-teleop-cloudxr-profiles` for more on custom profiles. + Run with Docker ---------------