Skip to content

[Refactor] Back OperationalSpaceController with Newton's operational-space controller - #7482

Closed
hujc7 wants to merge 5 commits into
isaac-sim:developfrom
hujc7:jichuanh/newton-operational-space-controller
Closed

hujc7 wants to merge 5 commits into
isaac-sim:developfrom
hujc7:jichuanh/newton-operational-space-controller

Conversation

@hujc7

@hujc7 hujc7 commented Sep 2, 2026 •

Copy link
Copy Markdown
Collaborator

Superseded by #7854, which consolidates the three supported controllers on the released Newton API. Ackermann remains a separate draft in #6654.

Summary

OperationalSpaceController now evaluates its control laws through Newton instead of an in-tree
Torch implementation: 2.6-2.8x faster per step (0.32 ms vs 0.88 ms at 4096 envs) with the
public configuration, command and output contracts unchanged.

Important

Blocked — do not merge or run CI yet. This depends on
newton-physics/newton#4105 (unmerged), plus
three further Newton changes described in §3 that are not yet proposed upstream. Against
develop's pinned newton[sim]==1.5.1, compute() raises
ImportError: cannot import name 'ControllerOperationalSpaceModelFree'. Opened to review the
Isaac Lab side of the change only.

Description

Fixes # (issue)

The task-space impedance law, the contact-wrench law and null-space control move to
newton.controllers.ControllerOperationalSpaceModelFree. Target resolution, the task frame and the
gain schedule stay in Isaac Lab, mirroring the JointImpedanceController port.

1. Frame handling

The task frame is handed to Newton as the operational frame, so gains, selection axes and targets
stay expressed in it instead of being rotated into the root frame first. That retires the
root-frame gain buffers, both selection matrices, the operational-space mass matrix and the cached
joint-space mass-matrix inverse, and lets the per-axis gains be stored as (num_envs, 6) vectors
rather than diagonal matrices. _motion_p_gains_task / _motion_d_gains_task change shape
accordingly; the one in-tree consumer (ExportPatcher) is updated.

2. Behavior

No intended behavior change. Verified two ways:

  • Parity oracle — test_operational_space_newton_integration.py reimplements the previous Torch
    law and compares against it across 14 configurations (absolute/relative pose, open- and
    closed-loop wrench, all three decoupling modes, null-space centering, both variable impedance
    modes, with and without a task frame). 14/14 on cpu and on cuda:0.
  • Existing sim suite — test_operational_space.py is 18/18, matching the pre-change baseline
    measured in the same container.

3. Required Newton changes

Prototyped locally and measured; not yet proposed upstream. Without them the sim suite is 12/18.

# Change Why
1 Adaptive Tikhonov ridge in _invert_spd_block_kernel It floors the Cholesky pivot at _FLOAT32_EPS * max(|diag|, 1) then divides by that floor, so a near-singular J M⁻¹ Jᵀ inverts to order 1/eps. At cond = 2.4e8 the torque error is 349%, which failed 5 tests. Damping only on a second pass, and only when the first hits a degenerate pivot, keeps well-conditioned blocks bit-identical.
2 Gate the >=6 controlled-DOF guard on that ridge Isaac Lab has always accepted inertial decoupling on under-actuated arms; Newton rejects it at construction. The ridge makes the rank-deficient case well-posed.
3 mask_motion_after_inertia flag Newton applies Khatib's Ω to the commanded acceleration, ahead of Λ; this controller applies it to the resulting force, after Λ. The two differ only when a motion axis is de-selected and decoupling is on, but there the pre-Λ ordering misses the shipped contact-force thresholds (5.8 N and 7.4 N against a 5.0 N band).

4. Known contract narrowing

  • compute() returns a view of Newton's output port rather than a freshly allocated tensor. Both
    in-tree callers copy or consume it immediately; external code holding the reference across steps
    would now see it mutate.
  • Inputs must be float32; the previous Torch law accepted float64.
  • The backend is built on the first compute() call, since num_DoF comes from the Jacobian, so
    configuration errors surface at the first step rather than at construction.

Type of change

  • Breaking change (existing functionality will not work without user modification)

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

…oller

Evaluate the task-space impedance, contact-wrench and null-space laws through
newton.controllers.ControllerOperationalSpaceModelFree instead of the in-tree
Torch implementation, keeping target resolution, the task frame and the gain
schedule in Isaac Lab so the public configuration, command and output contracts
are unchanged. This mirrors the JointImpedanceController port.

The task frame is handed to Newton as the operational frame, so gains, selection
axes and targets stay expressed in it rather than being rotated into the root
frame first. That removes the root-frame gain, selection-matrix and
operational-space mass-matrix buffers, and lets the per-axis gains be stored as
vectors instead of diagonal matrices.

Measured 2.6-2.8x faster per step than the previous Torch law (0.32 ms vs
0.88 ms at 4096 envs, 7 DoF, on one RTX PRO 6000).
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Sep 2, 2026
Follow the JointImpedanceController port's rule: a buffer this controller owns
and updates in place binds to its Newton input port once, and only a value
composed from several sources is written through a port view. The motion gains,
the task frame, the desired pose and wrench, the tool state and the null-space
target now bind directly, leaving the measured contact wrench as the single
per-step write.

Warp wrappers around caller tensors are cached per port and rebuilt only when the
caller hands over a different tensor, so a caller refilling a persistent buffer
pays no wrapper construction. Non-contiguous tensors are never cached, since the
wrapper would alias the throwaway copy rather than the caller's storage.

Per-step tensor copies drop from ten to two, and the step cost from 0.32 ms to
0.25 ms at 4096 envs (3.5x the previous Torch law, up from 2.8x).
Cache the damping ratio at construction instead of rebuilding it from the config
tuple on every set_command, write the resolved targets into persistent buffers
rather than allocating a fresh tensor per command, and refresh the measured
wrench's moment half per command instead of per step. The public target
attributes still read None until commanded, then name their buffer, so the
sentinel is unchanged and the Warp binding cache keys stay stable.

Zeroing the measured wrench on the uncommanded path must not be skipped: with
closed-loop force control a stale moment left behind by a reset is read as
Kp * (0 - stale) and drives torque from a command that no longer exists. The
reset test now steps once while commanded before resetting, so it exercises that
state; without the fix it fails by 3.21.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant