[Newton] Consolidate differential IK, joint impedance, and OSC controllers

[hujc7]

Description

Added Newton 1.6 model-free differential IK, joint impedance, and OSC through cfg.use_newton=True; the original Lab controllers remain the default. Consolidates #6693, #7482, and the differential-IK portion of #6654. Ackermann remains in its separate draft.

Both paths share constructor, command, and compute signatures and return independent results. Controller selection is independent of the physics backend. Newton initializes from compute inputs, or from supplied DiffIK limits when avoidance is enabled. Existing callers and tutorials retain their setup; SO101 retains its wrist-orientation mask through a Newton-only Jacobian adaptation.

Newton uses float32 computation. Its OSC selects motion before inertia decoupling, requires at least six joints for decoupling, and retains motion-force coupling that fails three hybrid force-tracking cases. Those Newton cases run as strict expected failures; Lab cases remain active, and thresholds are unchanged. Convergence checks now apply masks in the actual control frame.

Newton DiffIK requires joint limits before the first compute when avoidance is enabled; later limit updates retain its buffers. Its pinv also rejects fewer controlled joints than active task axes. These differences prevent blanket replacement of the Lab path. Joint impedance retains the same control law, subject to Newton's precision boundary. Includes the position-only SVD and joint-impedance gain-clamping/batched-inertia fixes.

Type of change

• New feature, bug fix, documentation update

Release backport

• Backport to the active release branch

Validation

• Post-merge CPU controller suite: 204 passed, 3 CUDA capture tests skipped. Formatting and changelog checks passed.
• Focused GPU hybrid simulation with current controller/test overlays, 16 environments: 3 Lab passed, 3 Newton xfailed. This was not a full-repository run of the final head.
• Warning-free current documentation build passed.
• Matched full-policy Lab/Newton videos were recorded for Reach-Franka-OSC on PhysX/MJWarp and Drawer DiffIK on MJWarp. SO101 keyboard rollout and training equivalence remain unverified.
• Fresh Docker/GPU CI requested for the updated head; results pending.

Checklist

• Contribution guidelines followed; contributor already listed
• Pre-commit checks run with uv run isaaclab -f
• Documentation and package changelog fragments updated
• Both-backend and regression tests added
• Newton hybrid force-tracking limitations resolved
• Changes generate no new warnings

[greptile-apps]

Confidence Score: 4/5

The PR is not safe to merge while three retained OSC simulation scenarios regress relative to the base implementation.

Findings

1.  OSC Simulation Paths Regress ▶
2.  SO-101 Rollout Is Unverified ▶

Summary

This PR replaces the differential-IK, joint-impedance, and operational-space controller implementations with Newton 1.6.0 model-free adapters, introduces persistent Torch/Warp port storage, requires fixed joint counts for DiffIK and OSC, and updates action terms, SO-101 specialization, documentation, and tests.

• DiffIK now delegates solver methods and optional joint-limit avoidance to Newton.
• Joint impedance delegates its control law while retaining Isaac Lab command shaping and gain scheduling.
• OSC delegates motion, wrench, inertia, gravity, and null-space control using operational-frame ports.
• The OSC migration still has three acknowledged simulation regressions that pass on the base implementation.
• SO-101’s Newton-backed wrist-mask path lacks simulation-level rollout validation.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Policy or standalone caller] --> B[Isaac Lab command shaping]
  B --> C{Controller}
  C --> D[Differential IK adapter]
  C --> E[Joint impedance adapter]
  C --> F[Operational-space adapter]
  D --> G[Persistent Torch/Warp input ports]
  E --> G
  F --> G
  G --> H[Newton 1.6 model-free step]
  H --> I[Joint position or effort snapshot]
  F --> J[Hybrid and task-frame simulation paths]
  J --> K[Three retained tests currently fail]

Loading

_{Reviews (1) · Last reviewed commit: "Merge remote-tracking branch 'upstream/d..."}

[greptile-apps]

OSC Simulation Paths Regress

The new pre-inertia motion-axis selection is not compatible with the retained OSC configurations. test_franka_hybrid_variable_kp_impedance, test_franka_taskframe_hybrid, and test_franka_taskframe_hybrid_with_nullspace_centering pass with the original controller but fail with this adapter. These hybrid and task-frame paths therefore produce incorrect tracking or force-control behavior until the Newton mapping or affected configurations are corrected.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[hujc7]

The force-tracking limitation remains; it has not been fixed by the adapter cleanup. Lab is now the default and Newton is explicitly selected with cfg.use_newton=True. The three Newton hybrid cases still execute as strict xfails, with unchanged convergence thresholds and active Lab counterparts. The task-frame convergence metric was also corrected. This keeps the unsupported Newton behavior visible while preserving the default Lab path; the PR description documents the limitation.

[hujc7]

run-ci

[isaaclab-review-bot]

Isaac Lab Review Bot

The Newton-backed consolidation introduces three concrete integration regressions: DiffIK action construction calls a lazy string as a class, the UR10 tutorial computes len() on a resolved slice, and OSC no longer supports the existing runtime gravity-compensation toggle used by backend validation.

• Design and architecture: The split between Isaac Lab command/configuration handling and Newton solver execution is coherent, but construction-time backend feature binding changes the mutability contract of OSC configuration. In particular, gravity compensation is now fixed when ports are created even though existing code changes the flag between control phases.
• API: The new cfg.num_joints and optional compute(out=...) paths are broadly propagated, including compatibility handling for custom four-argument controllers. However, controller factory support is currently broken because cfg.controller.class_type may remain a lazy import string and is invoked directly instead of being resolved before construction.
• Implementation: Stable Torch/Warp buffers and output ownership are handled consistently, but two concrete paths need correction: derive the tutorial joint count without calling len() on slice(None), and either rebuild OSC when gravity compensation changes or explicitly enforce construction-time immutability while updating the existing consumer. The DiffIK action must also resolve string-valued controller factories before invocation.

Significant concerns. Posted 3 actionable findings inline.

Automated review; human maintainers own approval decisions.

[hujc7]

run-ci

[kellyguo11]

if the breaking changes are unavoidable, I would recommend keeping the current controller path and marking it as deprecated. we can switch the legacy vs. newton path through a cfg setting, similar to the actuators.

it would also be good to validate that the behavior of the legacy and newton controllers are equivalent.

[kellyguo11]

what is the impact of this?

[hujc7]

For local testing, there's no impact atm. The failing tests are all on OSC

[kellyguo11]

why do we have this breaking change? does this mean newton OSC will be different in behavior?

[hujc7]

Newton OSC controller does have different behavior. Will confirm with Jeff.

[hujc7]

run-ci

[hujc7]

run-ci