[Newton] Add Ackermann steering controller and action term

[hujc7]

Summary

Keep Ackermann steering integration in a separate draft pending a public Newton Ackermann controller.

Description

This draft contains only the Ackermann controller, configuration, action term, documentation, and tests from the original proposal. Differential IK moved to the consolidated Newton controller PR #7854.

Blocked: the adapter still depends on the unpublished newton_controllers.ControllerAckermann API. Released Newton does not provide it; this draft is not ready to merge.

Type of change

• New feature

Release backport

• Backport to the active release branch

Screenshots

Not applicable.

Validation

Formatting and changelog checks passed. Runtime tests were not rerun because the required unpublished controller dependency is unavailable in the supported environment.

Checklist

• Documentation and package changelog fragment included
• Existing Ackermann tests retained
• Pre-commit checks passed with uv run isaaclab -f
• Public Newton implementation available and runtime tests passing

[greptile-apps]

Greptile Summary

This PR integrates Newton's controller library (newton_controllers) into Isaac Lab by replacing the pure-PyTorch DifferentialIKController solve path with Newton's ControllerDifferentialKinematicsModelFree, and by adding a new AckermannController/AckermannAction pair for Ackermann-steered vehicles. It also fixes the SO-101 pose-IK action to use class_type dispatch instead of a hardwired controller swap, and enables MuJoCo-native gravity compensation for Franka reach tasks under the newton_mjwarp preset.

• DifferentialIKController now lazily initializes a Newton/Warp solver bridge on the first compute() call, preserving the public API (shape, dtype, out parameter) while adding input validation and a float32 internal buffer; the prior pure-Torch implementation is removed.
• AckermannController / AckermannAction are new additions that delegate batched Ackermann kinematics to Newton's ControllerAckermann, with vehicle-geometry validation in the config, direction-multiplier support, and a non-finite command safety policy.
• SO-101 and Franka reach fixes are small, targeted corrections enabled by the class_type dispatch refactor in DifferentialInverseKinematicsAction.

Confidence Score: 3/5

The Ackermann additions and the SO-101/Franka fixes are ready to merge. The DifferentialIKController rewrite introduces an undeclared hard dependency: existing users of that controller who haven't installed newton_controllers will encounter an ImportError the moment they access the class, with no helpful diagnostic.

The DifferentialIKController is one of the most commonly used controllers in Isaac Lab. Replacing its implementation with one that hard-imports newton_controllers without a try/except will break any environment or downstream package that imports DifferentialIKController without the new dependency present. The missing .view() reshape for _joint_pos_des also couples correctness silently to Newton's internal storage choices.

source/isaaclab/isaaclab/controllers/differential_ik.py — the hard newton_controllers import and the unguarded _joint_pos_des shape assumption both live here.

Important Files Changed

Filename	Overview
source/isaaclab/isaaclab/controllers/differential_ik.py	Core change: rewrites DifferentialIKController to delegate all IK solves to Newton's ControllerDifferentialKinematicsModelFree via a lazily-initialized Warp bridge. Introduces hard top-level import of newton_controllers (P1), missing .view() reshape for _joint_pos_des (P2), and new input validation stricter than the historical contract.
source/isaaclab/isaaclab/controllers/ackermann.py	New AckermannController wrapping Newton's ControllerAckermann with Torch/Warp zero-copy bridge. Logic is sound; reset() uses unconventional tensor[None]=0.0 idiom (P2) but is functionally correct.
source/isaaclab/isaaclab/controllers/ackermann_cfg.py	New AckermannControllerCfg with thorough post_init validation of vehicle geometry and command limits. Float32 bridge edge case for max_steering_angle near pi/2 is handled correctly.
source/isaaclab/isaaclab/envs/mdp/actions/ackermann_actions.py	New AckermannAction MDP term. Joint validation, direction multipliers, scale/offset/clip processing, and non-finite command zeroing are all implemented correctly. reset() shares the tensor[None] idiom (P2).
source/isaaclab/isaaclab/envs/mdp/actions/task_space_actions.py	DifferentialInverseKinematicsAction updated to instantiate the IK controller via class_type, detect the out parameter, and use a pre-allocated _joint_pos_des buffer. Jacobian aliasing fix is correct and well-commented.
source/isaaclab_tasks/isaaclab_tasks/contrib/stack/config/so101/pose_ik_action_term.py	Removes the manual controller swap; now relies on class_type dispatch in the base DifferentialInverseKinematicsAction. Fix is correct and matches the test assertion in test_so101_pose_ik.py.
source/isaaclab_tasks/isaaclab_tasks/contrib/stack/config/so101/pose_ik_controller.py	Adds class_type field to SO101PoseIKControllerCfg so the base action term can instantiate SO101PoseIKController directly through class_type dispatch.
source/isaaclab_tasks/isaaclab_tasks/contrib/reach/config/franka/robot_cfg.py	New task-owned robot config that enables MuJoCo actuatorgravcomp under the newton_mjwarp preset via the preset() helper. Clean separation from PhysX default (None).

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant ActionTerm as DifferentialInverseKinematicsAction
    participant IKCtrl as DifferentialIKController
    participant Newton as Newton ControllerDifferentialKinematicsModelFree
    participant Warp as Warp bridge buffers

    ActionTerm->>IKCtrl: set_command(command, ee_pos, ee_quat)
    IKCtrl->>IKCtrl: validate + store ee_pos_des / ee_quat_des

    ActionTerm->>IKCtrl: compute(ee_pos, ee_quat, jacobian, joint_pos, out)
    IKCtrl->>IKCtrl: validate inputs and cast to float32
    IKCtrl->>IKCtrl: assemble task_jacobian and task_error
    IKCtrl->>IKCtrl: _initialize_controller on first call
    IKCtrl->>Warp: copy task_error, task_jacobian, joint_pos into bridge buffers
    IKCtrl->>Newton: compute(input, output, None, None, time_step)
    Newton-->>Warp: writes joint_target_q
    Warp-->>IKCtrl: _joint_pos_des zero-copy view
    IKCtrl-->>ActionTerm: return joint_pos_des snapshot or out buffer

    ActionTerm->>ActionTerm: set_joint_position_target_index

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant ActionTerm as DifferentialInverseKinematicsAction
    participant IKCtrl as DifferentialIKController
    participant Newton as Newton ControllerDifferentialKinematicsModelFree
    participant Warp as Warp bridge buffers

    ActionTerm->>IKCtrl: set_command(command, ee_pos, ee_quat)
    IKCtrl->>IKCtrl: validate + store ee_pos_des / ee_quat_des

    ActionTerm->>IKCtrl: compute(ee_pos, ee_quat, jacobian, joint_pos, out)
    IKCtrl->>IKCtrl: validate inputs and cast to float32
    IKCtrl->>IKCtrl: assemble task_jacobian and task_error
    IKCtrl->>IKCtrl: _initialize_controller on first call
    IKCtrl->>Warp: copy task_error, task_jacobian, joint_pos into bridge buffers
    IKCtrl->>Newton: compute(input, output, None, None, time_step)
    Newton-->>Warp: writes joint_target_q
    Warp-->>IKCtrl: _joint_pos_des zero-copy view
    IKCtrl-->>ActionTerm: return joint_pos_des snapshot or out buffer

    ActionTerm->>ActionTerm: set_joint_position_target_index

Loading

Comments Outside Diff (1)

1. source/isaaclab/isaaclab/controllers/ackermann.py, line 218-226 (link)

   tensor[None] = 0.0 is a correct but non-obvious reset idiom

   When env_ids=None, each of self._command[None] = 0.0, self._linear_speed_input[None] = 0.0, and self._steering_angle_input[None] = 0.0 does zero all elements — None creates an unsqueeze view and the assignment broadcasts back to the original storage. However, this is a NumPy/PyTorch trick that is easy to misread as a no-op. The rest of the Isaac Lab codebase typically handles this via tensor[:] = 0.0, tensor.zero_(), or an explicit if env_ids is None: env_ids = slice(None) guard. The same idiom appears in AckermannAction.reset().

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "Integrate Newton controllers for DiffIK ..." | Re-trigger Greptile}

[greptile-apps]

Hard import of newton_controllers at module level

from newton_controllers import ControllerDifferentialKinematicsModelFree is a top-level unconditional import. While isaaclab.controllers.__init__.py uses lazy_export() to defer submodule loading, that only delays the import until DifferentialIKController is first accessed — it doesn't skip it. Any existing Isaac Lab user who accesses DifferentialIKController without newton_controllers installed will immediately receive an unhelpful ModuleNotFoundError: No module named 'newton_controllers' instead of a diagnostic that explains what is missing and why.

A try/except at import time with an explanatory ImportError (re-raised from _initialize_controller) would make the dependency explicit and actionable, without requiring Newton for users who import the module at all.

[greptile-apps]

Missing explicit .view(num_envs, num_joints) reshape for _joint_pos_des

The Ackermann controller calls .view(num_envs, 2) and .view(num_envs, self._num_wheels) after wp.to_torch because Newton's Ackermann output arrays are 1-D flat buffers. Here, no reshape is applied to joint_target_q, which means correctness relies on Newton's ControllerDifferentialKinematicsModelFree storing its output as a 2-D (num_envs, num_joints) array. If Newton ever normalises its storage to match the Ackermann convention, out.copy_(self._joint_pos_des) will silently fail with a shape mismatch at runtime. Adding .view(num_envs, num_joints) after the wp.to_torch call here would make the expected shape explicit and defensive against future Newton changes.