Skip to content

[USD] Export deformables and Newton solver settings - #7857

Draft
hujc7 wants to merge 69 commits into
isaac-sim:developfrom
hujc7:jichuanh/deformable-solver-usd-export-codex
Draft

hujc7 wants to merge 69 commits into
isaac-sim:developfrom
hujc7:jichuanh/deformable-solver-usd-export-codex

Conversation

@hujc7

@hujc7 hujc7 commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Review Map

  1. The child PR page includes its dependency; “Exact changes” isolates the child's own changes.
  2. SHA links are refreshed after branch updates, starting with this parent PR's copy.
PR Status Depends on Exact changes
#7499 — Environment export Draft -- --
#7857 — Deformables and solver settings Draft #7499 Child changes

Summary

Export registered Newton and PhysX deformables, and optionally preserve Newton XPBD, VBD, Kamino and proxy-coupled solver settings. Depends on #7499.

# After asset initialization, before startup events, reset or stepping:
scene.export_to_usd(
    "environment.usda", env_id=0,
    preserve_source_contacts=True, include_solver_settings=True,
)

Description

  1. Newton deformables preserve source topology, rest geometry, masses, radii, flags and materials. The owner records the existing initialization rest-angle correction and exports that selected array; it no longer dumps computed particle properties or sampled points. Loading uses existing asset readers/builders.
  2. PhysX deformables retain cooked schemas, geometry and source material configuration. Export leaves live USD, backend buffers and normal initialization unchanged.
  3. Solver managers own effective settings and constructor provenance. Optional XPBD, VBD, Kamino and proxy-coupled metadata complements the parent's MJWarp path. Coupling relationships use prim identities and local node indices; generated free-joint identities resolve through their owning bodies. Configured soft-contact settings are retained without dumping unconfigured defaults.

See the contract and loading recipe.

Validation

  1. Experimental USD Exchange 3.0 validation — child Newton suite: 16 passed, including four VBD cloth/volume/flat-cloth and coupled cases. OVPhysX 2 passed; PhysX deformables 2 passed. Final merged shared/actuator/metadata/builder checks: 76 passed, 2 skipped. Parent follow-up also passed Newton 13, OVPhysX 2, and an actual Franka OVPhysX fresh load; these overlap earlier suites, not a new task matrix.
  2. Fresh fixtures compare stable entity identities, topology, rest geometry, mass/material properties, solver settings and coupling relationships. Multiple objects and selected-environment isolation are covered. Flat-cloth redundant-array and required-schema regressions fail before their fixes. Source USD and backend buffers remain unchanged by export.
  3. Eight input/output captures using experimental USD Exchange 3.0 were audited: 632 → 489 property/relationship edits, removing 142 Newton aliases and one flat-cloth roundoff array. Bent/soft rest-angle corrections remain. Two OV armature aliases remain because native loading requires them after an axis API is applied. No new mass/inertia/COM, state, transform or geometry-array opinions were found. Only cable expands instances and creates one shared material for its configured friction override.
  4. The PR retains USD Exchange 2.3.0, its lockfile and parsing workarounds. After merging the parent decorator correction, local CPU shared/actuator/metadata/builder checks passed 88 with 2 CUDA skips; parent checks passed 82 with 2 CUDA skips. Formatting and fragment checks passed against the parent branch. Configuration bindings now live on the existing data-property decorators; initialization discovers them and records successful fixed overrides. Copied dependencies use an unoccupied resource namespace. Both regressions fail with their previous implementations. Earlier native backend, importer and strict Sphinx results above belong to the testing-only 3.0 experiment. No new full task sweep or support-matrix refresh.

Boundaries

  1. The snapshot remains after fixed initialization and before startup/reset/stepping. Arbitrary subsequent setter/buffer changes are untracked. Sampled state is not written; existing source velocities are neither sampled nor cleared.
  2. Controllers, policy observations, sensor execution and transient solver state require deployment integration. External resources are not packaged: ANYmal retains a temporary native neural-actuator checkpoint reference from pre-startup setup, which deployment on another host must carry alongside textures/MDL dependencies.
  3. ADMM coupling, arbitrary custom solver factories, surface grippers and unregistered deformables are unsupported.
  4. Newton deformable supplements require the documented owner reader; native USD import alone does not read every field. Cross-backend deployment and arbitrary heterogeneous layouts require separate validation.

Type of change

New feature and documentation update.

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Screenshots

Not applicable: USD authoring and backend validation.

Checklist

  • Read the contribution guidelines; contributor already listed.
  • Added changelog fragments for each changed source package.
  • Added fresh-backend regression tests and updated public documentation.
  • Ran formatting/lint checks and a strict Sphinx build.
  • No warnings: existing dependency and native importer warnings remain.

hujc7 added 30 commits July 29, 2026 13:34
Isaac Lab applies most configuration directly to the solver rather than
to the stage, so a scene that has been loaded and overridden no longer
has a USD file describing what is actually being simulated.

Add export_model_to_usd(), the inverse of ModelBuilder.add_usd(). Core
physics is authored with standard UsdPhysics schemas and Newton-specific
properties as newton:* attributes, at the prim paths the model was
imported from, so a reimport reproduces the same model.

The correctness contract is model idempotence rather than USD fidelity:
the importer normalizes as it reads, so the exported stage differs from
the source by construction, but reimporting it must change nothing.

Known gaps are documented in the module and fail loudly rather than
exporting something wrong: D6/ball/distance joints, planes, heightfields
and SDF shapes, cloned multi-world scenes, and shape ordering, which
follows USD stage traversal.
Comparing model arrays shows the numbers match but not that the export
reproduces the same physics, and it reports spurious differences for
rebuilt acceleration structures and for shape orderings that differ
without changing behavior.

Add a rollout check that steps the source and exported models under
identical initial conditions and compares body trajectories, treating a
quaternion and its negation as the same rotation. The fixture's joint
carries an offset frame so the trajectory is sensitive to joint
geometry; without it the check cannot distinguish a correct export from
one that drops joint frames.

The two layers are complementary. The rollout misses parameter drops
that do not change this particular motion, such as mass in a pendulum
under gravity, which the array comparison catches.
Every existing check starts from an unmodified load, so an exporter that
re-derived its output from the source USD would pass all of them while
reproducing the asset rather than what is being simulated. That is the
property the feature exists for: Isaac Lab applies most configuration by
writing into the model after the stage is parsed.

Add a check that writes distinctive values into the model, as the asset
classes do, and asserts the export carries them rather than the source
file's values.

Also document that free-joint degrees of freedom carry no exportable
state: a floating body is expressed in USD by the absence of a joint, so
per-DOF values written to a free joint have nowhere to be authored.
Isaac Lab's cloner builds one Newton world per environment, all sharing the
source asset's prim paths. The exporter resolved indices as if the model held
a single world, so a cloned scene exported only the first environment's
entities under every environment's prim paths and reported success: a
3-world model of 6 bodies came back as 2.

Select the world explicitly. Entities are chosen by their world membership
plus the worldless global content, which is the same rule for an uncloned
model, and the count of exportable entities is checked against the source's
prim paths so a shortfall fails instead of shipping a partial file. Free
joints and sites are exempt: a floating body is expressed in USD by the
absence of a joint, and a site is a bare frame with no geometry.

Every task's ground is a plane, which raised NotImplementedError; author it
as a Z-axis UsdGeom.Plane, the form the importer reads back. Physics
materials are authored for every shape, since Isaac Lab randomizes friction
per shape regardless of whether it collides.

Also export the entry point from isaaclab_newton.sim; the lazy-export stub
had omitted it, so the documented import path raised AttributeError.
export_model_to_usd() needs the path maps that ModelBuilder.add_usd()
returns, and every build path discarded them: the clone-plan session dropped
the return of ctx.replicate(), and the per-prototype import result never
left _build_source_builder. The exporter could not be called on a real
environment at all.

Keep the results and merge them into one environment's provenance. Global
content is indexed against the model directly; each prototype's indices are
lifted by the landing offset replicate_builder_mapping already computes
before appending it, so a world with several prototypes (robot, object,
table) resolves exactly. NewtonManager.get_stage_info() exposes the result
in the shape add_usd() returns, so it feeds the exporter unchanged.
PhysX and OVPhysX keep the stage authoritative for structure, but every
runtime write (drive gains, masses, armature, friction, limits) goes to the
solver's buffers and never reaches a prim. Saving the stage of a running
scene therefore emits a file that looks complete while carrying the
spawn-time value of everything overridden since.

Author those properties back onto the prims they came from, reading through
BaseArticulationData so the same code serves any backend. Recovering prim
paths is the one backend-specific step, so each backend supplies them
through ArticulationPrimPaths. Values are joined to prims by name, because
the view is in backend order and the data in public order.

Angular drive gains are per degree on the stage and per radian in the
simulation; converting on export matters for every driven revolute joint,
and a task whose only revolute joint is passive cannot reveal it.
The PhysX tensor view records the prim each link and degree of freedom was
built from, so the paths are read straight off it and handed to the shared
exporter.
An OVPhysX binding reports names and the articulation prims it matched, not
per-link paths, so they are resolved by indexing the articulation subtree by
prim name. A body or joint with no prim fails the export rather than leaving
it partial.
Both PhysX and OVPhysX populate the joint friction the solver applies from
PhysxJointAxisAPI (static and dynamic friction effort, viscous coefficient),
not from the legacy physxJoint:jointFriction scalar the exporter wrote, so
randomized friction reimported as zero. The per-axis schema also shadows the
joint-level armature on OVPhysX, and its viscous coefficient is per degree
per second on angular axes: the exporter now writes the triple under the
drive axis, armature in both places, and viscous scaled like drive gains.

Applying a schema to a prim that is itself an articulation root invalidates
every PhysX articulation view on the stage for the rest of the session.
export_articulation_to_usd therefore authors onto a flattened snapshot;
write_articulation_state_to_stage takes the target stage explicitly and
applies schemas only where absent. Reads precede writes, prim paths on
OVPhysX resolve from a typed index that walks up from the root prim, and
each backend wrapper forwards the stage argument.
ModelBuilder.approximate_meshes(keep_visual_shapes=True), which the USD
importer runs on visible meshes with a collision approximation, copies the
mesh as a visual-only shape labelled <label>_visual that has no prim of its
own, so the coverage guard refused every scene with such an asset. The
resolution of a world's entities to prim paths moves into a public
resolve_world_prim_paths returning WorldPrimPaths; twins resolve to a
visual-only <prim>_visual sibling of their source and are authored as such.
Coverage is checked entity by entity rather than by count, unsupported
geometry is rejected by name, and joints outside the articulation are
exported with physics:excludeFromArticulation.
Retaining import provenance made replicate_builder_mapping return the
world-0 landing offsets as a third element against its declared two-tuple,
which broke four develop cloner tests and every caller that unpacks two
values. The offsets are recorded only on request by a private
implementation; replicate_builder_mapping keeps its contract and the two
provenance callers use replicate_builder_mapping_with_provenance.
…usd-export

# Conflicts:
#	source/isaaclab_newton/isaaclab_newton/cloner/newton_clone_utils.py
#	source/isaaclab_newton/isaaclab_newton/cloner/replicate.py
#	source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py
#	source/isaaclab_newton/isaaclab_newton/physics/visualization_builder.py
Newton 1.6 keeps shape_color in sRGB and reads primvars:displayColor as
linear, converting on import, so writing the model colour straight into the
attribute re-applied the transfer curve on every round-trip and the export
stopped being a fixed point. The exporter now authors the inverse. The test
loader imports the way Isaac Lab does, aligning builder colours with the
stage after add_usd, which is where the exported colour is read back.
Preserve authored scene content and declare initialized buffer mappings through a shared writer and thin source adapters. Export the actual task before events in an isolated rank-zero training worker, and preserve distinct Newton collider material bindings.

Remove the PR-only runtime snapshot, model reconstruction, and clone provenance paths. Validate complete fresh-backend loads, failure atomicity, and unchanged seeded training behavior.
Expose InteractiveScene.export_to_usd and let existing articulation, rigid object, and collection classes supplement one isolated stage through shared writers. Supply source adapters through backend managers.

Remove SceneExporter, construction-only export APIs, and intermediate body records. Keep task capture and constructor interruption in the isolated training worker, and exercise the public scene entry point in fresh-backend tests.
@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels Sep 16, 2026
@hujc7
hujc7 force-pushed the jichuanh/deformable-solver-usd-export-codex branch from 865335b to 19a209a Compare September 16, 2026 18:57
…lver-usd-export-codex

# Conflicts:
#	source/isaaclab/isaaclab/scene/interactive_scene.py
#	source/isaaclab/isaaclab/sim/usd_export.py
#	source/isaaclab_newton/test/sim/test_usd_export.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation infrastructure isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant