Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/source/api/lab/isaaclab.benchmark.rst
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,10 @@ Workflow Functions

.. autofunction:: run_runtime_benchmark

.. autofunction:: isaaclab.benchmark.stepping.profile_physics_steps

.. autofunction:: isaaclab.benchmark.stepping.profile_renderers

.. autofunction:: run_startup_benchmark

.. autofunction:: run_training_benchmark
Expand Down
9 changes: 9 additions & 0 deletions docs/source/developer-tools/benchmarking/benchmark_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,15 @@ The command prints the summary report. The paths in ``result.output_paths``
identify the schema and summary JSON files that were written. Use these paths in
automation instead of reconstructing the timestamped names.

With a non-``None`` task ``benchmark_mode`` and ``ISAACLAB_RENDER_PROFILE=1`` or
``ISAACLAB_PHYSICS_PROFILE=1``, the runtime workflow also includes scalar profiling
summaries in ``result.bundle.extra``: ``physics_mean_ms``, ``physics_std_ms``,
``physics_max_ms``, ``physics_calls``, and the corresponding ``render_*`` keys.
These report mean, standard deviation, maximum time per call [ms], and call count
over the captured samples. Disabled scopes contribute no keys. The summaries are
included in schema and OmniPerf output without changing schema version 1.4.
Raw ordered samples remain in ``<output_path>/profile_timings.json`` for local analysis.

Choose a request
----------------

Expand Down
28 changes: 28 additions & 0 deletions docs/source/developer-tools/benchmarking/run_benchmarks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,34 @@ environment-step rate. Runtime samples random actions before starting the
runtime run, ``runtime.collection_fps`` and ``runtime.total_fps`` describe the
same random-action stepping workload.

Render and physics scope profiling requires a non-``None`` ``benchmark_mode`` in the
task configuration. If the field is absent or ``None``, both scopes remain disabled
even when their profiling flags are set; standard runtime reports are still produced.

For ``Isaac-RenderBenchmark-Franka-Cabinet``, ``BENCHMARK_MODE=render`` (the default)
writes analytic joint poses after physics, while ``BENCHMARK_MODE=physics_render``
sets actuator targets before physics. Both modes still step physics. Direct posing
requires ``scene.lazy_sensor_update=True``. With Isaac RTX, use ``--visualizer none``
in this mode: a Kit visualizer would render before the pose write.

Set ``ISAACLAB_PHYSICS_PROFILE=1`` to collect synchronized physics-step timings during
the runtime measurement loop. The benchmark wraps the selected physics
manager's ``step`` through :func:`~isaaclab.benchmark.stepping.profile_physics_steps`
during the measurement loop. Similarly, ``ISAACLAB_RENDER_PROFILE=1`` wraps registered
renderers through :func:`~isaaclab.benchmark.stepping.profile_renderers`, timing only
``render()`` and excluding scene updates and output readback. Both context managers install
wrappers after warmup and restore the original methods when measurement ends, including on
failure, so subsequent benchmark runs are unaffected. The render sweep
enables both flags automatically. Ordered ``[scope, elapsed_ms]`` samples are written under
``timings_ms`` in the local ``<output_path>/profile_timings.json`` file. The benchmark bundle's
``extra`` dictionary holds scalar ``physics_mean_ms``, ``physics_std_ms``, ``physics_max_ms``,
``physics_calls``, and corresponding ``render_*`` summaries. Disabled scopes contribute no keys.
Schema version 1.4 remains unchanged, and OmniPerf output includes the same summaries.
These statistics describe individual calls after warmup; the render sweep instead groups physics
calls by rendered frame and discards its padding frames. The sweep reads the local profiling file
from a separate directory for each profile. Device synchronization changes execution overlap,
so these profiled runs are diagnostics rather than throughput measurements.

.. dropdown:: Canonical workstation output and provenance

Headless runtime summary (output abbreviated):
Expand Down
Loading
Loading