Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -169,13 +169,14 @@ jobs:
- 'ruff.toml'
- 'PSScriptAnalyzerSettings.psd1'
- '.github/workflows/**'
# Sphinx docs site: the doc sources themselves, plus README.md and
# CONTRIBUTING.md, which several pages single-source via MyST
# `{include}` directives, plus the build config/toolchain pins.
# Sphinx docs site: the doc sources themselves, plus README.md,
# CONTRIBUTING.md and docs/vllm.md, which pages single-source via
# MyST `{include}` directives, plus the build config/toolchain pins.
docs:
- 'docs/rocm-docs/**'
- 'README.md'
- 'CONTRIBUTING.md'
- 'docs/vllm.md'
- '.readthedocs.yaml'
- '.github/workflows/**'

Expand Down
134 changes: 82 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ rocm serve qwen
```

`qwen` is a built-in alias for a small assistant model that serves out of the
box. You can also serve any compatible Hugging Face model directly — see
box. You can also serve any compatible Hugging Face model directly. See
[Model serving](#model-serving) for the GGUF-vs-safetensors rule, since which
form works depends on the engine your GPU selects.

Expand Down Expand Up @@ -311,54 +311,84 @@ rocm update [--apply] [--runtime KEY] [--activate] [--dry-run]
```

`install sdk` downloads TheRock ROCm wheels into a Python environment managed
by rocm-cli; pass `--devel` to also install the compiler and headers needed to
build GPU code, roughly doubling the download. `--devel` is not an addition to
an existing runtime: a runtime is identified by the packages it was installed
from, so running `rocm install sdk` and later `rocm install sdk --devel` at the
same version leaves you with **two** side-by-side runtimes — the second is a
fresh full install, not a toolchain bolted onto the first — and the second one
becomes active. `rocm runtimes list` marks each one `toolchain=included` or
`toolchain=excluded`, `rocm examine` reports the active runtime's as
`active_runtime_toolchain`, and `rocm storage remove-old-installs` counts the
two kinds separately so neither evicts the other. To reclaim the space, uninstall
the one you do not want with `rocm runtimes uninstall <runtime-key>`.
An install with no active default runtime never prompts, but once a
managed runtime is the active default every `install sdk` asks first, because
the new install takes over as the active default. That gate is not scoped to the
family or channel you are installing: a `--family` or `--channel` you have never
installed before takes over the active default just as a same-family upgrade
does, so it asks too. To approve that non-interactively — in scripts or CI, where
the prompt would otherwise refuse — pass `--approve-replacing-active-default`,
which is also what the refusal itself recommends and what ROCm CLI's own
non-interactive surfaces (chat, MCP, the dashboard) pass. `--yes` grants the same
approval *and* approves installing required system packages (such as OpenMPI for
vLLM), which means `sudo`; reach for it only where something can answer a sudo
password prompt — which an unattended job cannot, unless it has passwordless sudo
configured. In the default managed install root, the root and its manifest are
keyed by version, so an upgrade or downgrade keeps the previous install on disk
and only a same-version reinstall reuses the same root. `--prefix` opts out of
that: the folder you name is used verbatim for every version, so successive
installs into one prefix replace each other in place — and if the venv already
there no longer runs its own Python, it is removed outright and rebuilt. The
consent gate does not cover that: it asks about changing the active default
runtime, not about what a named prefix loses. `install driver` installs the AMD
kernel driver on Linux (DKMS or native package). `update` checks for a newer
ROCm package; pass `--apply` to install it, or `--dry-run` to preview what
`--apply` would do without changing anything (`--dry-run` does not require
`--apply`). `--runtime` and `--activate` require `--apply` or `--dry-run` — pass
one of those instead of naming a runtime or requesting activation on its own.
`--json` prints the check result as a single line of JSON instead of text;
`--timeout-secs` bounds its network calls (`--timeout-secs` requires `--json`;
both `--json` and `--timeout-secs` conflict with `--apply`, and `--json` also
conflicts with `--dry-run`). `update --apply` never prompts and needs no
approval flag: selecting a runtime to update is itself the approval, and it
leaves the active default alone unless you add `--activate`. `update` does
accept `--yes`, for consistency with other mutating commands, but it grants
nothing there — the approval line the update path prints never credits it.

ROCm 10 and newer ship from a different source layout. It is opt-in, and asking
for it takes two things together: pin the version with `--version`, and name the
exact GPU arch — the raw `gfx` code, not a family label:
by rocm-cli.

#### Compiler toolchain (--devel)

Pass `--devel` to also install the compiler and headers needed to build GPU
code. This roughly doubles the download.

`--devel` isn't an addition to an existing runtime. A runtime is identified by
the packages it was installed from, so running `rocm install sdk` and later
`rocm install sdk --devel` at the same version leaves you with **two**
side-by-side runtimes. The second is a fresh full install, not a toolchain
bolted onto the first, and it becomes active.

To tell the two apart:

- `rocm runtimes list` marks each runtime `toolchain=included` or
`toolchain=excluded`.
- `rocm examine` reports the active runtime's toolchain as
`active_runtime_toolchain`.
- `rocm storage remove-old-installs` counts the two kinds separately, so
neither evicts the other.

To reclaim the space, uninstall the one you don't want with
`rocm runtimes uninstall <runtime-key>`.

#### Approval prompt

If no managed runtime is the active default, `install sdk` doesn't prompt.
Otherwise it asks first, because the new install becomes the active default.
The prompt applies to any install, including a `--family` or `--channel` you
haven't installed before, just as it does for a same-family upgrade.

To approve without a prompt, for example in scripts or CI, where the prompt
would otherwise refuse:

- `--approve-replacing-active-default` approves the change of active default.
The refusal message recommends it, and ROCm CLI's own non-interactive
surfaces (chat, MCP, and the dashboard) pass it.
- `--yes` gives the same approval and also approves installing required system
packages, such as OpenMPI for vLLM. That requires `sudo`, so use it only where
something can answer a sudo password prompt. An unattended job can't, unless
it has passwordless sudo configured.

#### Install location

In the default managed install root, the root and its manifest are keyed by
version. An upgrade or downgrade keeps the previous install on disk. Only a
same-version reinstall reuses the same root.

`--prefix` changes this. The folder you name is used as-is for every version, so
successive installs into one prefix replace each other in place. If the venv
already there no longer runs its own Python, it is removed outright and rebuilt.
The approval prompt doesn't cover this, because it asks only about changing the
active default runtime, not about what a named prefix loses.

#### Driver installation

`install driver` installs the AMD kernel driver on Linux, using DKMS or a native
package.

#### Updates

`update` checks for a newer ROCm package.

| Flag | Description |
| --- | --- |
| `--apply` | Installs the update. Never prompts and needs no approval flag, because selecting a runtime to update is the approval. Leaves the active default alone unless you add `--activate`. |
| `--dry-run` | Previews what `--apply` would do without changing anything. Doesn't require `--apply`. |
| `--runtime`, `--activate` | Require `--apply` or `--dry-run`. |
| `--json` | Prints the check result as a single line of JSON instead of text. Conflicts with `--apply` and `--dry-run`. |
| `--timeout-secs` | Bounds the network calls of the check. Requires `--json`. Conflicts with `--apply`. |
| `--yes` | Accepted for consistency with other mutating commands, but grants nothing on `update`. The approval line the update path prints never credits it. |

#### ROCm 10 and newer

ROCm 10 and newer ship from a different source layout. You opt in by passing two
things together: pin the version with `--version`, and name the exact GPU arch,
using the raw `gfx` code rather than a family label:

```
rocm install sdk --version 10.0.0 --family gfx1200 --dry-run
Expand All @@ -375,9 +405,9 @@ selected framework package carries the same ROCm build identifier before it
creates or changes a managed runtime.

Nothing about this happens on its own. Without a `--version` of 10 or newer,
`install sdk` resolves the same release and nightly sources it always has, and
it never quietly retries against the ROCm 10 sources when a lookup comes up
empty — it tells you what it could not find instead.
`install sdk` resolves the same release and nightly sources as before. It doesn't
quietly retry against the ROCm 10 sources when a lookup finds nothing; it tells
you what it couldn't find instead.

### Runtime management

Expand Down
4 changes: 4 additions & 0 deletions docs/rocm-docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ SPDX-License-Identifier: MIT

# Command reference

This page describes each `rocm` command, its options, and what it does. For a
short list of the commands and what each is for, see
[Getting started](getting-started.md).

```{include} ../../README.md
:start-after: "## Commands"
:end-before: "and a chat tab backed by any configured provider."
Expand Down
8 changes: 8 additions & 0 deletions docs/rocm-docs/engines/vllm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<!--
Copyright © Advanced Micro Devices, Inc., or its affiliates.

SPDX-License-Identifier: MIT
-->

```{include} ../../vllm.md

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file {include}s docs/vllm.md into the built Sphinx site for the first time, but .github/workflows/ci.yml's docs: path filter (around lines 165-174) only watches docs/rocm-docs/**, README.md, CONTRIBUTING.md, .readthedocs.yaml, and .github/workflows/** — it doesn't list docs/vllm.md. A future PR that edits docs/vllm.md alone (a broken include, a bad code fence) won't trigger the "Sphinx docs build (-W)" check that would catch it, even though this page now ships in the site. Worth adding docs/vllm.md to the docs: filter list.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @juhovainio. I've added docs/vllm.md to the path filter in .github/workflows/ci.yml and extended the comment above it.

```
16 changes: 15 additions & 1 deletion docs/rocm-docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,24 @@ SPDX-License-Identifier: MIT

```{include} ../../README.md
:start-after: "## Configure ROCm and serve a model"
:end-before: "Running the command when a"
```

<!-- The prose below is a deliberate copy of the README sentence, with the
cross-reference retargeted to this site. The link text is then reused as
the `:start-after:` anchor for the next include, which also matches the
original sentence in README.md; edit both together. -->
Running the command when a managed runtime is already the active default asks
first, because the new install takes over as the active default; see
[ROCm installation](commands.md#rocm-installation) for that gate and the flags
that approve it without a prompt.

```{include} ../../README.md
:start-after: "for that gate and the flags that approve it without a prompt."
:end-before: "You can also serve any compatible Hugging Face model directly"
```

You can also serve any compatible Hugging Face model directly — see
You can also serve any compatible Hugging Face model directly. See

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This duplicates the previous two lines ("You can also serve any compatible Hugging Face model directly — see / [Model serving]... for the GGUF-vs-safetensors rule,") with slightly different wording ("directly. See" vs "directly — see"). The old pair is still there right above as unchanged context — only one of the two should remain. Note the code comment a few lines up says to edit this sentence in lockstep with README.md's copy; README.md's own copy (git show 409ff164:README.md around "You can also serve") wasn't touched, so whichever version you keep here, please re-sync it with README.md too.

[Model serving](commands.md#model-serving) for the GGUF-vs-safetensors rule,
since which form works depends on the engine your GPU selects.

Expand Down
10 changes: 4 additions & 6 deletions docs/rocm-docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ adapters for Lemonade and vLLM.
.. important::

**Tech Preview:** This software is provided as-is, without warranty or
guarantee of stability. APIs, commands, and behavior may change without
guarantee of stability. APIs, commands, and behavior might change without
notice. Intended for experimentation and early feedback only.

The ROCm CLI public repository is located at
Expand All @@ -26,18 +26,16 @@ The ROCm CLI public repository is located at
.. grid:: 2
:gutter: 3

.. grid-item-card:: Demos

* :doc:`See ROCm CLI in action <demos>`

.. grid-item-card:: Install

* :doc:`Installing ROCm CLI <install/installation>`

.. grid-item-card:: Getting started

* :doc:`Getting started with ROCm CLI <getting-started>`
* :doc:`See ROCm CLI in action <demos>`

.. grid-item-card:: Commands
.. grid-item-card:: Use ROCm CLI

* :doc:`Command reference <commands>`
* :doc:`vLLM adapter <engines/vllm>`
8 changes: 7 additions & 1 deletion docs/rocm-docs/install/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,13 @@ ROCm CLI ships as a single prebuilt binary. Platform support:

Live dashboard telemetry requires Linux or WSL2 (see
[Interactive interfaces](../getting-started.md#interactive-interfaces)). vLLM
serving is Linux or WSL2 only (see `docs/vllm.md`).
serving is Linux or WSL2 only (see
[vLLM adapter](../engines/vllm.md)).

```{include} ../../../README.md
:start-after: "only (see [docs/vllm.md](docs/vllm.md))."
:end-before: "> [!IMPORTANT]"
```

```{include} ../../../README.md
:start-after: "## Installation"
Expand Down
11 changes: 6 additions & 5 deletions docs/rocm-docs/sphinx/_toc.yml.in
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
root: index
subtrees:
- caption: Demos
entries:
- file: demos
title: See ROCm CLI in action
- caption: Install
entries:
- file: install/installation
- caption: Getting started
entries:
- file: getting-started
- caption: Commands
- file: demos
title: See ROCm CLI in action
- caption: Use ROCm CLI
entries:
- file: commands
title: Command reference
- file: engines/vllm
title: vLLM adapter
- caption: About
entries:
- file: about/contributing
Expand Down
Loading
Loading