Repository navigation
ci(prek): check markdown links locally with lychee; cite repo files as links - #602
Conversation
565b92b to
4dcaeb1
Compare
jussielo-amd
left a comment
There was a problem hiding this comment.
Reviewed this change. Overall it looks solid — most of the diff (the lychee.toml regex, CI/pre-commit comment accuracy, version-pin consistency between lycheeVersion: v0.24.2 and cargo install lychee@0.24.2, Windows path-separator handling, and the script's "not found" vs. "found but erroring" distinction) checks out correctly. The notes below are non-blocking robustness and documentation-precision points, not correctness bugs.
scripts/lychee_if_available.py:217 — subprocess.run() has no timeout, so a hung or pathological lychee invocation blocks git commit indefinitely. Since this hook runs with always_run: true on stages: [pre-commit], a repo state lychee handles badly (e.g. a large generated tree accidentally un-ignored, a symlink loop, or a slow/contended filesystem) would hang every subsequent commit — worse than the intended "fail open when absent" behavior.
scripts/lychee_if_available.py:209 — shutil.which("lychee") only checks the hook subprocess's own PATH. If lychee is installed but not on the PATH the hook inherits (e.g. ~/.cargo/bin missing in a minimal shell or IDE-launched commit), it silently prints the "not found" warning and skips the check — indistinguishable from genuinely-not-installed, giving false confidence the check ran locally when it didn't.
.pre-commit-config.yaml:106 — The new lychee hook uses language: python, unlike every other local hook (commit-signing-configured, verify-commits, license-headers, cargo-manifest/catalog/tpn), which use language: system — with no comment explaining the deliberate choice. A future maintainer "fixing" the inconsistency to match siblings could reintroduce a "python: command not found" failure on systems without a python shim.
CONTRIBUTING.md:141 — Wording narrowed from "its path citations are relative markdown links" to "the file citations in its per-crate inventories are relative markdown links," which undersells what's checked — docs/architecture.md has file-link citations in ordinary prose too, and lychee checks the whole file regardless of this sentence's scope.
AGENTS.md:193 — The §5 bullet lumps .github/ and docs/rocm-docs/ (genuinely excluded/unreachable by lychee) together with "README.md and CONTRIBUTING.md" under "where the rule does not reach" — but README.md/CONTRIBUTING.md are not excluded from lychee's check; both are fully scanned by the docs-links CI job and this new local hook. The actual reason to avoid file-link citations there is a different, stricter failure mode (sphinx-build -W breaking on an out-of-tree relative link), which reads as if it belongs to the same "unreached" category as the truly-excluded paths.
jussielo-amd
left a comment
There was a problem hiding this comment.
LGTM — non-blocking notes left in a separate review comment.
|
Thanks @jussielo-amd. Your review was on 512615f, which had already replaced
|
Dismissing so the approval covers the current head: cf6daa9 changes wording in AGENTS.md, CONTRIBUTING.md and the xtask skip message in response to your notes. Re-requesting your review.
jussielo-amd
left a comment
There was a problem hiding this comment.
Looks good — no correctness issues found across a close read of the diff. This mirrors the existing docs-links CI job pattern well and comes with solid test coverage, including a pinned-version drift test for the lychee version check.
A few non-blocking notes for a possible follow-up:
-
xtask/src/lychee.rs:222— the Windows unit-test stub spawns a.cmdscript directly viaCommand::new(path), which relies on undocumentedCreateProcessbatch-file handling that Rust's own docs say may be removed (see rust-lang/rust#123728, related to CVE-2024-24576). The production path (invoking a reallychee.exe) isn't affected, but if that fallback is ever dropped, this test would start failing on Windows CI. Might be worth spawning viacmd.exe /c <script>explicitly instead. -
xtask/src/lychee.rs:40— theProbe/Checker/classify_checker/parse_lychee_versionscaffold duplicates the pattern already used for thetpncommand inxtask/src/tpn.rs. Since this is now the second occurrence of "probe an optional pinned external tool" in the crate, it might be worth extracting a shared helper before a third one copies it again. -
xtask/src/lychee.rs:102— the(Checker, if_available)match incheck()interleaves two warning arms whose text doesn't depend onif_availablewith the one arm that does branch on it. Separating the always-run warnings from the singleif_available-gated case would make the policy easier to scan.
None of these block the change.
The architecture-doc checker and its pre-push hook are gone; the docs-links CI job (lychee) is now the only thing that notices a moved or deleted file breaking a relative link, a full CI round-trip later. Add a `lychee` hook with the same config as that job, on pre-commit since it checks the whole repo offline in well under a second. It is `always_run` because a rename or deletion breaks links without touching any markdown file, and a deletion-only commit otherwise leaves the hook nothing to match. It is in the `local-tools` group, so the prek CI job skips it in favour of docs-links. The hook walks `.` with `--extensions md` instead of CI's './**/*.md' glob: lychee's glob ignores .gitignore, so gitignored scratch markdown would fail every local commit while CI never sees it. Both skip hidden directories, so they check the same tracked files; lychee.toml's rocm-docs exclusion now also matches the walk's `./` prefix (and either path separator, for Windows). CONTRIBUTING.md pins lychee 0.24.2, the version lychee-action v2.9.0 installs by default, notes the working-tree caveat, and the link-coverage claim now names the hidden directories (.github/) neither check reaches. The docs-links job now names lycheeVersion explicitly (the same v0.24.2 lychee-action v2.9.0 defaults to), so an action bump cannot silently drift from the local pin, and the ci.yml prek-job comment lists the new local-tools hook. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
docs-links can only catch a stale citation that is a link; a bare backtick path goes stale silently. State that rule in AGENTS.md's docs-sync bullets, where agents look while editing docs, and point CONTRIBUTING.md's link-checking paragraph at it rather than keeping a second copy that could drift. The rule extends the exceptions docs/architecture.md already makes (a deleted file referenced historically, generated output like report.json) to the other mentions that name no tracked file: a generic `main.rs` and globs. It also covers the link forms that trip lychee (`file.rs:42` is read as a filename; a `#L42` anchor is never checked), the need to commit a newly linked file with the link, and the places it does not reach: .github/ templates, the separately built docs/rocm-docs/ site, and links leaving a skill folder under skills/, which is published or embedded elsewhere. It covers citations being added or edited, not wholesale conversion. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
CONTRIBUTING.md and README.md are `{include}`d into the docs/rocm-docs/
Sphinx site, where a relative link to a file outside that tree is an
unresolved MyST cross-reference and fails `docs-build`'s `sphinx-build
-W`. The links to lychee.toml and to AGENTS.md §5 added there did exactly
that. Make them bare again, and list the two included files among the
places the link-citation rule does not reach.
Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
lychee.toml and CONTRIBUTING.md described all of docs/architecture.md's path citations as checked links, but only the file citations in its per-crate inventories are links; the section headings and directory names stay bare. docs/architecture.md already states the narrower guarantee. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The hook runs on every commit, so failing hard on a missing binary would block anyone with prek hooks installed from committing until they install lychee. Like cargo-tpn, warn and pass instead; the docs-links CI job stays the hard gate. A small wrapper keeps lychee's per-run summary out of the verbose hook output so only the skip warning or a real failure is shown. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Replace the Python wrapper with an xtask subcommand, matching how the cargo-tpn hook handles an optional tool. The pinned lychee version now lives in one constant, with a test that fails if ci.yml's lycheeVersion or CONTRIBUTING.md's install line drifts from it, and a lychee other than the pinned version runs with a warning. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Say "not found on PATH" rather than "not installed" when the hook skips, since an installed lychee off the hook's PATH looks the same. CONTRIBUTING.md now says architecture.md's file links are checked wherever they appear, not only in the per-crate inventories, and AGENTS.md says the link rule does not apply to README.md/CONTRIBUTING.md rather than that lychee does not reach them: it does scan them, but the Sphinx build fails on such links. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
cf6daa9 to
5918759
Compare
Stale: approved cf6daa9; head is now 5918759, after a rebase onto main. git range-diff shows four of the seven commits identical. The other three differ only in surrounding CONTRIBUTING.md context, from main's hawkeye paragraph landing next to the lychee one. No change to what was approved. A fresh approval on the current head is needed to merge.
Why
#582 replaced the
cargo xtask check-architecture-docchecker with lychee and deleted the checker'sarchitecture-docpre-push hook. Since then, thedocs-linksCI job is the only check that a relative markdown link still resolves, so a moved or deleted file that breaks a link is first reported a full CI round-trip later. And the check only protects citations written as links: a bare backtick path goes stale with nothing to notice.What changes
lycheeprek hook (.pre-commit-config.yaml), using the samelychee.tomlasdocs-links.pre-commit: it checks the whole repo offline in well under a second.always_run: true: a rename or deletion breaks links without touching any.md. Without it, a deletion-only commit leaves the hook "(no files to check) Skipped" while two links break.local-toolsgroup, so theprekCI job skips it anddocs-linksstays the gate.cargo xtask lychee --if-available(newxtask/src/lychee.rs, modelled oncargo xtask tpn --if-available). If lychee isn't installed, the hook prints a warning and passes, so a missing tool never blocks a commit. A lychee at another version, or one whose version can't be read, runs with a warning. Its output is shown only on failure. Cost: about 0.3 s per commit with xtask already built, about 5 s after an edit to xtask orrocm-core, the same build the existing xtask hooks share..with--extensions mdinstead of CI's./**/*.mdglob. lychee's glob does not honour.gitignore, so gitignored scratch markdown (plans/,target/, ...) would otherwise fail every local commit while CI never sees it. Both forms skip hidden directories, so they check the same tracked files.lychee.toml'sdocs/rocm-docs/exclusion now also matches the walk's./prefix and either path separator.CONTRIBUTING.mddocumentscargo install lychee@0.24.2 --locked, next to the pinnedhawkeye. 0.24.2 is thelycheeVersiondefault in lychee-action v2.9.0'saction.ymlat the pinned SHA. Thedocs-linksstep now setslycheeVersion: v0.24.2explicitly, so an action bump can't silently drift from the local pin, and a unit test ties both to the xtask constant.docs/architecture.md's intro: generic mentions, deleted files, globs, and gitignored or generated output stay bare.file.rs:42is read as a filename, and a#L42anchor is never checked..github/templates,docs/rocm-docs/, README.md and CONTRIBUTING.md, or links leaving askills/folder. README.md and CONTRIBUTING.md are{include}d into the Sphinx site, wheresphinx-build -Wfails on a link to a file outside it.ci.ymlprek-job comment lists the new local-tools hook.lychee.tomlandCONTRIBUTING.mdno longer describe all ofdocs/architecture.md's path citations as checked links. Only the file citations in its per-crate inventories are links; its section headings and directory names stay bare.docs/architecture.mdalready states this narrower scope, so the two now match it.Verification
Run locally with lychee 0.24.2 and prek:
prek run lychee --all-filespasses: 171 links, 0 errors. The same command as CI (lychee --config lychee.toml './**/*.md') gives the same count. Re-run after rebasing ontomainwith docs(architecture): rewrite citations as links, delete arch-doc checker #582 merged: still 171 links, 0 errors.--dump-inputslists the same 24 tracked files for the hook form and for the CI glob.docs/architecture.mdfails the hook.apps/rocm/src/uninstall.rswith no.mdedit fails the hook.always_run, and is skipped without it.#anchoron a link to a markdown file fails the hook.plans/and atactive-implementation-log.mdare ignored by the hook. The glob form flags them.docs/rocm-docs/stays excluded.prek run --all-files --no-group local-tools(the prek CI job's set, includingcheck-yaml) passes.sphinx-build -b html docs/rocm-docs <out> -Wwithdocs/rocm-docs/sphinx/requirements.txt(thedocs-buildjob's command) passes.cargo xtask verify-commitspasses: all six commits are signed and signed off.This is CI config and docs only, with no CLI behaviour change, so there is no e2e scenario and
cargo testwas not run.Not verified: the hook on native Windows. That includes the backslash branch of the
exclude_pathregex, which I tested only with Pythonre, and the Windows (.cmd) stub the new unit tests use, which I only type-checked for that target; CI'swindows-build-and-testruns it.cargo test -p xtaskpasses (229 tests) andcargo clippy -p xtask --all-targets -D warningsis clean. The new tests runcheckagainst a stub lychee to cover each path: the pinned version passing and failing, another version and an unreadable version still running, and a missing lychee (passes with--if-available, fails naming the install command without it). A test also fails ifLYCHEE_VERSIONdrifts fromci.yml'slycheeVersionor the CONTRIBUTING.md install line. Mutating the code to skip an unreadable version, or bumpingci.yml's version, made those tests fail.