docs(fern): scaffold Fern docs site mirroring published v0.4.0 sidebar

[lbliii]

Summary

conversion preview

Migrates the NeMo AutoModel docs from Sphinx (docs/) to a Fern MDX site (fern/), mirroring the published v0.4.0 sidebar at https://docs.nvidia.com/nemo/automodel/latest verbatim — engineers can side-by-side diff against the live site without spurious title shifts. The Sphinx tree under docs/ is left in place for now; removal is a follow-up PR after preview sign-off.

Content layout

fern/
├── docs.yml                  # site config (instances, versions, redirects, libraries, theme)
├── fern.config.json          # Fern CLI pin (≥ 4.62.4)
├── main.css                  # NVIDIA theme
├── assets/                   # NVIDIA SVGs
├── components/               # CustomFooter.tsx, BadgeLinks.tsx, Tag.tsx
├── versions/
│   ├── nightly.yml           # paths → ./nightly/pages/
│   ├── nightly/pages/        # bleeding-edge tree — edited every PR (130 pages)
│   ├── v0.4.yml              # paths → ./v0.4/pages/
│   ├── v0.4/pages/           # frozen 0.4.0 tree — back-ports only (130 pages)
│   └── latest.yml            # GA alias — paths → ./v0.4/pages/; repoints on next GA cut
├── product-docs/             # GENERATED Python API reference (gitignored)
└── README.md

nightly/pages/ and v0.4/pages/ are separate, independent content trees with their own copy of every file. They're byte-identical today (0.4.0 just shipped from this exact content) and will diverge as nightly accumulates post-release edits. latest.yml mounts the current GA's content (today: ./v0.4/pages/...).

Version dropdown matches the published Sphinx site:

display-name	slug	availability
nightly	nightly	beta
Latest	latest	stable
0.4.0 · 26.04	v0.4	stable

What's in this PR

1. Sphinx → Fern conversion of all 130 MyST .md files, plus 12 image assets, into MDX. Sidebar structure, section captions, page titles, and Model Coverage child ordering are taken verbatim from the published v0.4.0 site (not from the local source toctree, which has post-release additions).
2. Two physical content trees — nightly/pages/ and v0.4/pages/ — so 0.4.0 can stay frozen while nightly moves forward. latest.yml is the GA alias.
3. NVIDIA theme + custom-domain at docs.nvidia.com/nemo/automodel. experimental.basepath-aware: true set for subpath hosting.
4. Redirects for old Sphinx URL forms — /index.html, /<page>.html, /<version>/<path>/index.html, the legacy 0.4 → v0.4 rewrite, and explicit per-version-root rules (/<version>/index.html) because Fern's :path* does not match the empty-path case (docs: fix broken links — redirects, GitHub source links, drift Curator#1938).
5. Slug overrides on every page so URLs stay short (/get-started/installation) while sidebar titles match the verbose published H1s (Install NeMo AutoModel).
6. API library reference via Fern's libraries: (replaces the Sphinx autodoc2 setup). Pyright-extracts Markdown from nemo_automodel/ on every preview/publish; fern/product-docs/ is gitignored.
7. CI workflows with the FW-CI mirror-bot trigger pattern (push: pull-request/[0-9]+, matching existing build-docs.yml):
   • fern-docs-ci.yml — fern check on PRs
   • fern-docs-preview-build.yml + fern-docs-preview-comment.yml — split untrusted/trusted preview build that posts a 🌿 comment
   • publish-fern-docs.yml — publishes on push-to-main with fern/** changes, on docs/v* tag, or via manual dispatch
8. Makefile at repo root with make docs / docs-check / docs-preview / docs-publish targets so contributors don't have to memorize cd fern && fern … invocations.
9. fern/README.md — human-facing orientation (layout, local dev, sidebar fidelity rule, authoring conventions, versioning cookbook, troubleshooting).
10. skills/fern-docs/SKILL.md — agent skill registered in AGENTS.md and skills/README.md (entry ddp parity #10) covering page operations, slug + redirect cookbook, redirect quirks (:path* empty-path gotcha), pnpm 10 esbuild gotcha, and the cut-a-new-train procedure.

Required infrastructure

• DOCS_FERN_TOKEN org secret — already wired for the existing build-docs.yml; no new setup needed.
• FW-CI mirror bot — fern-docs-ci.yml triggers on the same pull-request/[0-9]+ branches the bot already pushes to for build-docs.yml.

What's NOT in this PR

• docs/ removal. Left in place so reviewers can diff Fern output against the Sphinx source. Remove in a follow-up after preview review.
• A real frozen v0.4.0 snapshot. Today's v0.4/pages/ is the same content as nightly/pages/ (current main); a true v0.4.0 snapshot would need to be migrated from the v0.4.0 git tag's docs/. Out of scope here; back-fill in a follow-up.
• Nightly publish automation. The Sphinx side has release-nightly-docs.yml; a Fern equivalent that publishes versions/nightly/pages/ on push-to-main is a follow-up. The current publish-fern-docs.yml only fires on docs/v* tags or manual dispatch.
• Sphinx workflow retirement. release-docs.yml and release-nightly-docs.yml continue to publish the Sphinx site; retire or repoint at fern/ once Fern is the canonical source.

Validation

All 7 convert-to-fern validation checks pass:

Check	Result
validate_fern_version_yml	✅
fern_check	✅
fern_docs_broken_links (advisory --skip-broken-links; strict mode false-positives on version-agnostic links)	✅
check_mdx_safety	✅
check_unconverted	✅
validate_fern_internal_links (URLMap-based, authoritative)	✅ — 0 invalid internal links across 128 nav-routed pages
sanity_check_migration	✅

Test plan

• Run make docs locally, confirm sidebar groups match the published 0.4.0 site one-for-one
• Spot-check ~10 page titles against docs.nvidia.com/nemo/automodel/latest (e.g. "Install NeMo AutoModel", "Supervised Fine-Tuning (SFT) and Parameter-Efficient Fine-Tuning (PEFT) with NeMo AutoModel", "Fine-Tuning Gemma 4 31B on CORD-v2 Receipts — End-to-End Guide")
• Confirm version dropdown shows nightly (Beta), Latest (Stable), 0.4.0 · 26.04 (Stable)
• Verify a few internal links resolve: /get-started/installation, /recipes-e2e-examples/sft-peft, /model-coverage/large-language-models/llama
• Verify image rendering: gemma3-3n.mdx (medpix images), databricks.mdx (GPU metrics), pretraining.mdx (gpt2 loss), fp8-training.mdx, latest-models.mdx (Brev launchable badge)
• After preview deploy, confirm redirects: docs.nvidia.com/nemo/automodel/v0.4/index.html → /v0.4 (the empty-path case), /nemo/automodel/about/index.html → /about, /0.4/<path> → /v0.4/<path>
• Confirm API Reference renders under Development → API Reference (mounts product-docs/nemo-automodel/Full-Library-Reference/)

Commits

0b3751ef	Scaffold Fern docs (130 MDX, sidebar parity)
5a953d70	CI workflows + basepath-aware
a8444868	API library reference + Makefile docs targets
233138dd	fern/README.md + skills/fern-docs/SKILL.md agent skill
420b59d2	Empty-path redirect fix (8 explicit version-root rules)
8d173557	Drop _nav_order / _nav_titles migration sidecars
4d65d764	Repoint nightly as the canonical YAML name
a181840c	Split nightly and v0.4 into separate physical content trees

All commits are DCO-signed.

🤖 Generated with Claude Code

[jgerh]

PR Review — Sphinx → Fern Migration Scaffolding Checks Completed

• Documentation file mapping — All 130 source .md files map cleanly to 131 .mdx files in both v0.4/pages/ and nightly/pages/ (the additional file is an expected index.mdx stub). No missing or orphaned pages.
• Non‑docs directories — All non‑docs directories match one‑to‑one across branches, with the only intentional delta being skills/fern-docs/SKILL.md. Divergence in examples/ is expected due to ongoing feature work.
• Navigation reachability — Every path: entry in both nav files resolves to a real file.
• Slug wiring — Representative spot checks confirm slugs are correctly assigned and consistent with v0.4.yml.
• Redirect rules — All redirect patterns described in the PR are present and correctly wired in docs.yml.
• Validation checks — All seven convert‑to‑fern validations pass, including zero invalid internal links across 128 nav‑routed pages.
• Local preview — Verified sidebar grouping, version dropdown behavior, internal routing, image rendering, and page titles in preview.

Verdict
Everything required for the migration scaffolding is in place. No blockers. v0.4 and nightly are fully aligned, navigation is intact, and validation is clean. Redirects and API Reference rendering appear correct but will be fully confirmed on live deploy. Safe to merge once reviewers sign off.