Skip to content

Include programatically generated API docs - initial first step - #3376

Merged
borland merged 10 commits into
mainfrom
orion/api-docs
Aug 18, 2026
Merged

Include programatically generated API docs - initial first step#3376
borland merged 10 commits into
mainfrom
orion/api-docs

Conversation

@borland

@borland borland commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

This is the first part of a staged introduction of our API docs into the documentation site.

It adds

  • A new folder /docs/api with all the auto-generated API docs, copy pasted from the Octopus Server repo
  • Supporting infrastructure
    • The API pages have a different layout to the main one
    • They have their own navigation tree as well
    • Includes CSS
    • Example requests and responses need special handling, so there's a satteri-api-examples plugin which processes them
  • The docs/api folder has been excluded from Spellcheck and Markdown lint as it doesn't comply with either. To achieve compliance, we would need to edit as many as 1,800 files in the Octopus Server repo, which is intractable. We'll chip away at this over time from here.

How to review this PR

  • Ignore all the stuff under pages/docs/api. It is programatically generated based on the Octopus Server codebase, and it is equivalent to our swagger content on https://demo.octopus.app/swaggerui/index.html. There are many flaws in the documentation content, however it is already public material and we haven't changed that, so it is not the responsibility of this PR to fix it.

  • Claude wrote the rest of the code. It may not be ideal, technically, but it is very limited in what it needs to do, and the blast radius is contained (a bug will only impact a small section of the site and should not affect other things) so it should be good enough

Treat this as a stepping stone. Once it merges I will proceed with the next steps:

Things this PR does not include

  • We need a landing page for /docs/api which can help people find common things, such as introductory material and explaining that this documentation is for octopus cloud and that LTS versions might be different.
  • We need to move the existing content from /docs/octopus-rest-api into /docs/api and mix it in with the generated content. The generated content may well move to a subfolder such as /docs/api/reference. Will cross that bridge when we get to it.
  • We need some non-generated pages to explain things like authentication, if the existing content is insufficient
  • We need a process to automatically apply changes from the Octopus Server repo. That side hasn't merged yet, and in the interim I'll just apply changes by hand.

When this PR merges, the new files under /docs/api will be live, but nothing will link to them. We will start linking to them and "light them up" as part of that other work.

The API docs format

This is best explained by taking one of the files as an example

---
layout: src/layouts/Api.astro
pubDate: 2026-08-11
modDate: 2026-08-11
title: Access Tokens
---

## Create an access token for the current user

:endpoint{method="POST" path="/api/users/access-token"}

**Response**

`200` — Contains the created access token.

- **`AccessToken`** :span[string]{.type-label}  
  Minimum length 1.

:::api-example{label="Response"}
```json
{
  "AccessToken": "string"
}
```
:::

It's normal markdown, with these enhancements:

  • We use :endpoint{method="POST" path="/api/users/access-token"} to emit <span class="api-post">POST</span> `/api/users/access-token` . CSS styles the API badge to make it look nice, and the navigation generator also has something it can latch onto for reliable nav-generation
  • We similarly use :span[string]{.type-label} to emit string with the type-label CSS class
  • There is a special :::api-example{label="Request|Response"} directive which emits a div for example request/responses. We need something special here because the examples go in a right-hand side column, and Markdown itself has no way to express columns or layout. The satteri plugin sees this directive and emits the example HTML in the right-hand column rather than in-line with the other content.

It looks like this

image

@borland
borland force-pushed the orion/api-docs branch 3 times, most recently from dfe365b to d6cbb33 Compare August 17, 2026 19:04
@team-marketing-branch-protections

Copy link
Copy Markdown

Pull request environment is available at https://stoctodocspr3376.z22.web.core.windows.net.

You can view the ephemeral environment status in Octopus Deploy.

This environment will be automatically deprovisioned when the pull request is closed, or after 7 days of inactivity.

@borland
borland force-pushed the orion/api-docs branch 3 times, most recently from f5ffa1a to 22d1aa1 Compare August 18, 2026 02:19
@borland borland changed the title Orion/api docs Include programatically generated API docs Aug 18, 2026
@borland borland changed the title Include programatically generated API docs Include programatically generated API docs - initial first step Aug 18, 2026
Comment thread src/layouts/Api.astro

@rosslovas rosslovas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't done a review, just had a quick play on the test site:

When this PR merges, the new files under /docs/api will be live, but nothing will link to them.

I'm able to find & navigate to them via search results, blocking the PR just in case

Image

@enf0rc3

enf0rc3 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

I have done some simple testing:

Feel free to take these offline into separate PRs, or do later if you feel its out of scope of this pr (hence why adding as a comment)

  • Left sidebar nav doesnt have "expand" carats, indicating it can expand until it already is
  • Left sidebar nav doesnt keep location when clicking on a top level, i need to re-scroll down to find the opened one.
  • When "Responsive new nav" is merged double check sidebar correctly works. (also make sure we wire up the search)

@rosslovas

Copy link
Copy Markdown
Contributor

@enf0rc3

  • Left sidebar nav doesnt keep location when clicking on a top level, i need to re-scroll down to find the opened one.

FYI that's the case for the main site as well (example page). It's been on my mind for a while as something I'd really like to see fixed!

@enf0rc3 enf0rc3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the infrastructure code only, skipping the generated content per your note. The plugin design is sound — deriving "first H2" from the tree rather than closure state keeps it stateless, and the ordering note for apiExampleDirective after attributeMarkdown in astro.config.mjs is exactly the kind of thing that gets lost otherwise. The Playwright tests are genuinely good, especially running them with javaScriptEnabled: false and asserting the column-stack breakpoint by bounding box.

One real bug and three routing issues. Grouped by urgency, keeping in mind that nothing links to /docs/api yet:

Fix now — these ship publicly on merge regardless of whether anything links to them:

  1. Breadcrumbs render empty on every API page (wrong props to Breadcrumbs). Pure code bug, ~4 lines.
  2. README.md publishes at /docs/api/README/, internal generator instructions and all.
  3. index.md has no frontmatter, so /docs/api/ renders as a bare unstyled page, and all 106 of its links 404.
  4. A decision on navSearch/navSitemap — the ~100 pages enter the sitemap and site search on merge, so Google will index them before you light the section up.

Can wait — the dead CSS rule, Astro.request.url, the crawl skip regex, and the microdata/JSON-LD inconsistency. All flagged inline as deferrable.

Worth noting on 3 and 4: the durable fix belongs in the generator in the Server repo, since these files get overwritten. The suggestions below fix the checked-in copy so this PR is correct today.

I verified the component signatures against source rather than by building, and confirmed the markdownlint ignore works by reproducing your CI invocation locally — that part is fine.

Comment thread src/layouts/Api.astro Outdated
Comment thread src/layouts/Api.astro
Comment thread src/layouts/Api.astro Outdated
Comment thread src/layouts/Api.astro Outdated
Comment thread src/pages/docs/api/_index.md Outdated
Comment thread src/pages/docs/api/access-tokens.md
Comment thread src/styles/api.css Outdated
Comment thread src/components/ApiNavigation.astro Outdated
Comment thread package.json Outdated
"astro": "astro",
"build": "astro build",
"crawl": "linkinator ./dist --skip \"^(?!http://localhost)\" --recurse --verbosity error",
"crawl": "linkinator ./dist --skip \"^(?!http://localhost)\" --skip \"/docs/api(\\$|/)\" --recurse --verbosity error",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferrable, but worth a note. Two things:

  1. The JSON \\$ survives as a literal $ under Windows cmd, which is what npm/pnpm shell out to, so the $ alternative never matches there. The / alternative still covers every subpage, so the only gap is the bare /docs/api URL. On CI's sh it works as intended.

  2. More importantly, this skip is why the 106 broken links on index.md don't fail the build. Once those are fixed it's worth checking whether the skip is still needed, or whether it can be narrowed — right now nothing validates links across 76k lines of generated content, and the generator could start emitting bad ones without anyone noticing.

Fine to leave for now; just worth not letting it become permanent.

Comment thread src/layouts/Api.astro Outdated
lang={lang}
breadcrumbs={breadcrumbs}
/>
<article itemscope itemtype="https://schema.org/Article">

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferrable. This uses schema.org microdata (itemscope/itemprop), which Default.astro dropped in favour of JSON-LD via <JsonLd>. Once the crumbs fix above is in, the site will emit two different structured-data schemes depending on which layout a page uses.

Not urgent and harmless for search engines, but worth collapsing onto JSON-LD next time this file is touched so the two layouts don't drift further apart.

@enf0rc3

enf0rc3 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Following on from my review above — I read the generator in OctopusDeploy/OctopusDeploy#46319 and left a longer review there about the format contract between the two repos:

https://github.com/OctopusDeploy/OctopusDeploy/pull/46319#pullrequestreview-4957018861

Summarising the part that lands on this repo, since some of it changes what I'd do with my earlier comments.

The short version

The generated Markdown encodes this site's implementation details — the Astro layout path and CSS class names — alongside the API content. The clearest symptom is here in src/plugins/satteri-api-examples.js:

function methodFromBadge(node) {
  return METHODS.find((method) => hasClass(node, `api-${method}`)) ?? null;
}

The generator has a MethodCssClasses dictionary mapping GET → api-get. So Swagger says get, the Server repo encodes it as .api-get, and we decode the class back into get to build the nav badge and its aria-label. The semantic value round-trips through a CSS class across two repositories.

The consequence for us: restyling a method badge currently requires a PR in the Server repo and 107 regenerated files. Adding PATCH support is the same. We can't style the endpoint path at all, because it arrives as plain inline code indistinguishable from any other backtick span.

What I've proposed over there

The generator emits data, we decide the paint:

:endpoint{method="POST" path="/api/users/access-token"}
:::deprecated

Plus dropping layout:, pubDate: and modDate: from the generated frontmatter entirely. Deferring the 16,893 .type-label spans and the bold pseudo-headings — big diffs, little reader-visible gain today.

How this changes my earlier review

The index.md and README.md fixes get better if we move /docs/api to an Astro content collection with a single [...slug].astro route, rather than patching the files:

  1. The layout comes from the route, so generated files need no layout: key — which is what lets the Server repo stop writing one.
  2. Zod validation on the collection makes a malformed file fail the build loudly. The index.md bug — no frontmatter, renders as a bare unstyled page — becomes impossible rather than something we happened to catch in review.
  3. README.md can't publish as a public page. It's already listed as hand-written on the Server side, so the shovel just doesn't copy it.

So I'd treat my index.md frontmatter suggestion and the _README.md rename as interim fixes if this merges first, and as unnecessary if we do the collection move now.

Still worth fixing here regardless of any of the above: the empty breadcrumbs (four suggestions above — that's a plain bug, unrelated to the format question), and the navSearch/navSitemap decision.

One thing to correct in this PR either way

The comment on methodFromCode in satteri-api-examples.js describes it as handling "the method an older page spells as inline code". It's actually the live fallback for any method the generator has no CSS class for — MethodOrder over there already lists PATCH/HEAD/OPTIONS/TRACE while MethodCssClasses covers four. The first PATCH endpoint added to the Server gets no badge and no aria-label, silently, in both repos. Zero exist today so it's dormant, but the comment as written invites someone to delete that branch as legacy.

Suggested sequencing

Given nothing links to /docs/api yet, the cheapest order is to agree the format with Orion first, then land both sides together — roughly a day each. If that's more coordination than you want right now, merging this as-is and treating the format as a follow-up is reasonable, with the caveat that the byte-for-byte approval tests over there make every later format change cost a full regeneration.

@borland

borland commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

@enf0rc3 @rosslovas I've pushed a commit which should address the concerns raised. It temporarily excludes the API folder from search and the sitemap, fixes search integration, json-ld, breadcrumbs, etc

Left sidebar nav doesnt have "expand" carats, indicating it can expand until it already is

This wasn't intentional (claude just did it and I didn't notice), but thinking about I think I'd prefer to leave it as-is.

In the main site, the carats mean "Expand" but that typically happens if a node represents a folder with multiple documents within it.

The nav here is one level down. Each entry is a single document and the only thing within the document is multiple H2's. This is equivalent to a normal page in the main docs, and those don't have expanders.

Also, in the near future we're going to intermix human-written content into the api area, and that might well have nested folders, where the expander would have meaning.

@rosslovas
rosslovas dismissed their stale review August 18, 2026 04:22

Concern addressed

@borland
borland force-pushed the orion/api-docs branch 2 times, most recently from b84a4d0 to 3836752 Compare August 18, 2026 04:47
@enf0rc3

enf0rc3 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Edited: an earlier version of this comment flagged a brace-escaping divergence between the two repos. That has since been fixed on the Server side and the section is updated below to say so.

Everything I raised is addressed, and a few of the fixes are better than what I suggested.

Verified fixed:

  • Breadcrumbs — and buildApiCrumbs is the right call rather than the plain buildCrumbs I suggested. Since _index.md is underscore-prefixed there's no page at /docs/api, so the generic walk would have skipped straight from Docs to the endpoint page. Splicing in a urlless section crumb is correct.
  • _index.md / _README.md — both out of routing, and removed on the Server side too.
  • Astro.url.pathname, and the dead code:not(pre code) rule is gone.
  • Microdata — no itemscope/itemprop left in Api.astro, so the two layouts no longer emit different structured-data schemes.
  • Search and sitemapunderConstruction.ts, replied on that thread. Better than my suggestion.
  • The crawl skip is gone, so linkinator now validates links across the API section. That was only possible because the 106 broken ./*.md links left with index.md.
  • :endpoint — the plugin reads data-method/data-deprecated, so the CSS-class round trip through methodFromBadge is gone. Reporting an error on a missing method and a warning naming api.css on an unbadged one closes the silent PATCH gap properly.

Brace escaping — resolved

Briefly the two repos disagreed here: the :endpoint paths in this repo were escaped and the ones committed to the Server PR were not, which would have truncated every path at the first } once a shovel ran.

Fixed on the Server side by EscapeDirectiveValue in ApiReferenceGenerator.cs (commit "We need to escape directive values"). Re-checked after the force-push:

  • source/ApiDocs/channels.md and src/pages/docs/api/channels.md are now byte-for-byte identical.
  • All 20 endpoints in that file are escaped, and both copies parse through Sätteri to the same path attribute, which unescapePath turns back into /api/{spaceId}/channels.

tests/api-endpoint.spec.ts:26 covers this from now on — it asserts the rendered path reads /api/{spaceId}/feeds, so a regression on either side goes red here.

One small nit

buildApiCrumbs in src/lib/apiNavigation.ts: if findIndex doesn't locate the /docs crumb it returns -1, and splice(insertAt + 1, ...) then inserts at index 0 — so the section crumb lands before Docs rather than the insert being skipped. I haven't found a route where the docs crumb is actually missing, so this may well be unreachable. Guarding it with if (insertAt === -1) return crumbs; would make it fail safe either way.

Still open upstream, affects this repo

pubDate/modDate are still hardcoded to 2026-08-11 in the generator, so every page here reports that as "last updated" in the footer, and it's what sitemap.xml.ts would emit as <lastmod> once the section comes out from under construction. Chased it on the Server PR. Not blocking here.

@enf0rc3 enf0rc3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The infrastructure code is in good shape and everything blocking from my earlier reviews is fixed.

Verified against a1fb952:

  • Breadcrumbs render properly via buildApiCrumbs, and Head gets crumbs so the JSON-LD is there.
  • _index.md and _README.md are out of routing, so nothing publishes unstyled and the internal README stays internal.
  • The crawl skip is gone, so linkinator validates links across the API section.
  • underConstruction.ts keeps the section out of site search and the sitemap until it goes live, and both call sites skip before any processing.
  • :endpoint carries method, path and deprecated as data. The plugin errors on a missing method and warns naming api.css on an unbadged one, so a future PATCH endpoint surfaces at build time instead of silently losing its badge.
  • Brace escaping is resolved on both sides. I diffed five generated pages against source/ApiDocs at a28f5d5 — byte-for-byte identical, and the only file that differs across the two repos is _README.md, which correctly stays in the Server repo.

Deliberately deferred, recorded here so they are findable later rather than lost:

  1. pubDate/modDate hardcoded to 2026-08-11 in the generator, so pages report a fixed "last updated" date. Matters most once the section leaves under-construction and reaches the sitemap.
  2. layout: src/layouts/Api.astro written by the generator, naming a docs-repo path from the Server repo. Worth revisiting when the shovel is built, along with a possible move to an Astro content collection.
  3. buildApiCrumbs inserts at index 0 if findIndex returns -1, putting the section crumb before Docs. I could not find a route where that happens, so this is defensive only.

None of these affect readers today. Companion PR is OctopusDeploy/OctopusDeploy#46319.

@borland
borland merged commit 25c52e8 into main Aug 18, 2026
7 checks passed
@borland
borland deleted the orion/api-docs branch August 18, 2026 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants