Browser testing convention — consumer pattern and adoption wave (Atlas, Topology, VirtualWindow)

[cshuttle]

Problem Statement

ADR 0017 (Homelab-Skills docs/adr/0017-browser-testing-convention.md) delivered the provider side of the browser testing convention: the reusable playwright-test.yml workflow, the estate Playwright pin, and lockstep enforcement, with nmon converted as the reference consumer. The consumer side is unfinished in three ways. First, the in-repo half of adoption — Playwright config baseline, how the app-under-test boots on localhost in the runner pod, test layout, what a starter test asserts — is documented nowhere; the README covers CI wiring and pins only, so each adopter would reverse-engineer nmon and diverge. Second, Atlas and Topology have zero browser coverage and no forcing function: "adopt as the suite grows" is idle when there is no suite to grow. Third, VirtualWindow duplicates browser provisioning at estate level — it builds and publishes its own Chromium image with a hand-bumped digest because its previous upstream image was abandoned — which is exactly the disease the convention exists to treat, in a repo with a documented, legitimate reason not to adopt Playwright-the-framework.

Solution

Finish the consumer side in one wave. Write the consumer pattern document in this repo, next to the reusable workflow it versions with, extracted from nmon as the living reference. Seed Atlas and Topology: ephemeral-storage bump on their ARC scale sets, a caller workflow pinned to a release tag, an exact npm Playwright pin under lockstep, and one shell-rendering smoke test each — turning the convention on for every future PR in those repos. Convert VirtualWindow provisioning-only: its node:test + CDP framework stands, but the browser behind its existing image indirection becomes the estate's mirrored Playwright image, retiring the self-built image and hand-bumped digest entirely (both the test fallback and the build-time DOM smoke). Amend ADR 0017 to state the sharpened invariant — the estate owns browser provisioning; the test framework is the repo's business — recording VirtualWindow as a framework opt-out that still consumes estate provisioning, and why it sits outside lockstep. Point the homelab-arc-runners skill at the consumer document.

User Stories

1. As a web-app repo maintainer, I want a consumer pattern document next to the reusable workflow, so that adopting browser testing does not require reverse-engineering the reference consumer.
2. As a coding agent adopting the convention in a repo, I want the in-repo shape (config baseline, app boot, test layout, starter smoke) written once, so that seeded repos make the same choices instead of diverging.
3. As the estate operator, I want Atlas to run a browser smoke on every PR and push to main, so that its kiosk SPA cannot ship a white screen unnoticed.
4. As the estate operator, I want Topology to run a browser smoke on every PR and push to main, so that its frontend gains e2e coverage before regressions accumulate.
5. As a maintainer debugging a red run, I want the smoke test to assert a real root selector rather than an HTTP 200, so that a bundle error that renders nothing fails CI.
6. As a maintainer debugging a red run, I want the Playwright report artifact uploaded on failure in every seeded repo, so that I can see what the browser saw.
7. As the estate operator, I want Atlas's and Topology's npm Playwright pins enforced by the shared lockstep job, so that version drift between package, lockfile, and image fails fast.
8. As VirtualWindow's maintainer, I want its browser tests to run against the estate's mirrored Playwright image, so that an abandoned upstream image can never wedge its CI again.
9. As VirtualWindow's maintainer, I want its self-built Chromium image and both of its uses retired, so that no hand-bumped digest or per-repo image pipeline remains.
10. As VirtualWindow's maintainer, I want to keep the zero-dependency node:test + CDP suite unchanged, so that the repo keeps one way to write tests.
11. As Renovate, I want VirtualWindow's estate-image reference carrying the standard datasource annotation, so that browser bumps flow automatically without lockstep.
12. As the estate operator, I want the Atlas and Topology ARC scale sets to carry the ephemeral-storage request, so that browser jobs are not evicted under disk pressure.
13. As a future reader of ADR 0017, I want the amendment to state the provisioning-vs-framework invariant and VirtualWindow's position, so that nobody "fixes" VirtualWindow into Playwright or back into its own image.
14. As a future reader, I want the amendment to explain why VirtualWindow sits outside lockstep, so that its absence from the lockstep job is not misread as a gap.
15. As an estate agent answering a browser-testing question, I want homelab-arc-runners to point at the consumer document and carry the adoption traps, so that recall lands on the one true copy.
16. As a maintainer of a fringe web-UI repo (Docs, Labelmaker, the Lovelace cards), I want adoption to remain opportunistic but cheap, so that the pattern is ready when a suite is worth writing.

Implementation Decisions

• The consumer pattern document lives in this repo alongside the reusable workflow and its README, so it versions with the interface it documents; the skill gets a pointer plus traps, never a copy (one fact, one home).
• The document covers the in-repo half only: Playwright config baseline with a webServer block booting built assets via vite preview, report output paths matching the workflow's artifact glob, test directory layout, the exact-pin + renovate-rule npm conventions, and the starter smoke shape. CI wiring stays in the README.
• Seeds call the reusable workflow pinned to an exact release tag with runner and the e2e command only; existing unit/lint CI in each repo is left untouched (no unit-command folding in this wave).
• Smoke tests are frontend-only: boot built assets, assert a known root selector renders and no page errors. Topology's backend is not booted; its no-backend render state is acceptable for the smoke.
• VirtualWindow adoption is provisioning-only, behind its existing image-indirection seam: the estate Playwright image replaces the self-built image in both the CDP harness fallback and the image-build DOM smoke. The --dump-dom capability of the estate image is verified before anything is deleted; if it fails, that is a finding to bring back, not a hedge to pre-build.
• VirtualWindow references the estate image directly with its own Renovate annotation and no lockstep job — raw CDP has no client-library version contract pinned at major.minor.
• ARC scale-set changes (ephemeral storage for the two seeded runners) land in the cluster GitOps repo and block the seeds; the workflow's adoption checklist already names this prerequisite.
• ADR 0017 is amended (not superseded): the invariant is sharpened to "the estate owns browser provisioning; the test framework is the repo's business", VirtualWindow is recorded as a framework opt-out consuming estate provisioning, and the implicit expectation that it eventually adopts Playwright is retired. "Browser testing convention" remains the canonical term; the new document is its consumer-facing half.
• Ordering: consumer document first (it is the reference the seeds follow), appset bumps before seeds, VirtualWindow independent, ADR amendment and skill pointer last so they record what actually landed.

Testing Decisions

• Tests assert external behavior only: a rendered shell (real selector), a green suite, an uploaded artifact — never component internals or implementation details.
• Atlas and Topology are tested entirely through the workflow_call seam: a green run of the shared workflow (lockstep passing, smoke passing, artifact wired) is the acceptance test for each seed. Prior art: nmon's caller workflow and suite.
• VirtualWindow's existing browser suite, unchanged, is the regression test for the image swap; its image-build smoke verifies the second retired use.
• The appset change is validated by the existing kustomize-validate CI and a live check that each seeded scale set schedules a runner with the ephemeral-storage request.
• Documents (pattern doc, ADR amendment, skill edit) are review-only; no test seam.

Out of Scope

• Post-deploy synthetic monitoring of live apps (explicitly excluded by ADR 0017; tracked as its own idea in the Monitoring repo).
• Fringe consumers: Docs, Labelmaker, WWW, the Lovelace cards — adoption stays opportunistic.
• Deeper e2e flows beyond the shell smoke, including any full-stack Topology boot — designed when those suites actually grow.
• A scaffold/generator for adoption — copy-paste template plus living references suffices at this consumer count.
• Lint/typecheck/build conventions (out of the reusable workflow's scope by ADR 0017).
• Porting VirtualWindow's tests to Playwright.
• Any remote browser pool (re-litigating ADR 0017).

Further Notes

• Blocking edges for ticketing: appset bump → {Atlas seed, Topology seed}; consumer doc should land first as the reference but blocks nothing mechanically; VirtualWindow conversion is independent; ADR amendment + skill pointer come last.
• Six PRs across five repos: workflows (doc), cluster GitOps (appset), Atlas, Topology, VirtualWindow, Homelab-Skills (ADR + skill).
• The grilling that produced this spec settled: provisioning-only for VirtualWindow over exempt/convert; seeding both greenfield repos over opportunism; amendment over new ADR; full retirement of the self-built image over tests-only.

[cshuttle]

Wave complete — all six child tickets closed, landed 2026-08-19 (merges ~15:04 EDT), each verified beyond merge:

• Consumer pattern document — the in-repo half of the browser testing convention #61 consumer pattern document → docs: the consumer pattern — the in-repo half of the browser testing convention #62: docs/playwright-consumer.md beside the reusable workflow, linked from the README's playwright-test section; nmon cited as the living reference; CI wiring left in the README.
• cshuttle/main#192 ephemeral storage → cshuttle/main#193: ephemeral: "8Gi" on arc-atlas and arc-topology (and arc-virtualwindow — its converted browser uses docker-run the same ~2GB image). kustomize-validate green; verified live: all three AutoscalingRunnerSets reconciled with the request and runner pods scheduled carrying ephemeral-storage: 8Gi.
• cshuttle/Atlas#90 seed → cshuttle/Atlas#91: lockstep + smoke green on PR and on push to main; smoke asserts the brand block (renderer-independent) with zero page errors, booting built assets via vite preview; report artifact wiring exercised (the first red run uploaded it).
• cshuttle/Topology#174 seed → cshuttle/Topology#175: root e2e harness per the doc's subdirectory section; frontend-only smoke green on PR and push to main; vitest/pytest CI untouched and green.
• cshuttle/VirtualWindow#195 provisioning conversion → cshuttle/VirtualWindow#196: estate image verified to serve --dump-dom before any deletion (Chrome for Testing 151 — #190's version sensitivity satisfied); all 22 CDP tests and the image-build DOM smoke green through the estate image in CI; vw-smoke-chrome image, workflow, Dockerfile and both hand-bumped digests gone; renovate annotations + custom manager in place.
• cshuttle/Homelab-Skills#266 record → cshuttle/Homelab-Skills#267: ADR 0017 amended (the invariant: the estate owns browser provisioning, the framework is the repo's business; VirtualWindow recorded as framework opt-out outside lockstep; Atlas/Topology recorded as seeded consumers); arc-runners skill points at the consumer doc and carries the traps.

Traps the wave surfaced, now recorded in the doc/skill: vite preview's IPv6-only localhost bind in the job container (--host 127.0.0.1), docker's 64MB /dev/shm hanging Chrome 151's renderer after the first navigation (--disable-dev-shm-usage), the estate image's non-browser entrypoint and chrome-linux64 path, cold-pull time vs the CDP harness's debug-port window, lockstep reading the checkout root, and gitignored local-only assets absent from CI checkouts.

Post-merge push-to-main runs are green in all three consumer repos. 🤖 Closed with Claude Code