Conversation
Every `matchImage` comparison is now recorded in a JSON manifest, by default `<screenshotsFolder>/cp-visual-regression-diff-manifest.<testingType>.json`, so tooling that runs after Cypress (PR comments, review dashboards, approval bots) can find out what passed, failed, created or updated its baseline without parsing logs or diffing the snapshots folder. Each entry carries the originating test (file, title path, retry), the status, diff ratio and threshold, project-relative paths and sizes of the baseline, `.actual.png` and `.diff.png` files, whether the baseline was written, and browser and viewport. Approving a screenshot from the headed review UI flips its entry to `approved`. The file is rewritten (write-then-rename) after every comparison, so it is complete even when the run is aborted; `before:run` starts from scratch, and re-running a spec in `cypress open` drops that spec's entries first. Retries purge the entries left by earlier attempts of the same test. The format is runner-agnostic: only the `runner` block is Cypress-specific. `pluginVisualRegressionManifestPath` moves the file or, with `false`, disables it. The `Manifest*` types are exported from `./plugins`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Resolve the two task.hook.ts hunks by keeping both sides: the manifest status bookkeeping and the stale .diff.png cleanup. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The manifest now carries what a CI consumer needs to find the run and to reproduce it, not only the comparison results: - run-level `createdAt`/`updatedAt`, `platform` (os, arch, os version), `ci` detected from GitHub Actions / GitLab CI env vars (repository, sha, ref, pull request incl. head sha, run id/attempt, workspace), the global `pluginVisualRegression*` options with the prefix stripped, and a richer `runner` block (mode, config file, launched browser, specs, spec pattern, base url, viewport, retries, Cypress Cloud fields) - per-entry `recordedAt`, `platform` (os, arch, browser; replaces the top-level `browser`) and the resolved `matchImage` `options` Run data is seeded at `initPlugin`, the launched browser is recorded from `before:browser:launch` (now a single handler shared with the device scale factor flags) and `before:run` merges its details. The format version stays 1 because nothing shipped yet. CI uploads the example manifest with the screenshot artifact. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every
matchImagecomparison now ends up in a JSON manifest, by default at<screenshotsFolder>/cp-visual-regression-diff-manifest.<testingType>.json. The idea is that whatever runs after Cypress in CI (a PR comment bot, a review dashboard, an approval tool) can read one file instead of parsing logs or diffing the snapshots folder.Each entry has:
passed,failed,missing-baseline,created,updatedorapproved.actual.pngand.diff.png,nullwhen the file isn't on diskbaselineWritten, so you know whether the working tree changed regardless of statusplatform(os, arch, browser incl. family/headless), viewport, and the resolvedmatchImageoptions it ran with (imagesPathkeeps its tokens unexpanded), plus arecordedAttimestampAnd the file itself now says where and how the run happened, which is what a bot needs to post on the right PR and grab the right artifact:
createdAt/updatedAtplatform: os, arch, os version of the machineci: auto-detected on GitHub Actions and GitLab CI - repository, sha, ref, branch, pull request number (with head sha/ref, sinceGITHUB_SHAis the merge commit on PR events), run id/attempt, job, workflow, run URL, and the checkout dir (workspace, handy in monorepos whereprojectRootisn't the repo root). Other CI systems get{ provider: null }, local runsnull. Only those allow-listed variables are ever copied.options: everypluginVisualRegression*option fromexpose/env, prefix stripped, values as configuredrunner: cypress version, testing type,run/openmode, config file, the launched browser, spec list (run mode only), spec pattern, base url, viewport, retries, Cypress Cloud run url/group/tag when recording. This is the only Cypress-specific block; everything else is meant to be shared with a future Playwright client.A few details worth knowing:
.tmp, then rename), so it's complete even if the run gets killed halfway.before:runwipes it and re-seeds the run data, and re-running a spec incypress opendrops that spec's entries first, so you don't get leftovers from old counters. Incypress openthere's nobefore:run, so the run data comes frominitPluginandbefore:browser:launch(which is now one handler together with the device-scale-factor flags, since Cypress only calls one per event).approved.pluginVisualRegressionManifestPathmoves the file,falseturns it off.versionis still1. Nothing has shipped, so the run metadata went in without a bump.Manifest*types are exported from./plugins.testartifact, so we can dogfood tooling on this repo's own PRs.Docs are in the README (new "Run manifest" section with a field table and consumer notes, including how to reproduce a run from
runner+options) and MIGRATION.md.Related: #416 removes stale
.diff.pngfiles when a comparison passes. Without it apassedentry can still point at a diff image left by a previous failed run, which contradicts the status table in the README. Best to land that one first or together.馃 Generated with Claude Code