Skip to content

Five faults in the platform resolver, found on merged pull requests nobody answered - #4779

Merged
meshweaver-cloud[bot] merged 9 commits into
mainfrom
fix/copilot-merged-sweep-resolver
Sep 19, 2026
Merged

meshweaver-cloud[bot] merged 9 commits into
mainfrom
fix/copilot-merged-sweep-resolver

Conversation

@rbuergi

@rbuergi rbuergi commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

What this is

A sweep of untreated Copilot findings on MERGED pull requests across the fleet. 240 merged PRs
were swept (the 30 most recently updated per repo, in eight repos): 325 findings, 182 of them
never answered
, across 93 PRs. Systemorph/MeshWeaver is 0 of 30 — the Automatic review answered required context works. Every other repo is the opposite, and in most of them the ratio is
N of N. That gap is core #4776; this PR does not touch it.

This PR treats the 25 findings that all pointed at one fileresolve-platform.py — raised
against five repositories' vendored copies of it. Five distinct defects, all still live in the
canonical, all fixed here. The rest are being treated in their own repositories.

The defects

1 · The attempt fallback was licensed by an unrecognised record, not an empty one
(Education#352, Crm#128, Reinsurance#221 — one defect, three copies.)
#4491 lets the reader consult an older attempt when a partial re-run re-creates a job record
carrying none of the annotations the attempt that ran it published. Its own comment says the licence
is an empty list. The code asked whether any row matched our title — strictly broader. A latest
attempt that published anything else (an ordinary ::warning from a step in the same job lands on
the same check run) walked back too and adopted an older attempt's set, under a note asserting the
newer attempt had carried none.
That note is false in exactly the way #4491's own sentence was
false, and the ceiling it publishes is stale — which is the single outcome --passed-on-main exists
to prevent.

2 · The walk back had no bound (SocialMedia#206.) Two API calls per attempt, run_attempt
unbounded: twelve runs each re-run a dozen times is 288 extra calls, spent by construction during
the incident that caused the re-runs. Bounded at four — like every other walk this module makes —
and a truncation is now said rather than passed off as an exhausted search.

3 · The run listing escaped as a stack trace (SocialMedia#186.) Every per-run read already
turned a ResolutionError into a skip with a named reason. The listing — the first call the
option makes — did not, so a GitHub failure left main() as an uncaught traceback: no ::error, no
step summary, and nothing saying whether main had passed nothing or GitHub had simply not answered.
Those have opposite remedies. The option whose entire purpose is to fail with a named RED verdict
failed with a stack trace.

4 · The refusal contradicted its own note (Crm#128.) #4493 split the two per-run notes and left
the aggregate counting present-but-unparseable notices as silent, under a sentence saying those
runs "published no notice" — three lines below a note correctly saying one of them had. The operator
was sent to fix main when the fault was one malformed message.

5 · The registry error blamed a credential it could not implicate (Manufacturing#82,
SocialMedia#181.) A dropped connection was retried three times while a 429 or 503 from the same
registry — the transient answer a registry actually gives under load — was wrapped and raised on the
first try, taking the CI critical path red on precisely the condition the surrounding retry
structure exists for. And whatever came back, the message read "A 401/403 is the registry
credential (ACR_USERNAME / ACR_PASSWORD)"
, so a reader handed a 503 went and rotated a credential
that was never involved. During an incident that is the most expensive kind of wrong an error
message can be, because it is the kind that gets acted on.

Also documents tester-name / portal-name / migration-name / lag, written to
$GITHUB_OUTPUT but absent from the output contract (SocialMedia#181).

Controls

15 new self-test cases, 140 → 155. Every one was run against the unfixed file first and failed
there
:

control on the unfixed file
non-empty-but-untitled latest attempt ceiling 8203, expected None — it resurrected the older set
…and claimed a fallback happened emitted read from attempt 1
a run re-run past the bound ceiling 8203, expected None — walked all the way back
an unreadable run listing escaped as ResolutionError, not a refusal
the refusal separates unparseable from silent said 1 DID carry that job and published no notice over a note saying the annotation was PRESENT
503 / 429 from the token endpoint calls=1 — died on the first answer
503 from the manifest HEAD calls=2 — same
a 500 must not blame the credential message contained ACR_USERNAME

The 401, 403 and manifest-404 cases passed before and after — they are the regression control
proving the new retry did not swallow a verdict or turn an absent tag into an error.

🚨 This REDS every satellite on merge — the re-copy is part of this work

I first claimed the opposite in this PR and in 25 replies, and had to correct all of them. Recording
why, because the wrong reading is the natural one:

node-repo-validate.yml declares platform-ref with default: main, and scripts-ref falls
back to it. Every satellite's validate: job passes no inputs, so the canonical is fetched at
core main, live — and check-resolver-copy.py is hard-red (RED_FROM 2026-09-15). The
platform-ref: literals in a satellite's ci.yml pin other jobs (compile-check, tag-modules,
the pack lanes); reading one of those as the validate lane's input is what produced the wrong
conclusion. The job log prints SCRIPTS_REF: main — that is the line to read.

Measured today: core #4773 merged 08:02:23Z, and by 08:16Z every satellite's
validate / Validate node repos — a required context in all of them — was failing with
32 code line(s) differ (76 raw). So merging this needs the re-copy in the same sitting:

gh api repos/Systemorph/MeshWeaver/contents/.github/scripts/resolve-platform.py --jq .content \
  | base64 -d > scripts/resolve-platform.py

A re-run does not rescue an open pull request — the guard reads the branch's copy — so open
branches need git merge origin/main after the re-copy lands on each satellite's main.

🚨 And the red masks 16 other guards — filed as #4784

Worse than the red: the drift check sits mid-job, so its failure skipped 16 later steps of
the same job (MeshWeaver.SocialMedia#210, job 105867277548) — including Every PR-reachable secret in this repo is asserted by a preflight, Every manifest.lock is current, Every module's version matches its content, the duplicate-key guard and the pin-comment guard. A skipped required context
counts as satisfied, so while a satellite's copy is behind, every pull request in it is unguarded
by all of those, and the only symptom is one red about an unrelated file. A skip-trapdoor made by
step ordering, not by a continue-on-error: or an if: — nothing in the workflow looks wrong on
inspection. Filed rather than fixed here: it changes the failure semantics of a required context in
six repositories, and moving just this step to the end would fix the instance and leave the class.

Declined, with reasons on the threads

  • ambiguous in the main-first discriminator (Crm#128, SocialMedia#206, Manufacturing#102) — the
    wording those findings asked for already landed at fix(resolve-platform): the ceiling's refusal leads with the listing, not with "Fix main" (#4664) #4681 ("named no set this reader can use"), and
    keeping ambiguous in the selector is deliberate and documented: a run that carried the
    reporting job and failed to name one usable set is evidence about main, not about the listing.
  • The Ellipsis sentinel (Manufacturing#82) — None is a real value there, so the state is
    genuinely three-valued; the suggested boolean pair is two variables that can disagree.
  • Making PLUGINS_LOOKBACK configurable (Manufacturing#82) — the bound is documented in the
    API-call budget and reported when it is spent without finding a publication.

Filed rather than half-done

The --verify-source + SHA-freeze ordering (SocialMedia#189, Manufacturing#90) is real and still
live
: freeze_names_this_run compares the run's head_sha while, under --verify-source, the
authoritative sha is the receipt's source-sha read later — so a frozen run whose plugins seal is
still pending can be passed over and the freeze silently not honoured. It needs a design decision
about where attribution happens, not a patch; filed separately and named on both threads.

🤖 Generated with Claude Code

Every one of these was raised by the automatic reviewer on a change that had
already merged, and never answered — the `Automatic review answered` context is
required only on this repository, so in the six others a finding against shipped
code is structurally never read (core #4776). Five of the satellites vendor this
file, so the same defect was reported up to three times over.

THE FALLBACK IS LICENSED BY AN EMPTY RECORD, NOT AN UNRECOGNISED ONE
(MeshWeaver.Education#352, MeshWeaver.Crm#128, MeshWeaver.Reinsurance#221 — one
defect, three copies.) #4491 permits reading an older attempt when a partial
re-run re-creates a job record carrying none of the annotations the attempt that
ran it published. The guard asked instead whether any row matched our TITLE,
which is strictly broader: a latest attempt that published anything else — an
ordinary `::warning` from a step in the same job lands on the same check run —
walked back too, and an older attempt's set was adopted under a note asserting
the newer attempt carried none. That note is false in exactly the way #4491's own
sentence was false, and the ceiling it publishes is STALE, which is the one
outcome `--passed-on-main` exists to prevent. The walk now stops at any non-empty
answer; whether that answer names a set is the next branch's question, and it
already had two distinct sentences for it (#4493).

THE WALK BACK IS BOUNDED (MeshWeaver.SocialMedia#206.) Each attempt consulted
costs a jobs read and an annotations read, and `run_attempt` has no ceiling, so
twelve runs re-run a dozen times is 288 extra calls — spent, by construction,
during the incident that caused the re-runs. Bounded at four, like every other
walk this module makes, and a truncation is SAID rather than passed off as an
exhausted search.

THE RUN LISTING IS READ INSIDE THE REFUSAL (MeshWeaver.SocialMedia#186.) Every
per-run read already turned a ResolutionError into a skip with a named reason.
The listing — the FIRST call the option makes — did not, so a GitHub failure left
main() as an uncaught traceback: no ::error, no step summary, and nothing saying
whether main had passed nothing or GitHub had not answered. Those have opposite
remedies.

PRESENT-BUT-UNPARSEABLE IS COUNTED APART FROM SILENT (MeshWeaver.Crm#128.) #4493
split the two per-run NOTES and left the aggregate counting both as `silent`,
under a sentence saying those runs published no notice — three lines below a note
correctly saying one of them had. The operator was sent to fix main when the
fault was one malformed message.

THE REGISTRY GETS THE TREATMENT GITHUB ALREADY GOT (MeshWeaver.Manufacturing#82,
MeshWeaver.SocialMedia#181.) A dropped connection was retried three times while a
429 or 503 from the same registry — the transient answer a registry gives under
load — was wrapped and raised on the first try, taking the CI critical path red
on a condition the surrounding retry structure exists for. And whatever came
back, the message said `A 401/403 is the registry credential`, sending a reader
of a 503 to rotate a credential that was never implicated. During an incident
that is the most expensive kind of wrong an error message can be, because it is
acted on. Transient statuses are retried and bounded; the message is told by the
status, and says so when the status tells it nothing.

Also documents `tester-name` / `portal-name` / `migration-name` / `lag`, which
were written to $GITHUB_OUTPUT but absent from the output contract
(MeshWeaver.SocialMedia#181).

CONTROLS. 15 new self-test cases, 140 -> 155. Every one of them was run against
the unfixed file first and FAILED there — the four ceiling cases resurrecting
8203 or escaping as a ResolutionError, the refusal case reproducing the exact
contradiction ("1 DID carry that job and published no notice" over a note saying
the annotation was PRESENT), and five registry cases dying on call 1 with the
credential message. The 401, 403 and manifest-404 cases passed before and after:
they are the regression control that the retry did not swallow a verdict.

NOT A FLEET-WIDE RED. `node-repo-validate.yml` fetches the canonical at the
CALLER's `scripts-ref`/`platform-ref`, and every satellite pins that to a fixed
core sha — so `check-resolver-copy.py` compares each copy against the canonical
as of its own pin, and this change reds nobody on merge. Each vendored copy picks
it up when that repository next moves its pin, which is where the re-copy belongs;
re-copying now would make five copies drift AHEAD of their own pinned canonical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Two new diagnostics still contain inaccurate retry or attempt-history statements.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 Medium severity

Open (2)
What changed in this PR

Fixes five platform-resolver defects involving stale attempts, bounded lookups, error handling, refusal diagnostics, and registry retries.

Changes:

  • Corrects attempt fallback and bounds historical lookups.
  • Improves refusal messages and transient registry handling.
  • Adds resolver self-tests and a What's New entry.
File Description
.github/​scripts/​resolve-platform.py Updates resolver logic, diagnostics, output documentation, and self-tests.
src/​MeshWeaver.Documentation/​Data/​WhatsNew/​2026-09-19-the-resolver-that-picks-a-platform-answered-from-an-older-attempt.md Documents the user-visible resolver fixes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/scripts/resolve-platform.py Outdated
Comment on lines +1043 to +1046
f"main run {run_id}: attempts {latest_attempt}..{floor_attempt} all carry an "
f"EMPTY annotation list — the walk back stopped after "
f"{CEILING_ATTEMPTS_WALKED} attempt(s) and this run is SKIPPED rather than "
f"read from an attempt {CEILING_ATTEMPTS_WALKED} re-runs old")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and it is the same class of defect this PR is about — fixing it.

Traced it: an intermediate attempt with no matching job takes the continue at the top of the loop
without incrementing searched, so it contributes nothing to the evidence. My note then asserts
attempts {latest}..{floor} all carry an EMPTY annotation list over a range that may include
attempts which carried no job at all. That is a sentence stating more than was read — which is
precisely what the other five fixes in this PR are about, so it would be a poor thing to leave in
the fix for them.

Taken as you scope it: the note now counts the attempts that actually returned empty
annotations, using searched, and says the walk was bounded without claiming anything about the
attempts it never looked into.

Comment thread .github/scripts/resolve-platform.py Outdated
Comment on lines +1215 to +1216
return (f"HTTP {code} is TRANSIENT and was retried 3 times before this. The registry is "
"refusing or rate-limiting right now; the credential is not implicated. Re-run.")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both halves correct — fixing both.

The count. for attempt in range(3) with if ... and attempt < 2: means attempts 0 and 1 retry
and attempt 2 raises: three attempts, two retries. "retried 3 times before this" overstates it by
one, and in a message whose whole purpose is to stop a reader chasing the wrong cause, an inflated
retry count invites the opposite error — concluding the registry is harder down than it is. Now
stated as two retries across three attempts.

408. Right, and the distinction is worth keeping: a 408 is the request timing out, not the
registry declining to serve it. Lumping it under "refusing or rate-limiting" would be a smaller
version of the fault this change fixes — one sentence asserted over statuses that do not share a
cause. 408 now gets its own wording (the request did not complete in time), 429 says rate-limited,
and 5xx says the registry answered with a server error; all three still say they were retried and
are safe to re-run.

It stays in REGISTRY_TRANSIENT — a timeout is exactly the kind of thing a retry is for — so only
the wording changes, not which statuses are retried.

rbuergi and others added 3 commits September 19, 2026 10:06
main's 0403be0 replaced the single `per_page={limit}` listing call with a read that pages until
`limit` runs SURVIVE the event filter, and 0f19745 added the distinction between a page budget
running out and a listing genuinely ending. Both touch the exact lines this branch guarded.

Resolved in main's favour and the guard moved into its loop, where it composes better than it did
standing alone: main's `exhausted` flag already separates "the read stopped early" from "this is
main's state", which is the same distinction a failed page needs. So page 1 failing is a refusal
naming the LISTING (nothing was read), and a LATER page failing is a note leaving `exhausted` False,
falling through to main's own "not an exhaustion condition" sentence with the rows already in hand.

Three self-test cases added for the new branch, and the branch was deliberately broken
(`if page == 1:` -> `if False:`) to prove they fail: the page-1 case reports `nothing said 'run
LISTING'`. Restored, 162 cases pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both findings were correct, and both are the defect class this PR is about — a sentence asserting
more than the code measured — so leaving them in the fix for it would have been poor.

THE TRUNCATION NOTE COUNTS WHAT IT READ. An intermediate attempt carrying no `Resolve the released
platform` job takes the `continue` WITHOUT touching `searched`, so `attempts N..M all carry an EMPTY
annotation list` could assert emptiness for attempts that were never looked into. It now counts the
attempts that actually RETURNED an empty list, and says nothing about the rest.

THE REGISTRY MESSAGE STATES THE RETRIES THE LOOP MAKES, AND STOPS LUMPING THREE CAUSES TOGETHER.
`range(3)` retrying while `attempt < 2` is two retries over three attempts, not three; an inflated
count invites the opposite wrong conclusion (that the registry is harder down than it is). And a 408
is the request not completing, not the registry declining to serve it — so 408 (timeout), 429 (rate
limiting) and 5xx (server error) each say their own cause. Which statuses are RETRIED is unchanged;
a timeout is exactly what a retry is for.

CONTROLS. 5 new cases, 162 -> 167. Both wordings were reverted to their pre-review form and all five
fail there — the two ceiling cases on `nothing said 'that carried a ... job each returned an EMPTY
annotation list'` and on `said 'all carry an EMPTY annotation list', which must not be said here`,
and the three registry cases on the retry count and on 408/429 wearing the 5xx sentence. Restored,
167 pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…et measures that

AGENTS.md: a finding produced while working gets committed, every time — an investigation that lives
only in a pull request body is invisible to the next session. This is the 2026-09-19 merged-PR sweep,
recorded in the page that already owns the subject.

WHAT WAS MEASURED. The 30 most recently updated merged pull requests in each of eight repositories:
240 merged, 325 findings, 182 never answered, across 93 pull requests. Core is 0 of 59 — the check
working, and also the control that makes the other rows readable, because the same instrument finds
both states. Everywhere else the usual ratio is N of N: where a pull request carried findings at all,
none was answered. Not occasionally missed — structurally never read, because nothing asks.

ALSO RECORDED, because both cost time to establish:

· The sweep PROCEDURE, with the bound stated. ~5,700 merged pull requests fleet-wide means every
  sweep is partial, so the denominator swept and the remainder belong in the report or the next
  session cannot tell a treated repository from an untreated one. A finding counts as answered only
  when some comment's in_reply_to_id is that root's id; a PR-level comment answers nothing.

· 🚨 FIX THE CANONICAL, AND DO NOT RE-COPY YET. Findings cluster on vendored files because the
  reviewer reads each copy independently — 25 of the 182 were five satellites' copies of
  resolve-platform.py and collapsed to five defects in one canonical. But node-repo-validate.yml
  fetches the canonical at the CALLER's pinned scripts-ref, so each copy is compared against the
  canonical as of its own pin: a canonical fix reds nobody, and each copy picks it up when that
  repository next moves its pin. Re-copying first would put five copies AHEAD of their own pinned
  canonical, and a drift guard reports DISTANCE, NOT DIRECTION — so it would red exactly as if the
  copies were stale.

Controls: Documentation.Test builds 0 Error(s) under -c Release -warnaserror, and 6 tests pass
including DocumentationLinkIntegrityTest — which caught the sibling link written as
`RenamingARequiredCheck.md` instead of `../RenamingARequiredCheck` before this was committed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n just landed

main's cc445ec/15d5c384d2 landed a section on exactly this subject from the parallel 13-finding
sweep, with the per-repo protection mechanism and the gen-manifests collapse. Theirs is the better
record of WHY the gap exists and is kept whole; this merge keeps their section and nests only what
this sweep adds and theirs does not have:

· THE WIDER DENOMINATOR. Their 120 merged over six repos extended to 240 over eight — 325 findings,
  182 unanswered, 93 pull requests — which adds Memex (the worst of the eight at 38 unanswered, and
  absent from the earlier sample) and the per-repo finding counts. Plus the 45 unanswered findings on
  24 open DRAFTS, deliberately untreated.

· THE SWEEP PROCEDURE, with the bound-stating rule and the `sort=updated` caveat (an old pull request
  that got a comment recently enters the window, which is why the spans differ per repo).

· 🚨 FIX THE CANONICAL, AND DO NOT RE-COPY IN THE SAME BREATH. Theirs covers porting the GATE as a
  workflow_call lane; this is the different question of treating findings against a vendored SCRIPT.
  node-repo-validate.yml fetches the canonical at the CALLER's pinned scripts-ref, so a canonical fix
  reds nobody and each copy picks it up when that repo next moves its pin — and re-copying first puts
  five copies AHEAD of their own pinned canonical, which a distance-not-direction drift guard reds
  exactly as if they were stale.

No duplication: their tables, their rollout-order paragraph and their gen-manifests account are
untouched, and the "What it does not see" bullet is theirs.

Controls: resolve-platform self-test 167 cases pass after the merge; Documentation.Test builds
0 Error(s) under -c Release -warnaserror and 6 tests pass, DocumentationLinkIntegrityTest included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

265 tests  ±0   265 ✅ ±0   2m 20s ⏱️ -30s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 1ae1214. ± Comparison against base commit ac22f6b.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

436 tests  ±0   436 ✅ ±0   53s ⏱️ ±0s
  3 suites ±0     0 💤 ±0 
  3 files   ±0     0 ❌ ±0 

Results for commit 1ae1214. ± Comparison against base commit ac22f6b.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

1 553 tests  +2   1 553 ✅ +2   3m 7s ⏱️ +7s
    2 suites ±0       0 💤 ±0 
    2 files   ±0       0 ❌ ±0 

Results for commit 1ae1214. ± Comparison against base commit ac22f6b.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

710 tests  ±0   518 ✅ ±0   5m 34s ⏱️ ±0s
  3 suites ±0   192 💤 ±0 
  3 files   ±0     0 ❌ ±0 

Results for commit 1ae1214. ± Comparison against base commit ac22f6b.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

    3 files  ±0      3 suites  ±0   5m 44s ⏱️ +10s
1 857 tests ±0  1 857 ✅ ±0  0 💤 ±0  0 ❌ ±0 
1 858 runs  ±0  1 858 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 1ae1214. ± Comparison against base commit ac22f6b.

♻️ This comment has been updated with latest results.

rbuergi and others added 4 commits September 19, 2026 10:32
I had this exactly backwards in the section this branch added, and the wrong version nearly merged.

WHAT I CLAIMED. That `node-repo-validate.yml` fetches the canonical at each satellite's own pinned
`platform-ref`, so a canonical fix is invisible to them until that pin moves — and that re-copying
first would put five copies AHEAD of their pinned canonical and red them.

HOW I GOT IT WRONG. I grepped a `platform-ref:` literal out of a satellite's ci.yml and read it as
the validate lane's input. Those literals pin OTHER jobs (compile-check, tag-modules, the pack
lanes). The `validate:` job passes NO inputs, and the lane declares `platform-ref` with
`default: main`, `scripts-ref` falling back to it. The canonical is fetched at core `main`, LIVE. The
guard's own docstring says so — "a canonical fetched at @main is live on merge for every caller"
(#4027) — and I read that sentence and reasoned past it.

WHAT IS TRUE, MEASURED. Core #4773 merged 08:02:23Z; by 08:16Z every satellite's
`validate / Validate node repos` — a REQUIRED context in all of them — was red with
`32 code line(s) differ (76 raw), RED since 2026-09-15`. The job log prints `SCRIPTS_REF: main`;
that is the line to read, and a literal elsewhere in ci.yml is not evidence about this job. So the
re-copy belongs in the same sitting as the canonical fix, not at some later pin move.

AND THE PART THAT IS WORSE THAN THE RED. The drift check sits MID-JOB, so its failure skipped 16
later steps of the same job — measured on MeshWeaver.SocialMedia#210 job 105867277548 — including
`Every PR-reachable secret in this repo is asserted by a preflight`, `Every manifest.lock is
current`, `Every module's version matches its content`, the duplicate-key guard and the pin-comment
guard. A skipped required context counts as SATISFIED, so while a satellite's copy is behind, every
pull request in it is unguarded by all of those and the only symptom is one red about an unrelated
file. That is a skip-trapdoor made by STEP ORDERING rather than by a `continue-on-error:` or an `if:`
— nothing in the workflow looks wrong on inspection. Filed as #4784 rather than fixed here: it
changes the failure semantics of a required context in six repositories, and moving just this one
step to the end would fix the instance and leave the class.

The 25 threads that carried the wrong footer have each had this correction posted on them.

Controls: Documentation.Test builds 0 Error(s) under -c Release -warnaserror; 6 tests pass,
DocumentationLinkIntegrityTest included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…pass

`gh api ... -f body=@reply.md` sends the LITERAL string `@reply.md` as the comment body, and the POST
returns a normal comment id with a 201. So it reads as a successful reply; the root acquires a comment
whose `in_reply_to_id` points at it; and EVERY detector built on that field counts the finding as
answered — the sweep query in this page, and `check-review-answered.py`'s own predicate.

Measured 2026-09-19: of 208 replies posted during one sweep, 74 were these stubs (38 Memex, 23 Crm,
13 Manufacturing), each a `@`-prefixed filename or absolute path. Three of the four sessions that hit
it reported thread counts as proof they had replied — the proof being the one field that cannot tell
the two apart. The fourth caught it only by reading a reply back.

So the verification is READ THE BODY BACK AND CHECK ITS LENGTH, never the reply's existence; a length
near 20 is the bug; and the repair is PATCH on the reply, never a second reply, which would leave the
stub standing beside the real answer.

This is the sweep's own instance of the class it exists to find — an answer that reads like a pass —
so it belongs in the page that owns the subject rather than in a session's terminal.

Controls: Documentation.Test builds 0 Error(s) under -c Release -warnaserror; 6 tests pass,
DocumentationLinkIntegrityTest included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y backlog figure

Two corrections to the paragraph added a few commits ago, both from peer measurement.

THE COUNT WAS 100, NOT 74 — and the gap is structural, not arithmetic. My audit enumerated the
review threads on the SWEPT pull requests and checked their replies. The flag is a property of the
CALL SITE, so it also hits PR-level issue comments (`issues/comments`, a different endpoint) and
replies on the sweep's OWN pull requests, neither of which a thread-centric audit can reach. Memex 41,
Crm 24, Reinsurance 20, Manufacturing 15. Re-audited by call site — every comment authored today on
both endpoints matching `^@` — which found the remaining 26 and now reads 0 of 495.

A STUB INFLATES EVERY "HOW MUCH IS LEFT" COUNT. A stubbed thread HAS a reply, so the sweep query in
this page calls the finding treated — which makes any backlog figure published while stubs are
outstanding a floor, by an unknowable margin. Re-measure after repairing, and say whether a reported
count was taken before or after. Two of this sweep's per-repo denominators were re-taken for this.

Also: a length check alone is not the verification. `@r_115_4028258629.md` is 20 characters, which is
SHORT, not obviously wrong. Assert a content signature you know is in the file — the verdict string
the reply opens with.

Controls: Documentation.Test builds 0 Error(s) under -c Release -warnaserror; 6 tests pass,
DocumentationLinkIntegrityTest included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Peer correction on the `-f`/`-F` paragraph, and the generalisation is better than what it replaces.

Reasoning about WHICH FLAG is the wrong level: `-f` treats `@path` as a literal string, and `-F`
type-coerces a value that merely looks numeric or boolean, so the pair is silent in BOTH directions
and `-F` is right here only because the value happens to be `@file`. The response is not evidence
either — it carries an id and a 201 whatever went in. So the rule is now the one that holds whatever
the flags do: ANY `gh api` write whose payload came from a file is verified by READING THE FIELD BACK.

And the comparison is not a length check. 20 characters is SHORT, not wrong, and a genuinely terse
reply would fail the same test; the check is a content signature known to be in the file, or a
byte-compare allowing for the trailing newline GitHub appends.

On authorship: the earlier commit blamed the rules file, and that was accurate — its example line read
`-f body=@<file>` verbatim. Four sessions followed it. Recording that here so the next reader does not
conclude the flag is obscure knowledge somebody should have had.

Controls: Documentation.Test builds 0 Error(s) under -c Release -warnaserror; 6 tests pass,
DocumentationLinkIntegrityTest included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

    5 files  ± 0      5 suites  ±0   12m 42s ⏱️ -7s
3 678 tests +17  3 676 ✅ +17  2 💤 ±0  0 ❌ ±0 
3 682 runs  +17  3 680 ✅ +17  2 💤 ±0  0 ❌ ±0 

Results for commit 1ae1214. ± Comparison against base commit ac22f6b.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results

   17 files  ± 0     17 suites  ±0   30m 23s ⏱️ -21s
8 499 tests +19  8 305 ✅ +19  194 💤 ±0  0 ❌ ±0 
8 504 runs  +19  8 310 ✅ +19  194 💤 ±0  0 ❌ ±0 

Results for commit 1ae1214. ± Comparison against base commit ac22f6b.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants