Guard every already-covered file with a repo-wide coverage ratchet#4385
Guard every already-covered file with a repo-wide coverage ratchet#4385TaprootFreak wants to merge 11 commits into
Conversation
The only coverage gate so far was the Frick one: seven files, 0.4% of production code. Nothing stopped a change from silently dropping coverage anywhere else. Measure the whole suite once and pin every production file that already reaches 100% on branches, functions, lines and statements - 401 of 1593 files. Dropping below 100 on any of them now fails CI. This is a regression gate, not a claim about test quality. Overall coverage is 57.7% of statements and 39.5% of branches; the pinned set is simply what is complete today. Of the 401, 218 carry real logic and 183 are declarative (modules, constants) - kept as separate blocks in the config so the count is not mistaken for test depth. The declarative ones are pinned on purpose: adding logic to one of them now requires tests. Two properties of the measurement matter and are documented so the list stays reproducible: - Full compilation via tsconfig.coverage.json, as the Frick gate already does. Transpile-only emit reports phantom uncovered branches on injected constructors, which would understate dozens of files. - The whole suite runs, because files are frequently covered by specs other than their own. The Frick gate stays untouched and separate: running only its seven specs, it proves those specs alone reach 100%, which a whole-suite run cannot assert. Verified locally: the gate passes with no threshold violation (299/306 suites, 5170 tests), and parallel execution yields byte-identical coverage totals to a --runInBand reference run, which is why the CI script does not serialise. docs/coverage-gate.md records the measured gaps, including the 29 files already at >=90% on all four metrics as the next batch.
Two files under shared/utils are test support, not production code: test.util.ts (60 importers) and test.shared.module.ts (28), every one of them a *.spec.ts. Both were collected and pinned, which made two things wrong. It coupled a production gate to test scaffolding: extending a test helper without testing the helper itself would have turned CI red although no production coverage moved. And it made the documented inventory inaccurate, since both were counted as production files. Exclude them from collectCoverageFrom and drop their threshold entries. Pinned files 401 -> 399 (217 with logic, 182 declarative). Correct the remaining numbers against a fresh verified run: 1,591 measured files, statements 57.66%, branches 39.46%, functions 31.84%. The collection glob matches 1,643 files; the 52 that never appear in the report are type-only files that compile to no executable statements, so they cannot be pinned. Also document what happens when a pinned file is deleted or renamed - jest exits 1 with "Coverage data ... was not found", so the gate cannot silently stop protecting a file - and fix the stale "three job groups" comment in the workflow, which this change turns into four.
Two documentation defects, both about claims a reader cannot check.
The inventory said all 52 uncollected files are type-only. Two of them are not:
p2b.service.ts and payin/enums/index.ts consist entirely of commented-out code.
The counts and the conclusion were right, but lumping dead code in with type
declarations hides a second, different reason a file cannot be measured. Both
are pre-existing and left untouched; removing them is separate cleanup.
The parallelism claim rested on a one-off measurement ("a --runInBand run
produced identical totals"), which nothing in the repo lets a maintainer
reproduce. Replace it with the structural reason it holds: istanbul merges
per-worker counters additively, so scheduling cannot turn a covered statement
into an uncovered one. The measurement stays as corroboration, not as the
argument. Likewise drop the precise "~16 min" from the workflow comment and
date the approximation in the doc, so neither goes stale silently.
Dating a measurement records when someone asserted it, not whether it holds. The doc still carried two claims a maintainer cannot check from the repository: that a serial reference run once produced identical totals, and that this job is the slowest of the four at roughly a quarter hour. Drop both. What remains is derivable from the repository itself: coverage merging is additive across workers, so parallelism cannot lower a per-file percentage, and the gate runs the whole suite under full compilation while the test job shards three ways and the Frick gate runs seven specs. That explains the runtime without asserting a number nobody can reproduce. Measured values stay in the pull request, where they belong - a PR reports what its author ran, with the CI run alongside it. Durable documentation is read when nobody remembers who measured what.
The test named "should execute in parallel" measured wall-clock duration and required it to stay under 100 ms while five mocked calls each slept 10 ms. That assertion answered the wrong question in both directions. It was too strict under load: scheduling delay alone pushed it to 181 ms on a busy machine, failing a test whose subject was correct. It was also too weak to catch what it existed for. Sequential execution of five 10 ms calls totals about 50 ms, comfortably under the limit. Verified by replacing Promise.all in BitcoinBasedFeeService.getTxFeeRates with a sequential loop: the old test stayed green, 27 passed. It could only ever fail because of machine speed, never because of the defect it was written to detect. Assert the property itself instead. The mock now counts how many calls are in flight simultaneously and the test requires that maximum to equal the number of txids. Concurrent execution reaches five; a sequential implementation never exceeds one. Against the same sequential mutation the new test fails with "Expected: 5, Received: 1", so it detects the regression the old one missed, and it does not depend on timing at all. Coverage is unchanged: the pinned BitcoinBasedFeeService still reports 100% on all four metrics under the full gate run.
…e claim CONTRIBUTING requires the `// --- NAME --- //` form for major sections; the two principal blocks of the gate config used a different decorative style. The doc also claimed the pinned list "was derived from a run without" Postgres. That is a statement about a past measurement, and nothing checked in records it. State the current, checkable behaviour instead: no pinned file belongs to the migration suites MIGRATION_TEST_PG enables, and enabling further suites can only raise coverage.
The same unverifiable statement removed from docs/coverage-gate.md survived in the workflow: "matching the measurement the pinned list was derived from". Nothing checked in records that historical environment, and the workflow and the documentation should not make different-strength claims about the same fact. State only what a reader can check: no pinned file belongs to the migration suites that need Postgres.
|
6 review passes to zero findings, two independent lenses each round (conformance against CONTRIBUTING, and logic/correctness), re-run in full against What the rounds turned up:
The round-4 finding was the substantive one. The Mechanism verified rather than assumed, against jest 29.7's
All 12 checks green, including the new ratchet job at ~15 min. |
The threshold block spelled out the same four-metric object for each of the 399 pinned files, which made the config 2037 lines and turned every rename into a five-line diff. The paths now live in PINNED_LOGIC and PINNED_DECLARATIVE and coverageThreshold is built from them, so the two groups are data rather than a comment, and adding a file is one line. The pinned set itself is unchanged: same 399 paths, same order, same grouping. Also emit an lcov report, which the workflow uploads when the gate fails.
The job had no timeout and would have held a runner for the default six hours if it hung. It is also the most expensive job in the workflow: an unsharded whole-suite run under full compilation. When the gate fails, jest names the file and the metric but not the uncovered lines, which left a local rerun of roughly fifteen minutes as the only way to find out what to fix. The lcov report is now uploaded as an artifact instead. Also run the gate with --silent, matching the test script. An unsilenced whole-suite run buries the threshold message under the console output of every suite.
The assertion sits between the point where the mocked getMempoolEntry calls are parked and the point where they are released. On failure the release never ran, leaving five pending promises and a getTxFeeRates call that could not settle. Moving the release into a finally block keeps the failure clean.
Three gaps a reader would otherwise find the hard way: - A newly added production file with no coverage passes the gate. The list only protects what is already on it, and it grows by hand. - When a purely declarative file first gains executable logic, the function metric moves from 0/0 to 0/N and the gate turns red. Tests stay the preferred fix, but unpinning that one file is an allowed outcome when the pull request says so. For the logic-carrying files the rule is unchanged. - The lcov report is uploaded as an artifact when the gate fails. Also describe the two pinned-path arrays, since that is where a file is added now.
|
Four review findings addressed, plus three follow-ups split out as issues. Fixed here
Verification
Follow-ups, tracked separately
|
Why
The repo has exactly one coverage gate today:
jest.frick.config.js, covering 7 files — 0.4 % of production code. Everywhere else, a change can drop coverage to zero and CI stays green.What this does
Measures the whole suite once and pins every production file that already reaches 100 % on all four metrics. A drop below 100 on any pinned file now fails CI.
This freezes coverage that already exists — no test was added to raise it.
One existing test is rewritten (see below), which means the pre-existing
checksand shardedtestjobs run a changed test too. This PR is therefore not configuration-only.What it deliberately does not claim
It is a regression gate, not a quality statement. Overall coverage is 57.66 % of statements and 39.46 % of branches — the pinned set is simply what is complete today.
Of the 399 files, 217 carry real logic (functions and/or branches) and 182 are purely declarative (NestJS modules, constant files). They are kept as two commented blocks in the config so the number is not mistaken for test depth. Pinning the declarative ones is intentional: the moment logic is added to one, it must arrive with tests.
Measurement
Two properties matter, and
docs/coverage-gate.mdrecords both so the list stays reproducible:tsconfig.coverage.json,isolatedModules: false) — same as the Frick gate. Transpile-only emit reports phantom uncovered branches on injected constructors and would understate dozens of files.The gate job runs without the Postgres service the sharded
testjob uses, matching the measurement. The suites it enables are not in the pinned list, and running them can only raise coverage.The Frick gate stays separate and untouched
It runs only its seven specs and therefore proves those specs alone reach 100 % — an assertion a whole-suite run cannot make, since there any spec may contribute the coverage. Two gates, two different questions.
Verification
--runInBandreference run — hence no--runInBandin the CI scriptthreshold ... not met, and a deleted or renamed pinned file exits 1 withCoverage data ... was not found. The gate cannot silently stop protecting a file.format:checkgreen, workflow YAML parses, config loads with 399 keysInventory
The collection glob matches 1,643 files under
src/; 1,591 carry instrumentable code and appear in the report. The remaining 52 are type-only files that compile to no executable statements and therefore cannot be pinned. Test scaffolding (shared/utils/test.util.ts,shared/utils/test.shared.module.ts— 60 and 28 importers, all specs) is excluded, so an untested change to a test helper cannot fail a production gate.One test changed, and why it belongs here
bitcoin-fee.service.spec.tsasserted parallel execution by wall-clock time (duration < 100msfor five mocked 10 ms calls). This gate runs the whole suite a second time per PR, which doubles the exposure to that timing assertion — a flaky required check is the worst kind, because it teaches a team to ignore red.Checking it turned up more than flakiness. Replacing
Promise.allingetTxFeeRateswith a sequential loop left the old test green (27 passed): five sequential 10 ms calls total ~50 ms, under the limit. The assertion could only fail from machine load, never from the defect it existed to catch — it failed at 181 ms under load in exactly that way.The test now counts how many calls are in flight at once and requires that maximum to equal the txid count. Against the same sequential mutation it fails with
Expected: 5, Received: 1. Stable and, for the first time, effective. Coverage of the pinnedBitcoinBasedFeeServiceis unchanged at 100 %.Scope boundary
1062 partially covered and 130 uncovered files stay open — closing them means writing tests, which is deliberately not part of this change.
docs/coverage-gate.mdlists the gaps by area and names the 29 files already at ≥90 % on all four metrics as the next batch, so the boundary is documented rather than silent.