feat(ci): opt-in coverage ratchet (default OFF — closes PMAT build-system audit gap #1) - #37
Merged
Merged
Conversation
…stem audit gap #1) The strongest quality signal (line coverage) was purely advisory: the coverage job runs `cargo llvm-cov` but never enforced a floor, and the codecov upload is continue-on-error. A coverage regression therefore merged SILENTLY despite the 95% / ZERO-tolerance rule. This adds two OPT-IN inputs to the fleet-shared reusable workflow: - coverage_min (default '') — line-coverage floor - coverage_baseline_file (default .pmat/coverage-baseline.txt) — ratchet baseline path When coverage_min is EMPTY (the default for all 72 non-pilot callers), a new `if: inputs.coverage_min != ''` guard SKIPS the enforcement step entirely — coverage stays exactly as advisory as before. Zero behavior change for every repo that does not opt in. When coverage_min is set, the new step derives line coverage from the just-produced lcov.info (no extra compile/test cost), computes an effective floor = max(coverage_min, committed baseline), and FAILS the coverage job on a drop below it. The coverage job result is already wired into the top-level `gate` (needs: [...coverage...]), so a regression now blocks merge. It is a RATCHET, not an absolute jump: a repo green today records its current % as the baseline, so enabling the gate can never break the currently-green state. Pilot: aprender only (paiml/aprender PR). NOT enabled fleet-wide. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gap
PMAT build-system audit gap #1: the strongest quality signal (line coverage) is purely advisory in the fleet-shared
sovereign-ci.yml. Thecoveragejob runscargo llvm-covbut never enforces a floor, and the codecov upload iscontinue-on-error. A coverage regression therefore merges silently, despite the documented "95% coverage / ZERO tolerance" rule.Change
Two opt-in inputs added to the reusable workflow:
coverage_min''(empty)coverage_baseline_file.pmat/coverage-baseline.txtA new enforcement step in the
coveragejob:lcov.info(sum of hitDA:records ÷ total) — no extra compile/test cost.max(coverage_min, committed-baseline).coveragejob result is already wired into the top-levelgate(needs: [...coverage...]), so a regression now blocks merge instead of merging silently.It is a ratchet, not an absolute jump: a repo green today records its current % as the baseline, so enabling the gate can never break the currently-green state.
Safety — zero behavior change for every non-pilot repo
The enforcement step is guarded by
if: ${{ inputs.coverage_min != '' }}. With the empty default, the step is skipped entirely — coverage stays exactly as advisory as today. Verified: of 73 fleet callers ofsovereign-ci.yml, only aprender (the pilot, separate PR) setscoverage_min. The other 72 see no change.Rollout ordering
This PR must merge before the aprender pilot PR — aprender's
@mainreference to the newcoverage_mininput is only valid once this lands.🤖 Generated with Claude Code