Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,40 @@

## Unreleased

- **No corpus case is graded against `insufficient_evidence` any more, and four
`blocked` cells hold one case instead of two.** (#520, #508) A verdict exists
to route a change somewhere: `passed` merges, `review_required` hands a human
a named capability, `blocked` stops. `insufficient_evidence` routes nowhere —
it is what the gate says when *its own* extraction failed, which is a
statement about shipgate rather than about the change. So it cannot be the
answer to "what should a correct gate do here?", and both named release
policies stop demanding cases that expect it: `beta` goes from 28 strata /
100 cases to 21 / 80, `pre_1_0` from 28 / 56 to 21 / 38. The value stays in
the enum, the verifier still emits it, and it is scored as a miss against
whatever the case expected — a coverage failure, which is what it is.

The first blind labelling round is what settled it. All four cases where both
independent raters chose `insufficient_evidence` had named a capability the
diff introduced, so they could have decided; one slot sourced as
`insufficient_evidence` was placed at `blocked` by both raters; and 12 of the
15 slots for the outcome had been sourced from the engine's own verdict.

The same round measured a second thing: of the seven profiles, only four
produce a real-world `blocked` case at all, and each produces exactly one. So
`openai_agents_sdk`, `langchain_crewai`, `google_adk` and
`coding_agent_trust_roots` now ask for one `blocked` case rather than two.
Filling those cells to two is reachable today only by building four more
constructions, and a cell filled with constructions measures our imagination
rather than the world. **No rate moved.** Every exact-match floor is still
production's rate applied to the population it governs, rounded up —
`minimum_blocked_exact` falls to 10 because there are 10 `blocked` cases, at
the same 100% demand — and the origin floor stays 40% of the corpus, which is
why holding it at a fixed *count* was refused: that would have raised
production's demand to half the corpus as a side effect of deleting a
decision. `docs/release-evidence-policy-decision.md` § Amendment 3 records
the ruling, and `benchmark/safety-qualification/strata-inventory.md` records
the per-cell evidence for the scarcity.

- **A preview wheel now reports one version, not two.** (#491) The first
published preview stamped `pyproject.toml` and not
`src/agents_shipgate/__init__.py`, so its METADATA said
Expand Down
38 changes: 36 additions & 2 deletions STABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,16 +211,50 @@ cannot parse. Both release gates enforce the pairing — the standard-library
sealer on raw JSON, since it never parses the envelope otherwise. Nothing emits
v4 any more, and v3 is still not read, as before.

**What this is for.** `0.x` tags are now governed by a named 56-case `pre_1_0`
**What this is for.** `0.x` tags are now governed by a named 38-case `pre_1_0`
policy, decided under
[#341](https://github.com/ThreeMoonsLab/agents-shipgate/issues/341) and recorded
in [`docs/release-evidence-policy-decision.md`](docs/release-evidence-policy-decision.md).
It reduces evidence *coverage* only: the zero-unsafe-auto-pass rule, per-case
receipts, the holdout fraction, the κ floor and `static_only` are unchanged, and
every exact-match floor is the production rate rounded up. `1.0` and later still
require the 100-case `beta` artifact, and there is no promotion shortcut.
require the 80-case `beta` artifact, and there is no promotion shortcut.
<a id="migration-note-unreleased-setup-error-envelope"></a>

## Migration Note: 0.16.0 — no corpus case targets `insufficient_evidence`

**No schema version moves.** `shipgate.safety_qualification` stays at v5, the
corpus and receipt-index envelopes are unchanged, and no field is added,
renamed, or removed. `insufficient_evidence` remains in
`ReleaseDecisionStatus`, the verifier still emits it, and
`minimum_insufficient_evidence_exact` remains a required field of the
`requirements` block.

**What changes is the contents of the two named policies.** No stratum targets
`insufficient_evidence` any more, so both tiers lose seven cells: `beta` goes
28 cells / 100 cases → 21 / 80, and `pre_1_0` 28 / 56 → 21 / 38. Four
`pre_1_0` `blocked` cells hold one case rather than two, because no second real
case exists for them. The floors that count those cases follow:
`minimum_insufficient_evidence_exact` becomes `0` in both tiers,
`minimum_blocked_exact` becomes `10` for `pre_1_0`, and
`minimum_qualified_origins` becomes `32` / `16` — the same 40% share of a
smaller corpus, not a smaller share.

**No rate moved**, which is the property to check if you are reading this to
find out whether the bar got easier. Every exact-match floor is still
production's rate applied to the population it governs, rounded up, and
`test_the_pre_1_0_policy_is_never_laxer_than_production_per_rate` fails if a
floor is one case laxer than that.

**Who is affected.** Nobody consuming a published artifact: none exists yet at
either tier. A qualification artifact built against the previous shape is
rejected by both release gates — with `case profile/outcome strata do not match
the <tier> policy` and a case-count error — and must be rebuilt. The reasoning
is recorded in
[`docs/release-evidence-policy-decision.md`](docs/release-evidence-policy-decision.md)
§ Amendment 3, under
[#520](https://github.com/ThreeMoonsLab/agents-shipgate/issues/520).

## Migration Note: 0.16.0 — the setup control envelope reaches both streams

`contract_version` moves **26 → 27**. `minimum_control_contract_version` stays
Expand Down
Loading
Loading