Skip to content

Commit 2433941

Browse files
committed
docs(t0): the acceptance leaf is the cell, not a median over workloads
A cross-artifact conflict the Phase-3 checks did not look for: T0 judged `D = median_c(d_c)` over a gate's whole workload population, while the frozen instrument requires a preregistered decision per `(phase, workload_id, platform, regime)` cell and then rolls up `cell -> workload_class -> phase -> overall_g3`. Internal consistency was never the problem; agreement with the accepted D7 gate was. The population median did two wrong things at once. It **hid a casualty** — one decisive workload degraded past the fail margin vanishes behind eleven healthy ones — and it **double-weighted an alias**: `large-solution-control` is `alias_of` `oss-ShareX.sln`, the same path at the same pin, declared an alias precisely so the pair is never counted twice in a denominator. The instrument already resolves aliases and refuses a duplicate canonical identity by name; T0 was the artifact out of step. So the leaf is the cell. Per cell: `relative = median(Rust)/median(Python) - 1`, `absolute = median(Rust) - median(Python)`, judged by the unchanged R5 rule against its family's unchanged R6/R7 budgets — no budget moves in this commit. The eight stratum x regime x resource combinations become **gate classes**, and the roll-up is the same operator twice: cell verdicts to a class verdict, class verdicts to overall. One failing workload fails its class however many pass. `INVALID` is not a roll-up verdict at either level; admissibility is settled before a roll-up exists. Population is now counted in canonical identities rather than manifest entries — 13 decisive entries resolve to 12 canonical identities today, read from the frozen artifacts rather than asserted — and a duplicate canonical identity joins the invalidation list. Recorded so nobody has to infer it: the time classes are the `end-to-end-csharp` phase; memory is not a phase at all but each cell's `rss_policy`; the other seven D7 phases carry an explicit `not_applicable` with a reason, which is how a diagnostic surface stays non-gating while D7 completeness is still satisfied. Nothing here asks `perf_baseline.py` to change, so the harness digest does not move. Checked by enumeration rather than by reading: a class over 12 cells is total and single-valued across all 531441 combinations, the overall roll-up over 8 class verdicts across all 6561, verdicts are invariant under 20000 random reorderings of the workloads, one FAIL makes its class FAIL, one NO_DECISION makes it NO_DECISION, and `expected_d7_cells()` contains the 12 canonical ids with the alias absent. Status unchanged: NOT_FROZEN, collection_authorized false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
1 parent 5bf47fd commit 2433941

1 file changed

Lines changed: 151 additions & 50 deletions

File tree

docs/notes/p022-263-t0-protocol-freeze.md

Lines changed: 151 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,8 @@ ratified four-part cell identity of §2.1 with `engine` projected out and the
4848
stratum named, so the pair exists exactly where both engines have a timed cell on
4949
**one** platform.
5050

51-
**Pairing never crosses a stratum.** `median_c` is computed only within one
52-
primary gate's stratum, and no `R` is ever formed over Linux and Windows cells
51+
**Pairing never crosses a stratum.** A cell comparison is computed inside one
52+
stratum, and no comparison of any kind is formed over Linux and Windows cells
5353
together. A Linux cell and a Windows cell that agree on rung, workload and regime
5454
are two different cells, because since #355 their memory quantities are not even
5555
the same physical thing:
@@ -60,18 +60,40 @@ windows / launcher-e2e / W / warm Python vs Rust a different pair
6060
across those two no pair, and no R
6161
```
6262

63-
Every later use of *cell*, *matched pair*, *the gate's cell set*, `median_c` and
64-
`R` inherits this key. There is no second term for it — a "platform-cell" would
63+
Every later use of *cell*, *matched pair* and *a class's cell set* inherits this
64+
key. There is no second term for it — a "platform-cell" would
6565
be the same idea with a second name, and two names for one thing is how a
6666
contract starts disagreeing with itself.
6767

68-
For each matched pair:
68+
**The cell is also the acceptance leaf.** For each matched pair:
6969

70-
d_c = log(median(Rust_c)) - log(median(Python_c))
71-
D = median_c(d_c)
72-
R = exp(D)
70+
P_c = median(Python_c)
71+
R_c = median(Rust_c)
7372

74-
`R` is the reported Rust-vs-Python ratio: `R < 1` means Rust is faster.
73+
relative_regression_c = R_c / P_c - 1
74+
absolute_regression_c = R_c - P_c
75+
76+
and the rule of T0-5 is applied to **that cell**, on its own, against its
77+
family's budgets. A cell verdict is `PASS`, `FAIL` or `NO_DECISION`; the
78+
roll-ups of T0-3 then combine cell verdicts, never cell numbers.
79+
80+
**Withdrawn: the population-level statistic.** An earlier revision computed
81+
`d_c = log(median(Rust_c)) − log(median(Python_c))`, `D = median_c(d_c)` and
82+
`R = exp(D)` across the whole workload population of a gate, and judged *that*.
83+
It is removed from the decision path for two reasons, both of which the frozen
84+
instrument already rules on:
85+
86+
- a median over workloads **hides a casualty**. One decisive workload degraded
87+
far past the fail margin disappears behind eleven healthy ones, and the gate
88+
reports a comfortable ratio for a product that got materially worse on a real
89+
input;
90+
- it **double-weights an alias**. `large-solution-control` is `alias_of`
91+
`oss-ShareX.sln` — the same path at the same pin — so a population median over
92+
manifest entries gives that one solution two votes.
93+
94+
Neither is repaired by choosing a different average. The leaf has to be the cell,
95+
because that is the level at which the accepted D7 gate requires a preregistered
96+
decision, and no aggregate may stand in for it.
7597

7698
**Index pairing is forbidden.** `Rust_i / Python_i` for the i-th iteration is not
7799
a paired observation here and may not be computed, reported or gated on.
@@ -95,13 +117,12 @@ instrumentation evidence, not a T0 detail.
95117
**An unpaired cell cannot reach the statistic.** A cell whose invocation did not
96118
do the rung's work is not timed at all and the collection is refused
97119
(instrument §2). Therefore a missing side is an admissibility failure — see
98-
T0-4 — and never a pair dropped quietly from `median_c`.
120+
T0-4 — and never a pair dropped quietly from a class's cell set.
99121

100122
**Recorded consequence.** Cell-level pairing cancels *condition* noise (same
101123
stratum, rung, workload, regime, same session, interleaved order). It does not
102-
cancel
103-
per-iteration noise, because the data cannot support that claim. No stronger
104-
noise-cancellation property may be asserted for `R`.
124+
cancel per-iteration noise, because the data cannot support that claim, and no
125+
stronger noise-cancellation property may be asserted for a cell comparison.
105126

106127
---
107128

@@ -113,24 +134,24 @@ Two coordinates, never collapsed into a score:
113134

114135
| term | definition |
115136
|---|---|
116-
| `relative_regression` | `R - 1` for the gate's cell set, `R` per T0-1 |
117-
| `absolute_regression` | the median over matched pairs of `median(Rust_c) - median(Python_c)`, in the gate's own unit |
137+
| `relative_regression_c` | `median(Rust_c) / median(Python_c) - 1`, for one cell |
138+
| `absolute_regression_c` | `median(Rust_c) - median(Python_c)`, for one cell, in its unit |
118139
| unit, time gates | milliseconds — the instrument records `perf_counter_ns`, so the conversion belongs to the reading, never to the evidence |
119140
| unit, memory gates | bytes of the stratum's own `memory_metric`, after the instrument's unit normalisation — resident on `linux`, committed on `windows`, never mixed |
120141

121142
Two margins, because neither alone is meaningful: a relative-only rule fails a
122143
`0.20 ms -> 0.24 ms` change that no user can perceive, and an absolute-only rule
123144
is blind to scale.
124145

125-
Each gate carries **two pairs** of budgets — one pair that admits a pass, one
126-
pair that compels a failure — and the decision rule over them is T0-5:
146+
Each cell is judged against **two pairs** of budgets — one pair that admits a
147+
pass, one pair that compels a failure — and the decision rule over them is T0-5:
127148

128149
Each budget family below applies to **both regimes** of its resource, on the
129-
stratum named. The pass pair and the fail pair belong to the same gate: no regime
130-
owns one of them.
150+
stratum named, and to every canonical workload cell inside those classes. The
151+
pass pair and the fail pair belong to the same cell: no regime owns one of them.
131152

132153
```yaml
133-
# R6 — elapsed time. Both strata, both regimes: four of the eight gates, one family.
154+
# R6 — elapsed time. Both strata, both regimes: four of the eight classes, one family.
134155
time:
135156
M_pass: 0.05 # +5 %
136157
A_pass: 50 # milliseconds
@@ -157,7 +178,7 @@ agree today because both were chosen from the same product budget before any dat
157178
existed, and either may later move without the other. Reading the coincidence as
158179
one cross-platform metric is exactly the error #355 removed from the instrument.
159180
160-
Required of every gate, and true of all three budget families above:
181+
Required of every cell, and true of all three budget families above:
161182
162183
M_pass < M_fail
163184
A_pass < A_fail
@@ -276,9 +297,27 @@ owner-ratified amendment, not as a silent passenger inside a preregistration.
276297

277298
The gates are computed over **exactly the decisive workloads of the D7-bound
278299
workload manifest that are applicable to the `launcher-e2e` rung**, and over
279-
nothing else. At the currently frozen manifest that is all **13** decisive
280-
workloads — every one of them is a source tree, and the launcher rung admits
281-
every one.
300+
nothing else — as **canonical identities**, which is not the same as manifest
301+
entries:
302+
303+
```text
304+
manifest decisive entries at the frozen manifest 13
305+
canonical decisive identities 12
306+
large-solution-control is alias_of oss-ShareX.sln
307+
```
308+
309+
**An alias earns no second vote.** `large-solution-control` and
310+
`oss-ShareX.sln` are the same path at the same pin; the manifest declares the
311+
alias precisely so the pair is never counted twice in a denominator. The
312+
instrument resolves an alias to the identity it aliases and refuses a duplicate
313+
canonical identity by name. A population counted in manifest entries would give
314+
that one solution two votes, so the population is counted in canonical
315+
identities and this document says "12 canonical identities at the current
316+
manifest", never "13 voting workloads".
317+
318+
The number is read from the frozen artifacts, not asserted here: it is whatever
319+
the manifest's decisive entries resolve to under the instrument's alias rule, and
320+
it is 12 today.
282321

283322
**Calibration workloads never enter a cutover gate.** They exist to size the
284323
instrument; a gate computed over them would be answering a different question
@@ -290,23 +329,30 @@ say which cells a gate covers without reading a function, and two readers must
290329
not be able to answer differently.
291330

292331
Missing cells are not a smaller denominator. Each of these makes the attempt
293-
`INVALID`:
332+
`INVALID`, and the semantics match the accepted D7 verifier rather than being
333+
restated loosely here:
294334

295335
```text
296-
a decisive workload absent from the gate set => INVALID
297-
a cell present for one engine and not the other => INVALID
298-
an unexpected extra decisive cell => INVALID
299-
a calibration cell inside the gate set => INVALID
336+
a canonical decisive identity absent from the gate set => INVALID
337+
a cell present for one engine and not the other => INVALID
338+
an unexpected extra decisive cell => INVALID
339+
a duplicate canonical identity (an alias counted twice) => INVALID
340+
a calibration cell inside the gate set => INVALID
300341
```
301342

302343
D7 binds the exact workload-manifest digest, and therefore the exact decisive
303344
set: the population cannot be re-read later as "whatever was measured".
304345

305-
### The eight gates
346+
### The eight primary gate classes
306347

307-
Since #355 the memory quantity is platform-local, so the gate identity carries
308-
its stratum. The primary set is exactly **eight** independent gates, four per
309-
stratum:
348+
Since #355 the memory quantity is platform-local, so the identity carries its
349+
stratum. The primary set is exactly **eight gate classes**, four per stratum.
350+
351+
They are **classes, not leaves**: each one covers the canonical decisive
352+
workloads, and the acceptance decision happens per workload cell inside it
353+
(T0-1). Calling them eight gates was the previous revision's error — it implied
354+
one verdict per class computed from an aggregate, which is exactly the masking
355+
this section now forbids.
310356

311357
```text
312358
linux / launcher-e2e / process-cold / elapsed (ms)
@@ -320,18 +366,34 @@ windows / launcher-e2e / process-cold / max_process_peak_commit (bytes)
320366
windows / launcher-e2e / warm / max_process_peak_commit (bytes)
321367
```
322368

323-
**No compensation in any direction**: not between `process-cold` and `warm`, not
324-
between time and memory, and not between the two strata. `U_linux` and
325-
`U_windows` are never pooled, and no ratio is formed across them.
369+
**No compensation in any direction**: not workload against workload, not
370+
`process-cold` against `warm`, not time against memory, and not Linux against
371+
Windows. `U_linux` and `U_windows` are never pooled, and no ratio is formed
372+
across them.
326373

327-
The roll-up over the eight is the automaton of T0-4, applied without addition:
374+
### Two roll-ups, one operator
328375

329376
```text
330-
any gate FAIL => FAIL
331-
else any NO_DECISION => NO_DECISION
332-
else => PASS
377+
inside a class, over canonical workload cell verdicts:
378+
any FAIL => class FAIL
379+
else any NO_DECISION => class NO_DECISION
380+
else => class PASS
381+
382+
over the eight class verdicts:
383+
any FAIL => overall FAIL
384+
else any NO_DECISION => overall NO_DECISION
385+
else => overall PASS
333386
```
334387

388+
**One failing workload fails its class**, even when every other workload in it
389+
passes. That is the property the withdrawn population median destroyed, and it is
390+
the reason the leaf is the cell.
391+
392+
`INVALID` is **not** a roll-up verdict and never appears at either level: an
393+
admissibility failure invalidates the attempt under T0-4 and T0-6 *before* any
394+
roll-up is computed. A session that reached a roll-up is a session whose evidence
395+
was already complete.
396+
335397
**Memory is gated on `launcher-e2e` only.** The instrument captures
336398
`peak_memory_bytes` on every measured cell and those values are **published as
337399
diagnostic evidence**, but a memory number on a diagnostic rung never gates.
@@ -342,6 +404,38 @@ here rather than left to a reader.
342404
Every other surface in the table is **published, never gating**. A derived view
343405
may inform a reading and may never serve as a gate.
344406

407+
### How this lands in the accepted D7 payload
408+
409+
The frozen instrument already fixes the shape D7 must fill, and this contract is
410+
written to fit it rather than asking it to move:
411+
412+
```text
413+
D7_CELL_DIMENSIONS = (phase, workload_id, platform, regime)
414+
D7_ROLLUP_LEVELS = (workload_class, phase, overall_g3)
415+
cell universe = 8 phases x 12 canonical workloads x 2 platforms x 2 regimes
416+
= 384 cells, each needing a rule or an explicit not_applicable
417+
```
418+
419+
The mapping, stated so nobody has to infer it:
420+
421+
- the **time** classes are the `end-to-end-csharp` phase, per platform and
422+
regime — the user-visible path, and the only phase this contract gates;
423+
- the **memory** classes are not a phase. Memory is decided on the same cells
424+
through the cell's `rss_policy` key, which is why this document never asks D7
425+
for a memory phase it does not have;
426+
- the other seven phases carry an explicit `not_applicable` with the reason
427+
"published as diagnostic evidence; not a cutover gate under T0-3" — under R4 a
428+
diagnostic surface never becomes a veto, and D7 completeness is satisfied by a
429+
recorded decision rather than by silence;
430+
- the per-cell rule keys are filled by this contract: `pass_fail_rule` and
431+
`inconclusive_band` by T0-5, `bound` by the T0-2 families, `comparison_statistic`
432+
by T0-1's per-cell ratio and difference, `repetition_ladder` by `N`,
433+
`rss_policy` by the stratum's memory metric, `allocation_policy` by R2's
434+
diagnostic-only ruling.
435+
436+
Nothing here requires a change to `scripts/perf_baseline.py`, and therefore
437+
nothing here moves the harness digest.
438+
345439
---
346440

347441
## T0-4 — Decision automaton
@@ -366,12 +460,13 @@ eligibility (T0-7: host qualified, every required primary metric has a
366460
│ └─ budget exhausted ─► NO_DECISION
367461
└─ clean
368462
369-
each of the eight primary gates, by the two-dimensional rule of
370-
T0-5 over the T0-2 budgets
463+
every canonical workload cell, by the two-dimensional rule of
464+
T0-5 over its family's T0-2 budgets, then rolled up inside its
465+
class and over the eight classes (T0-3)
371466
372-
├─ any gate FAIL ────────────► FAIL ─► NO_GO
373-
├─ any gate NO_DECISION ─────► NO_DECISION
374-
└─ all eight gates PASS ──────► PASS ─► GO
467+
├─ any class FAIL ────────────► FAIL ─► NO_GO
468+
├─ any class NO_DECISION ─────► NO_DECISION
469+
└─ all eight classes PASS ────► PASS ─► GO
375470
```
376471

377472
The three null-metric situations are **different states**, and collapsing them
@@ -401,8 +496,8 @@ median elapsed > 0
401496
median memory > 0 (the stratum's own metric)
402497
```
403498

404-
and for the derived quantities `R`, `relative_regression` and
405-
`absolute_regression`, any of
499+
and for each cell's derived quantities `relative_regression_c` and
500+
`absolute_regression_c`, any of
406501

407502
```text
408503
None · NaN · +inf · -inf · undefined arithmetic
@@ -455,7 +550,8 @@ rather than in a reading nobody re-reads.
455550
**RESOLVED (R5): a deterministic gray zone in two dimensions.**
456551

457552
The gray zone is defined on the same two coordinates the margins are, and no
458-
synthetic scalar score is constructed from them. Per gate:
553+
synthetic scalar score is constructed from them. Per **cell**, against the
554+
budget family its class belongs to:
459555

460556
```text
461557
PASS iff relative_regression <= M_pass
@@ -537,7 +633,7 @@ Invalidation fires only on machine-detectable predicates frozen in advance:
537633
- candidate byte drift within a stratum after collection started.
538634

539635
**A performance result is never an invalidation condition.** Not a slow cell, not
540-
a gray-zone outcome, not a disappointing `R`. Every predicate above is
636+
a gray-zone outcome, not a disappointing cell comparison. Every predicate above is
541637
machine-detected, so no operator chooses to invalidate a session.
542638

543639
One retry attempt is one full re-collection of the invalidated session on the
@@ -841,14 +937,19 @@ Content completeness is not the freeze; it is what makes the freeze reviewable.
841937
| can a host with no mechanism for a required metric start a session? | **no** — T0-7 eligibility, T0-4 case A |
842938
| can a stale manifest carry a campaign? | **no** — fresh per session, rechecked after, drift is `INVALID` |
843939
| can anyone start collecting because hosts and binding are ready? | **no** — T0-0 revoked that; `collection_authorized: false` |
844-
| can one evidence set yield both PASS and FAIL under two admissible readings? | **no** — per gate the two conditions are mutually exclusive by `M_pass < M_fail` and `A_pass < A_fail`, and the roll-up is a total function of the eight gate outcomes |
940+
| can one evidence set yield both PASS and FAIL under two admissible readings? | **no** — per cell the two conditions are mutually exclusive by `M_pass < M_fail` and `A_pass < A_fail`, and both roll-ups are total functions of the verdicts beneath them |
845941
| does the declaration of single tenancy masquerade as proof? | **no** — provisioning evidence and runtime invariants are separated, and only the latter is called machine-verified |
846942
| can a calibration workload enter a cutover gate? | **no** — T0-3 owns the population: decisive workloads applicable to `launcher-e2e`, and a calibration cell in the set is `INVALID` |
847943
| can a missing decisive workload quietly shrink the denominator? | **no** — a missing workload, a missing engine side or an unexpected extra cell each make the attempt `INVALID` |
848944
| can a cell with fewer than `N` samples still produce a median? | **no** — a primary cell holds exactly `N`, every sample finite and positive; one `null` invalidates the session rather than yielding a median of the survivors |
849945
| can an undefined quantity reach a verdict through the rule's `OR`? | **no** — the numeric domain is checked before T0-5; `NaN` on either coordinate is `INVALID`, never `PASS` |
850946
| can Linux resident bytes be compared with Windows committed bytes? | **no** — the metric is part of the gate identity, the strata are never pooled, and `resident == commit` is asserted nowhere |
851947
| can a replacement host buy another attempt? | **no** — the retry budget belongs to the stratum, replacement does not reset it, and a valid outcome closes the stratum |
948+
| can one degraded workload hide behind the others? | **no** — the leaf is the cell; one `FAIL` fails its class however many workloads pass, and the population median that allowed it is withdrawn by name |
949+
| can an alias vote twice? | **no** — the population is counted in canonical identities, an alias resolves to what it aliases, and a duplicate canonical identity is `INVALID` |
950+
| can a workload be dropped to improve a class? | **no** — a missing canonical identity is `INVALID`, not a smaller denominator |
951+
| does reordering the workloads change a verdict? | **no** — the roll-up operator is order-independent by construction, and it is checked that way |
952+
| does T0 ask D7 for a payload it cannot express? | **no** — time maps to the `end-to-end-csharp` phase, memory to each cell's `rss_policy`, and the seven non-gating phases carry an explicit `not_applicable` with a reason |
852953
| can a valid result be re-measured on a new host? | **no** — re-measurement needs a previous `INVALID` **and** remaining budget; a new binding starts a campaign beside the old one and erases nothing |
853954

854955
No "yes" answer remains that a decision could close. The one permitted item — a

0 commit comments

Comments
 (0)