From a8f79502d5b11a3237b959578c4ff81541afc535 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 16:16:18 +0000 Subject: [PATCH 01/14] =?UTF-8?q?test(swang):=20SWG-4A-06=20red=20?= =?UTF-8?q?=E2=80=94=20level=20and=20root=20dispatch,=20and=20the=20level-?= =?UTF-8?q?2=20skeleton?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The acceptance contract for the level-2 entry path, written before it exists. Spec §5.4 requires a build to route to one level's entry point and never branch inside a shared grammar, "because such a grammar has no way to prove that level 1's behaviour survived the addition of level 2". These tests are that proof obligation. RED, for the intended reasons: swang/tests/level_two_dispatch.rs E0432: unresolved imports `format_document`, `parse_document`, `Document` — the dispatched public path does not exist yet. swang/src/syntax/tests.rs E0433: could not find `v2` in `parser`, and `couldn't read swang/src/syntax/parser/v2/lexer.rs` — the level-2 lexer, and with it the retained-token representation, does not exist yet. The matrix pins, beyond the happy path: an unknown newer level still refused by the frozen pre-parser and now naming `1..=2`; a malformed or truncated header still `SWG0002`; a level-2 `pattern` root refused (§5.7); a level-1 `score` root keeping the *identical* frozen refusal, compared diagnostic-for-diagnostic against the frozen entry point (Law A's invalid-body half, §5.5); malformed root prefixes; trailing and concatenated material; the grammatical `score` words this slice does not implement failing closed rather than being skipped; and the two registry refusals reachable from the one scalar it reads. The three inherited SWG-INF-06 obligations are tests, not prose. The source-byte breach pads a *grammatically valid* minimal score with whitespace, so the source would be accepted if the byte check were not consulted first — which is what makes it a witness that no successful `swang 2` result precedes budget wiring, rather than a witness that a large string fails. The token breach spells four million tokens plus one under the byte cap. Both run the declared limits through the public path, with no scaling and no test-only reconstruction of the parser. The `wasm32` token-storage proof asserts the retained token fits twelve bytes, that its text is sliced from the source span rather than owned, and that the binding assertion is the compile-time one — a host-only runtime check would prove x86_64 and call it a platform proof. `level_two_budget_boundary.rs` gains the two level-2 modules on its EXEMPT list, one at a time as its own comment requires. `syntax.rs` and the dispatcher deliberately stay off it: a budget consulted before the level branch is a level-1 bound whatever file it lives in. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- fuzz/fuzz_targets/swang_parse.rs | 45 ++- swang/src/syntax/tests.rs | 116 ++++++- swang/tests/level_two_budget_boundary.rs | 13 +- swang/tests/level_two_dispatch.rs | 407 +++++++++++++++++++++++ 4 files changed, 576 insertions(+), 5 deletions(-) create mode 100644 swang/tests/level_two_dispatch.rs diff --git a/fuzz/fuzz_targets/swang_parse.rs b/fuzz/fuzz_targets/swang_parse.rs index b0306e3..0a5b846 100644 --- a/fuzz/fuzz_targets/swang_parse.rs +++ b/fuzz/fuzz_targets/swang_parse.rs @@ -15,8 +15,22 @@ //! `SWG\d{4}` and a span inside the source (`start <= end <= len`). //! * On `Ok`: `format` emits canonical text that reparses to the same AST //! (law 3) and is its own fixed point (law 2). +//! * `parse_document`: the dispatched path — level 1 or level 2 — is +//! `Ok(Document)` xor a non-empty `Vec`, every diagnostic +//! carries a registry code and an in-source span, and `format_document` +//! obeys the same two laws whichever level answered. +//! +//! SWG-INF-06 declared level 2's input bounds but could not honestly claim +//! an end-to-end breach oracle: public parsing could not reach a level-2 +//! parser at all, so the oracle would have covered a path the binary could +//! not enter. SWG-4A-06 is the task that opens it, so the oracle lands +//! here. A budget breach is a typed `SWG0509` reaching the caller — not an +//! abort, not an allocation death, not a silent success. -use griff_swang::syntax::{format, header_level, parse, Diagnostic, LANGUAGE_LEVEL}; +use griff_swang::syntax::{ + format, format_document, header_level, parse, parse_document, Diagnostic, Document, + LANGUAGE_LEVEL, +}; use libfuzzer_sys::fuzz_target; /// The one diagnostic contract, applied to the header pre-parser and the @@ -74,4 +88,33 @@ fuzz_target!(|source: &str| { } } } + + match parse_document(source) { + Ok(document) => { + let canonical = format_document(&document); + let reparsed = parse_document(&canonical) + .unwrap_or_else(|d| panic!("canonical text must reparse (law 2): {d:?}")); + assert_eq!( + format_document(&reparsed), + canonical, + "format_document is its own fixed point (law 2), at either level" + ); + match (&document, &reparsed) { + (Document::Pattern(a), Document::Pattern(b)) => { + assert_eq!(a, b, "parse(format(ast)) == ast (law 3)"); + } + (Document::Score(_), Document::Score(_)) => {} + _ => panic!("the canonical text of a document keeps its root"), + } + } + Err(diagnostics) => { + assert!( + !diagnostics.is_empty(), + "a dispatched refusal names at least one diagnostic" + ); + for d in &diagnostics { + assert_diagnostic(d, len); + } + } + } }); diff --git a/swang/src/syntax/tests.rs b/swang/src/syntax/tests.rs index a58fd28..12c879f 100644 --- a/swang/src/syntax/tests.rs +++ b/swang/src/syntax/tests.rs @@ -109,7 +109,10 @@ fn first_error(source: &str) -> Diagnostic { #[test] fn the_frozen_header_form_pins_the_level() { assert_eq!(header_level("swang 1\nrest").expect("frozen form"), 1); - assert_eq!(LANGUAGE_LEVEL, 1, "this build parses level 1"); + // The frozen form is frozen; the range this build supports is not, and + // SWG-4A-06 moved it. Level 1 still reads exactly as it did — that is + // the line above, and Law A's baseline is the fuller proof. + assert_eq!(LANGUAGE_LEVEL, 2, "SWG-4A-06 admits level 2"); } #[test] @@ -157,13 +160,21 @@ fn the_pre_parser_reads_at_most_64_bytes() { #[test] fn a_newer_level_is_swg0001_naming_the_supported_range() { - let d = header_level("swang 2\n").expect_err("newer than this build"); + // `swang 2` was this case's fixture until SWG-4A-06 made it supported. + // The refusal is not what changed — the range is — so the case moves up + // to the next unsupported level and now pins the range exactly, rather + // than pinning that the message contains the digit one. + let d = header_level("swang 3\n").expect_err("newer than this build"); assert_eq!(d.code, "SWG0001"); assert!( - d.message.contains('1'), + d.message.contains("1..=2"), "the message names the supported range: {}", d.message ); + assert_eq!( + header_level("swang 2\n").expect("level 2 is this build's"), + 2 + ); } #[test] @@ -897,3 +908,102 @@ mod level_two_budget { assert_eq!(budget.diagnostics(), 2, "the terminal one is counted"); } } + +/// SWG-4A-06's inherited SWG-INF-06 obligation: the level-2 token proves the +/// `MAX_TOKENS` heap derivation instead of asserting it in prose. +mod level_two_token_storage { + use std::mem::size_of; + + use crate::syntax::parser::v2::lexer::{lex_level_two, Level2Token, Level2TokenKind}; + use crate::syntax::span::span_of; + + /// The budget derivation in spec §5.11 is `<= 12` bytes per retained + /// token on `wasm32`. `Span` is two `u32` by the determinism law, so the + /// only way to exceed it is to start owning something. + const MAX_RETAINED_TOKEN_BYTES: usize = 12; + + #[test] + fn a_retained_level_two_token_fits_the_declared_budget() { + assert!( + size_of::() <= MAX_RETAINED_TOKEN_BYTES, + "a retained token is at most {MAX_RETAINED_TOKEN_BYTES} bytes; \ + this one is {}", + size_of::() + ); + } + + #[test] + fn the_size_bound_is_asserted_at_compile_time_so_it_travels_to_wasm32() { + // This test runs on the host only. The binding obligation is the + // `const` assertion in the lexer itself, which every target this + // crate builds for must satisfy — including the `wasm32` frontend + // CI builds. A host-only runtime check would prove the host and + // call it a platform proof. + let shipped = include_str!("parser/v2/lexer.rs"); + assert!( + shipped.contains("const _RETAINED_TOKEN_FITS_THE_BUDGET"), + "the compile-time assertion is what reaches wasm32; a runtime \ + test on x86_64 does not" + ); + } + + #[test] + fn level_two_token_text_is_recovered_from_the_source_span() { + // "Token text is recovered from the source span." Level 1's + // `String`-owning `Token` is not the level-2 storage representation, + // so the lexeme has to be sliced back out on demand. + let source = "swang 2\n\nscore {\n ppqn 960\n}\n"; + let tokens = lex_level_two(source, 9, &mut budget_for(source)).expect("lexes"); + let words: Vec<&str> = tokens + .iter() + .filter(|t| t.kind == Level2TokenKind::Word) + .map(|t| t.text_in(source)) + .collect(); + assert_eq!(words, ["score", "ppqn"]); + let numbers: Vec<&str> = tokens + .iter() + .filter(|t| t.kind == Level2TokenKind::Number) + .map(|t| t.text_in(source)) + .collect(); + assert_eq!(numbers, ["960"]); + } + + #[test] + fn every_level_two_token_carries_a_span_that_slices_its_own_lexeme() { + let source = "swang 2\n\nscore {\n ppqn 960\n}\n"; + let tokens = lex_level_two(source, 9, &mut budget_for(source)).expect("lexes"); + assert!(!tokens.is_empty()); + for token in &tokens { + let text = token.text_in(source); + assert!(!text.is_empty(), "a token spans at least one byte"); + let start = token.span.start as usize; + assert_eq!( + source.get(start..token.span.end as usize), + Some(text), + "the span is the lexeme, not an approximation of it" + ); + } + } + + #[test] + fn the_lexer_spends_the_token_budget_it_is_given() { + // Accounting attached to the real transition it constrains: one + // admitted token per token retained, on the budget the caller owns. + let source = "swang 2\n\nscore {\n ppqn 960\n}\n"; + let mut budget = budget_for(source); + let tokens = lex_level_two(source, 9, &mut budget).expect("lexes"); + assert_eq!( + budget.tokens(), + tokens.len() as u64, + "no token is retained without being admitted first" + ); + } + + fn budget_for(source: &str) -> crate::syntax::limits::Level2Budget { + let budget = crate::syntax::limits::Level2Budget::declared(); + budget + .admit_source(source, span_of(0, source.len())) + .expect("this fixture is far under the source cap"); + budget + } +} diff --git a/swang/tests/level_two_budget_boundary.rs b/swang/tests/level_two_budget_boundary.rs index cc34c83..6371092 100644 --- a/swang/tests/level_two_budget_boundary.rs +++ b/swang/tests/level_two_budget_boundary.rs @@ -340,7 +340,18 @@ fn every_shipped_module_but_the_budget_itself_is_scanned() { /// like), those go on this list explicitly, one at a time. Shared dispatch /// never joins it: a budget consulted before the level branch is a level-1 /// bound, whatever file it lives in. -const EXEMPT: &[&[&str]] = &[&["syntax", "limits.rs"], &["syntax", "tests.rs"]]; +const EXEMPT: &[&[&str]] = &[ + &["syntax", "limits.rs"], + &["syntax", "tests.rs"], + // SWG-4A-06's level-2 modules, added one at a time as the comment above + // requires. Both are level-2-only: the entry point that constructs the + // budget, and the lexer that spends its token allowance. Neither is on + // any level-1 path, and `syntax.rs` and the dispatcher stay off this + // list, because a budget consulted before the level branch is a level-1 + // bound whatever file it lives in. + &["syntax", "parser", "v2.rs"], + &["syntax", "parser", "v2", "lexer.rs"], +]; /// Whether a path relative to `swang/src` is one of the exempt modules. /// diff --git a/swang/tests/level_two_dispatch.rs b/swang/tests/level_two_dispatch.rs new file mode 100644 index 0000000..935a014 --- /dev/null +++ b/swang/tests/level_two_dispatch.rs @@ -0,0 +1,407 @@ +//! SWG-4A-06: level dispatch, root dispatch, and the level-2 skeleton. +//! +//! ```text +//! header_level -> level dispatch -> root dispatch (pattern | score) +//! ``` +//! +//! Spec §5.4 says a build "routes to one of them and never mixes them: +//! there is no single grammar with level-conditioned branches, because such +//! a grammar has no way to prove that level 1's behaviour survived the +//! addition of level 2". These tests are that proof obligation, written +//! down: every case here either pins where a source is routed, or pins that +//! a source which routes nowhere is refused rather than half-read. +//! +//! # The slice boundary +//! +//! This task accepts exactly one level-2 program shape — the minimal empty +//! score, `score { ppqn }`. Every other grammatical `score` word +//! (`master_bar`, `track`, `source`, `loss`) is real grammar owned by +//! SWG-4A-08 and is **refused here**, not tolerated: a parser that ignores +//! a word it does not implement is how exact text stops being exact. The +//! scalar layer is SWG-4A-07's; the two refusals this slice implements +//! (`SWG0505`, `SWG0506`) are the ones reachable from the single word it +//! reads, because the alternative is to accept `ppqn 0` — text §6.6 +//! declares invalid — into the accepted set of a level that has not frozen. + +// Reason: integration-test code. `unwrap`/`expect`/`panic` abort loudly with +// a clear message, which is exactly what a test harness wants. +#![allow( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::missing_assert_message +)] + +use griff_swang::syntax::{ + format, format_document, header_level, parse, parse_document, Diagnostic, Document, + LANGUAGE_LEVEL, +}; + +/// The one level-1 program every dispatch test measures level 1 against: +/// spec §3.1's reference program, byte for byte, so "level 1 is unchanged" +/// is measured against the text level 1 froze rather than a sketch of it. +const LEVEL_ONE: &str = r#"swang 1 + +pattern dgd_fractal { + ascii "X.X/XX./.XX" + |> fractalize depth 1 max_cells 4096 density 9500bps seed 4 + |> linearize snake + |> map_rhythm unit 1/16 tail rest_pad + |> generate { + source "corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5" + bars 8 + seed 42 + candidates 2 + strategy repeat_variation + corpus "corpus" + } + |> export midi "dgd_fractal_dense.mid" +} +"#; + +/// The same program under a level-2 header: a level-1 body is not level-2 +/// text, however well formed it is at its own level. +const LEVEL_ONE_BODY_UNDER_TWO: &str = r#"swang 2 + +pattern dgd_fractal { + ascii "X.X/XX./.XX" +} +"#; + +/// The minimal level-2 score: `ppqn` is the construct's only `1` word, and +/// `master_bar`/`track`/`source`/`loss` are omissible (§6.2, §6.4b). +const MINIMAL_SCORE: &str = "swang 2\n\nscore {\n ppqn 960\n}\n"; + +fn codes(diagnostics: &[Diagnostic]) -> Vec<&str> { + diagnostics.iter().map(|d| d.code).collect() +} + +fn refusal(source: &str) -> Vec { + parse_document(source).err().unwrap_or_else(|| { + panic!("this source must be refused, and it was accepted:\n{source}"); + }) +} + +// ── the level surface ─────────────────────────────────────────────────────── + +#[test] +fn this_build_supports_level_two() { + assert_eq!( + LANGUAGE_LEVEL, 2, + "SWG-4A-06 is the task that admits level 2" + ); + assert_eq!(header_level(MINIMAL_SCORE), Ok(2)); + assert_eq!(header_level(LEVEL_ONE), Ok(1)); +} + +#[test] +fn an_unknown_newer_level_is_still_refused_by_the_frozen_pre_parser() { + // §1.1: the pre-parser never changes across releases. What changes is + // the supported range it names, and it must name the range this build + // actually has. + let d = header_level("swang 3\n\nscore {\n}\n").expect_err("level 3 is not supported"); + assert_eq!(d.code, "SWG0001"); + assert!( + d.message.contains("1..=2"), + "SWG0001 reports the supported range: {}", + d.message + ); + assert_eq!(codes(&refusal("swang 3\n\nscore {\n}\n")), ["SWG0001"]); + assert_eq!( + codes(&refusal("swang 999999999\n\nscore {\n}\n")), + ["SWG0001"] + ); +} + +#[test] +fn a_malformed_or_truncated_header_is_still_swg0002() { + for source in [ + "swang\n\nscore {\n}\n", + "swang 2", + "swang 2\n", + "swang 02\n", + " swang 2\n", + "SWANG 2\n", + "swang 2x\n", + "", + ] { + assert_eq!( + codes(&refusal(source)), + ["SWG0002"], + "a malformed header is refused before any grammar sees it: {source:?}" + ); + } + assert_eq!(codes(&refusal("\u{feff}swang 2\n")), ["SWG0003"]); +} + +// ── level dispatch ────────────────────────────────────────────────────────── + +#[test] +fn a_level_one_source_routes_to_the_frozen_level_one_parser() { + let Ok(Document::Pattern(program)) = parse_document(LEVEL_ONE) else { + panic!("a valid `swang 1` source routes to the level-1 parser"); + }; + let direct = parse(LEVEL_ONE).expect("the frozen entry point still accepts it"); + assert_eq!(program, direct, "dispatch adds no reinterpretation"); +} + +#[test] +fn dispatching_a_level_one_source_formats_byte_for_byte_as_before() { + // Law A observable 3, through the new entry point. + let document = parse_document(LEVEL_ONE).expect("accepted"); + let program = parse(LEVEL_ONE).expect("accepted"); + assert_eq!(format_document(&document), format(&program)); +} + +#[test] +fn a_minimal_level_two_score_reaches_the_exact_parser() { + let Ok(Document::Score(_)) = parse_document(MINIMAL_SCORE) else { + panic!("`swang 2` with a `score` root routes to the level-2 parser"); + }; +} + +#[test] +fn the_level_one_entry_point_does_not_read_level_two() { + // §5.4: one build routes to one entry point and never mixes them. The + // frozen entry point must not half-read a level it does not own — and + // must never hand back a level-1 `Program` carrying level 2. + let diagnostics = parse(MINIMAL_SCORE).expect_err("the level-1 parser does not read level 2"); + assert_eq!(codes(&diagnostics), ["SWG0401"]); + let under_two = parse(LEVEL_ONE_BODY_UNDER_TWO) + .expect_err("not even a level-1 body under a level-2 header"); + assert_eq!(codes(&under_two), ["SWG0401"]); +} + +// ── root dispatch ─────────────────────────────────────────────────────────── + +#[test] +fn a_level_two_pattern_root_is_refused() { + // §5.7: "Level 2 does not admit the level-1 `pattern` root." + let diagnostics = refusal("swang 2\n\npattern riff {\n}\n"); + assert_eq!(codes(&diagnostics), ["SWG0401"]); + let first = diagnostics.first().expect("one diagnostic"); + assert!( + first.message.contains("score") && first.message.contains("pattern"), + "the refusal names the root it wanted and the root it found: {}", + first.message + ); + // And a real level-1 body under a level-2 header is refused too. Its + // string literal reaches the level-2 lexer before the root check does, + // so the message is the lexer's — the verdict is what this case pins, + // and the verdict is closed either way. + assert_eq!(codes(&refusal(LEVEL_ONE_BODY_UNDER_TWO)), ["SWG0401"]); +} + +#[test] +fn a_level_one_score_root_keeps_its_frozen_refusal() { + // Law A's invalid-body half (§5.5): a level-2 keyword in a `swang 1` + // script raises exactly what level 1 already raised — not a friendlier + // "`score` requires language level 2". + let diagnostics = refusal("swang 1\n\nscore {\n ppqn 960\n}\n"); + assert_eq!(codes(&diagnostics), ["SWG0401"]); + let through_frozen = + parse("swang 1\n\nscore {\n ppqn 960\n}\n").expect_err("level 1 refuses a `score` root"); + assert_eq!( + diagnostics, through_frozen, + "identical code, message, span and order — the frozen verdict is not \ + rewritten by the arrival of level 2" + ); +} + +#[test] +fn a_malformed_prefix_is_not_accepted_as_a_root() { + for source in [ + "swang 2\n\nscor {\n ppqn 960\n}\n", + "swang 2\n\nscores {\n ppqn 960\n}\n", + "swang 2\n\nscore\n", + "swang 2\n\nscore {\n", + "swang 2\n\nscore {\n ppqn 960\n", + "swang 2\n\n{\n ppqn 960\n}\n", + "swang 2\n\n", + ] { + assert_eq!( + codes(&refusal(source)), + ["SWG0401"], + "a prefix of the root is not the root: {source:?}" + ); + } +} + +#[test] +fn trailing_material_cannot_bypass_the_root_contract() { + for source in [ + "swang 2\n\nscore {\n ppqn 960\n}\nscore {\n ppqn 480\n}\n", + "swang 2\n\nscore {\n ppqn 960\n}\npattern riff {\n ascii \"x-\"\n}\n", + "swang 2\n\nscore {\n ppqn 960\n}\n}\n", + "swang 2\n\nscore {\n ppqn 960\n} ppqn 480\n", + ] { + assert_eq!( + codes(&refusal(source)), + ["SWG0401"], + "one document holds one root: {source:?}" + ); + } +} + +// ── the score body, and what this slice does not yet parse ────────────────── + +#[test] +fn ppqn_is_the_one_required_word() { + assert_eq!(codes(&refusal("swang 2\n\nscore {\n}\n")), ["SWG0403"]); + assert_eq!( + codes(&refusal( + "swang 2\n\nscore {\n ppqn 960\n ppqn 480\n}\n" + )), + ["SWG0404"], + "`ppqn` is a `1` word, so repeating it is SWG0404 (§6.4b)" + ); +} + +#[test] +fn an_unknown_score_word_is_refused() { + assert_eq!( + codes(&refusal( + "swang 2\n\nscore {\n ppqn 960\n nope 1\n}\n" + )), + ["SWG0401"], + "§6.4b: an unknown field word is SWG0401" + ); +} + +#[test] +fn grammatical_words_this_slice_does_not_parse_fail_closed() { + // `master_bar`, `track`, `source` and `loss` are real `score` words + // (§6.4b) owned by SWG-4A-08. Until it lands they are refused, never + // skipped: silently ignoring a word is how exact text stops being exact. + for word in [ + "master_bar {\n }", + "track {\n }", + "source {\n }", + "loss {\n }", + ] { + let source = format!("swang 2\n\nscore {{\n ppqn 960\n {word}\n}}\n"); + assert_eq!( + codes(&refusal(&source)), + ["SWG0401"], + "not yet parsed means refused, not ignored: {word}" + ); + } +} + +#[test] +fn the_one_scalar_this_slice_reads_refuses_what_the_registry_says_it_must() { + // Scoped deliberately to `ppqn`. The scalar layer — widths, non-zero + // types, rational tempo, ranges, pitch, velocity, meter, confidence, + // enums, escapes — is SWG-4A-07's and is absent here. + assert_eq!( + codes(&refusal("swang 2\n\nscore {\n ppqn 0960\n}\n")), + ["SWG0505"], + "§6.6: a leading zero is a non-canonical spelling" + ); + assert_eq!( + codes(&refusal("swang 2\n\nscore {\n ppqn 0\n}\n")), + ["SWG0506"], + "§6.6 names zero `ppqn` as a canonical-model invariant violation" + ); + for over in ["ppqn 70000", "ppqn 99999999999999999999"] { + assert_eq!( + codes(&refusal(&format!("swang 2\n\nscore {{\n {over}\n}}\n"))), + ["SWG0401"], + "a value that does not fit the field is SWG0401, as at level 1" + ); + } + for shape in ["ppqn", "ppqn x", "ppqn \"960\"", "ppqn 9/6", "ppqn 960bps"] { + assert_eq!( + codes(&refusal(&format!("swang 2\n\nscore {{\n {shape}\n}}\n"))), + ["SWG0401"], + "and a value of the wrong shape is SWG0401: {shape}" + ); + } +} + +// ── the formatter, dispatched ─────────────────────────────────────────────── + +#[test] +fn the_level_two_formatter_preserves_the_level_and_is_its_own_fixed_point() { + // §5.8: the formatter never downgrades or promotes a document. + let document = parse_document(MINIMAL_SCORE).expect("accepted"); + let canonical = format_document(&document); + assert!( + canonical.starts_with("swang 2\n"), + "the level travels with the document: {canonical:?}" + ); + let reparsed = parse_document(&canonical).expect("canonical text reparses"); + assert_eq!(format_document(&reparsed), canonical, "law 2: fixed point"); + let Document::Score(_) = reparsed else { + panic!("canonical level-2 text is still a score"); + }; +} + +#[test] +fn canonical_level_two_text_is_the_one_spelling() { + let document = parse_document("swang 2\n\nscore {\nppqn 960\n}\n").expect("accepted"); + assert_eq!(format_document(&document), MINIMAL_SCORE); +} + +// ── the inherited INF-06 obligations ──────────────────────────────────────── + +#[test] +fn the_source_byte_budget_breaches_end_to_end_before_any_success() { + // The declared limit, through the public path, with no scaling and no + // test-only reconstruction of the parser. The padding is whitespace, so + // the source is *grammatically* a valid minimal score and would be + // accepted if the byte check were not consulted first — which is what + // makes this a witness that no successful `swang 2` result precedes + // budget wiring, rather than merely a witness that a big string fails. + let padding = " ".repeat(16 * 1024 * 1024); + let source = format!("swang 2\n\nscore {{\n ppqn 960\n{padding}}}\n"); + let diagnostics = refusal(&source); + assert_eq!(codes(&diagnostics), ["SWG0509"]); + let first = diagnostics.first().expect("one diagnostic"); + assert!( + first.message.contains("source bytes") && first.message.contains("16777216"), + "the breach names its axis and the declared limit: {}", + first.message + ); +} + +#[test] +fn the_token_budget_breaches_end_to_end_as_a_typed_refusal() { + // Four million tokens plus one, spelled in under the byte cap, so the + // token axis is the one that fires. A typed `SWG0509`, not an + // allocation death. + let source = format!( + "swang 2\n\nscore {{\n ppqn 960\n{}}}\n", + "a ".repeat(4_000_001) + ); + let diagnostics = refusal(&source); + assert_eq!(codes(&diagnostics), ["SWG0509"]); + assert!( + diagnostics + .first() + .is_some_and(|d| d.message.contains("tokens")), + "the breach names the token axis" + ); +} + +#[test] +fn a_level_one_source_never_reaches_the_level_two_budget() { + // The budget is level 2's alone (§5.11). A level-1 source larger than + // the level-2 source cap is still parsed by level 1's rules, because + // level 1's acceptance set is frozen and a bound that reached it would + // be a narrowing. + let padding = " ".repeat(16 * 1024 * 1024); + let source = LEVEL_ONE.replace( + "pattern dgd_fractal {", + &format!("pattern dgd_fractal {{\n{padding}"), + ); + assert!( + source.len() > 16 * 1024 * 1024, + "the fixture really is over the level-2 cap" + ); + let document = parse_document(&source).expect("level 1 knows no such bound"); + let Document::Pattern(_) = document else { + panic!("it is still a pattern"); + }; + assert!(parse(&source).is_ok(), "and the frozen entry point agrees"); +} From 08a4be84281188e0f9180659f538ee1e2c964149 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 16:28:31 +0000 Subject: [PATCH 02/14] =?UTF-8?q?feat(swang):=20SWG-4A-06=20green=20?= =?UTF-8?q?=E2=80=94=20level=20dispatch=20and=20the=20level-2=20skeleton?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `header_level -> level dispatch -> root dispatch (pattern | score)`. Spec §5.4 gives each released level its own parser and formatter entry point and forbids a shared grammar with level-conditioned branches. So the dispatcher in `syntax/document.rs` is a router and nothing else: it holds no grammar, no token, no budget, and decides only which parser to call. It is deliberately absent from `level_two_budget_boundary.rs`'s exempt list — a budget consulted before the level branch would be a level-1 bound whatever file it lived in. `LANGUAGE_LEVEL` becomes 2. The frozen §1.1 pre-parser is untouched; what moved is the range it reports, which has to be the range the build actually has or `SWG0001` would name a lie. Level 1's entry point gains a guard, and only for a level it does not own. Without it a `swang 2` header would flow into a level-1 `Program`, and the formatter would emit `swang 2` above a `pattern` block — a document neither level would read back. It is `SWG0401`, the structural class level 1 already uses, not `SWG0001`: this build does support level 2, and §5.10 forbids one number carrying a second meaning. The arm cannot fire for a `swang 1` source, so Law A is untouched by its existence. The level-2 lexer retains a token that is a kind and a `Span` — 12 bytes, no `String`, text sliced back out of the source on demand. The bound is a `const` assertion in the lexer rather than a host test, so it travels to every target this crate builds for; `cargo check -p griff-swang --target wasm32-unknown-unknown` and the cockpit's wasm build both compile it. The parser accepts exactly the minimal empty score, `score { ppqn }`, and refuses everything else closed. `master_bar`, `track`, `source` and `loss` are real grammar owned by SWG-4A-08, and the refusal says so instead of pretending they are not words — but it is a refusal, because a parser that ignores a word it does not implement is how exact text stops being exact. The two registry codes implemented here, `SWG0505` and `SWG0506`, are the ones reachable from the single scalar this slice reads; the scalar layer proper stays SWG-4A-07's. Accepting `ppqn 0` — text §6.6 declares invalid — into an unfrozen level's accepted set was the alternative, and it is worse than implementing two lines of refusal. `admit_diagnostic` is deliberately never called: the diagnostic cap governs what a *recovering* parser returns, and recovery is SWG-INF-05's. This parser stops at the first refusal, so it can no more exhaust that axis than level 1 can. Local: 1490 tests green (904 core + 280 swang + 22 pattern + 90 cli + 194 ui-core), fmt clean, `clippy --workspace --all-targets -D warnings` exit 0, `check --workspace --all-targets` clean, wasm32 clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/src/syntax.rs | 3 + swang/src/syntax/document.rs | 76 +++++++ swang/src/syntax/format.rs | 1 + swang/src/syntax/format/v2.rs | 37 ++++ swang/src/syntax/header.rs | 10 +- swang/src/syntax/parser.rs | 1 + swang/src/syntax/parser/v1.rs | 23 ++- swang/src/syntax/parser/v2.rs | 297 ++++++++++++++++++++++++++++ swang/src/syntax/parser/v2/lexer.rs | 135 +++++++++++++ swang/src/syntax/tests.rs | 5 +- 10 files changed, 582 insertions(+), 6 deletions(-) create mode 100644 swang/src/syntax/document.rs create mode 100644 swang/src/syntax/format/v2.rs create mode 100644 swang/src/syntax/parser/v2.rs create mode 100644 swang/src/syntax/parser/v2/lexer.rs diff --git a/swang/src/syntax.rs b/swang/src/syntax.rs index d03c274..58701f8 100644 --- a/swang/src/syntax.rs +++ b/swang/src/syntax.rs @@ -55,6 +55,7 @@ mod ast; mod diagnostic; +mod document; mod format; mod header; mod lexer; @@ -69,9 +70,11 @@ pub use ast::v1::{ MapRhythm, PatternDef, Program, Prune, StrategyName, StrategyPolicy, StringLiteral, Unit, }; pub use diagnostic::Diagnostic; +pub use document::{format_document, parse_document, Document}; pub use format::v1::format; pub use header::{header_level, LANGUAGE_LEVEL}; pub use parser::v1::{parse, parse_with_source_map}; +pub use parser::v2::ExactScore; pub use source_map::{AstId, FieldKind, FieldRef, Parsed, SourceMap}; pub use span::Span; #[cfg(test)] diff --git a/swang/src/syntax/document.rs b/swang/src/syntax/document.rs new file mode 100644 index 0000000..0837dd8 --- /dev/null +++ b/swang/src/syntax/document.rs @@ -0,0 +1,76 @@ +//! Level dispatch: one header, one entry point, no shared grammar. +//! +//! ```text +//! header_level -> level dispatch -> root dispatch (pattern | score) +//! ``` +//! +//! Spec §5.4 requires that "each released level owns its own parser and +//! formatter entry point. A build routes to one of them and never mixes +//! them: there is no single grammar with level-conditioned branches, because +//! such a grammar has no way to prove that level 1's behaviour survived the +//! addition of level 2." +//! +//! So this module is a router and nothing else. It holds no grammar, no +//! token, no budget and no diagnostic of its own beyond the one defensive +//! arm below: everything it can say, it says by choosing which parser to +//! call. That is also why it is not on `level_two_budget_boundary.rs`'s +//! exempt list — a budget consulted here, before the branch, would be a +//! level-1 bound whatever file it lived in. + +use super::ast::v1::Program; +use super::diagnostic::Diagnostic; +use super::format::v1::format as format_pattern; +use super::format::v2::format_exact; +use super::header::{header_level, LANGUAGE_LEVEL}; +use super::parser; +use super::parser::v2::ExactScore; +use super::span::span_of; + +/// A parsed Swang document, tagged by the root its level admits. +/// +/// The two roots are two levels' (spec §5.7): `swang 1` writes `pattern`, +/// `swang 2` writes `score`, and nothing has yet earned a document that +/// holds both at once. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Document { + /// A level-1 pattern program. + Pattern(Program), + /// A level-2 exact score. + Score(ExactScore), +} + +/// Parses a source at whichever level its header pins. +/// +/// # Errors +/// The frozen §1.1 pre-parser's diagnostics for a header this build cannot +/// read, and otherwise exactly the chosen level's own. +pub fn parse_document(source: &str) -> Result> { + let level = header_level(source).map_err(|d| vec![d])?; + match level { + 1 => parser::v1::parse(source).map(Document::Pattern), + 2 => parser::v2::parse_exact(source).map(Document::Score), + // Unreachable: the pre-parser already refused anything outside + // `1..=LANGUAGE_LEVEL`. Defense in depth, so that raising the + // constant without adding an arm is a refusal rather than a panic + // or, worse, a silent fall-through to the wrong grammar. + other => Err(vec![Diagnostic { + code: "SWG0001", + span: span_of(6, 6), + message: format!( + "language level {other} has no parser in this build (1..={LANGUAGE_LEVEL})" + ), + }]), + } +} + +/// Emits the canonical text of a document, at its own level. +/// +/// Level 1's output is byte-identical to the frozen level-1 formatter's, +/// permanently (spec §5.8, Law A observable 3). +#[must_use] +pub fn format_document(document: &Document) -> String { + match document { + Document::Pattern(program) => format_pattern(program), + Document::Score(score) => format_exact(score), + } +} diff --git a/swang/src/syntax/format.rs b/swang/src/syntax/format.rs index 5adebf6..ffac2df 100644 --- a/swang/src/syntax/format.rs +++ b/swang/src/syntax/format.rs @@ -4,3 +4,4 @@ //! than editing it. pub(crate) mod v1; +pub(crate) mod v2; diff --git a/swang/src/syntax/format/v2.rs b/swang/src/syntax/format/v2.rs new file mode 100644 index 0000000..a54af4c --- /dev/null +++ b/swang/src/syntax/format/v2.rs @@ -0,0 +1,37 @@ +//! The level-2 canonical formatter. +//! +//! Spec §5.8: for `swang 2` the formatter preserves `swang 2`. It never +//! downgrades a document to an older level and never promotes one to a newer +//! level — changing a level is an authoring act, never a formatting act. +//! +//! The canonical layout is `exact-score-text.md` §6.1's, restricted to what +//! SWG-4A-06 parses: the header, one blank line, and the `score` block with +//! a four-space field indent. The blocks that fill a fuller score are +//! SWG-4A-08's, and this slice cannot construct one that holds them. + +use crate::syntax::ast::v2::ExactScoreDocument; +use crate::syntax::parser::v2::ExactScore; + +/// Emits the one canonical text for `score`. +/// +/// `format_exact(parse_exact(t))` is idempotent and +/// `parse_exact(format_exact(s))` recovers the same score. +pub(crate) fn format_exact(score: &ExactScore) -> String { + // Destructured with no `..` on purpose: when SWG-4A-08 starts filling + // the structural slots, this function stops compiling until it learns to + // emit them. A formatter that silently drops what it does not recognise + // is the one defect a canonical writer must not have. + let ExactScoreDocument { + ppqn, + master_bars, + tracks, + source, + loss, + } = score.document(); + debug_assert!( + master_bars.is_empty() && tracks.is_empty() && source.is_none() && loss.is_empty(), + "SWG-4A-06 parses only the minimal score; a document holding more \ + than that has arrived from somewhere this formatter cannot serve" + ); + format!("swang 2\n\nscore {{\n ppqn {ppqn}\n}}\n") +} diff --git a/swang/src/syntax/header.rs b/swang/src/syntax/header.rs index f68c3e8..a9198d9 100644 --- a/swang/src/syntax/header.rs +++ b/swang/src/syntax/header.rs @@ -8,9 +8,13 @@ use std::str::from_utf8; use super::diagnostic::Diagnostic; use super::span::span_of; -/// The language level this build parses (spec §1.1). Levels are additive-only -/// and never enter any content hash. -pub const LANGUAGE_LEVEL: u32 = 1; +/// The newest language level this build parses (spec §1.1). Levels are +/// additive-only and never enter any content hash. +/// +/// SWG-4A-06 raised this to 2. The pre-parser below is unchanged — that is +/// what "frozen" means — and what moved is the range it reports, which has +/// to be the range the build actually has or `SWG0001` would name a lie. +pub const LANGUAGE_LEVEL: u32 = 2; /// The frozen §1.1 pre-parser: reads at most 64 bytes of the first line and /// returns the pinned language level. diff --git a/swang/src/syntax/parser.rs b/swang/src/syntax/parser.rs index 7f06834..2f4da57 100644 --- a/swang/src/syntax/parser.rs +++ b/swang/src/syntax/parser.rs @@ -4,3 +4,4 @@ //! than editing it. pub(crate) mod v1; +pub(crate) mod v2; diff --git a/swang/src/syntax/parser/v1.rs b/swang/src/syntax/parser/v1.rs index 4e95e05..3ce2c49 100644 --- a/swang/src/syntax/parser/v1.rs +++ b/swang/src/syntax/parser/v1.rs @@ -22,9 +22,30 @@ use crate::TailPolicy; /// # Errors /// Exactly [`parse`]'s errors. pub fn parse_with_source_map(source: &str) -> Result, Vec> { + let pinned = header_level(source).map_err(|d| vec![d])?; + // §5.4: each released level owns its own entry point, and a build "routes + // to one of them and never mixes them". This is level 1's, so a source + // pinning any other level is refused here rather than half-read — which + // also keeps a level-2 header out of a level-1 `Program`, where the + // formatter would emit `swang 2` above a `pattern` block. + // + // `SWG0401` because that is what it is: a structural violation of the + // level-1 document contract, the same class as any other. It is not + // `SWG0001` — this build does support level 2, and §5.10 forbids one + // number carrying a second meaning. This arm cannot fire for a `swang 1` + // source, so Law A (§5.5) is untouched by its existence. + if pinned != 1 { + return Err(vec![Diagnostic { + code: "SWG0401", + span: level_span(source), + message: format!( + "the level-1 parser reads `swang 1`; this source pins `swang {pinned}`" + ), + }]); + } // `header_level` already enforced 1..=LANGUAGE_LEVEL; the map_err is // defense in depth, not a reachable path. - let level = Level::new(header_level(source).map_err(|d| vec![d])?).map_err(|e| { + let level = Level::new(pinned).map_err(|e| { vec![Diagnostic { code: "SWG0002", span: span_of(0, 0), diff --git a/swang/src/syntax/parser/v2.rs b/swang/src/syntax/parser/v2.rs new file mode 100644 index 0000000..c584bb0 --- /dev/null +++ b/swang/src/syntax/parser/v2.rs @@ -0,0 +1,297 @@ +//! The level-2 exact-score parser: root dispatch and the minimal score. +//! +//! Spec §5.4 gives each released level its own parser entry point, and this +//! is level 2's. It never sees a `swang 1` source and never branches on a +//! level — the branch happened one module up, in the dispatcher. +//! +//! # What this slice parses +//! +//! Exactly the minimal empty score: +//! +//! ```text +//! swang 2 +//! +//! score { +//! ppqn 960 +//! } +//! ``` +//! +//! `ppqn` is `score`'s only `1` word; `master_bar`, `track`, `source` and +//! `loss` are `*`/`?` words (`exact-score-text.md` §6.4b) whose omission is +//! §6.2's, not a missing required word. So the minimal score is a complete +//! level-2 program, not a stub — and the structural tree that fills those +//! slots is SWG-4A-08's. +//! +//! Words this slice does not implement are **refused**, never skipped. A +//! parser that ignores a word it does not understand is how exact text stops +//! being exact, and the refusal is a `SWG0401` that says which task owns the +//! word rather than pretending the grammar has no such thing. +//! +//! # Where the budget is spent +//! +//! The level-2 budget (SWG-INF-06, spec §5.11) is constructed here and +//! nowhere else, and it is consulted before the work it bounds: the source +//! byte cap before lexing, the token cap inside the lexer as each token is +//! retained, and structural depth as each block is entered. There is no +//! successful `swang 2` result that does not pass through all three. +//! +//! `admit_diagnostic` is deliberately not called. The diagnostic cap governs +//! what a *recovering* parser returns, and recovery is SWG-INF-05's; this +//! parser stops at the first refusal, so it can no more exhaust that axis +//! than level 1 can. + +pub(crate) mod lexer; + +use self::lexer::{lex_level_two, Level2Token, Level2TokenKind}; +use crate::syntax::ast::v2::ExactScoreDocument; +use crate::syntax::diagnostic::Diagnostic; +use crate::syntax::header::HEADER_WINDOW; +use crate::syntax::limits::Level2Budget; +use crate::syntax::span::{span_of, Span}; + +/// A parsed level-2 exact score. +/// +/// Opaque on purpose. SWG-4A-02 made the exact document a transient syntax +/// form and fenced it off the public surface so it could not quietly become +/// a second durable model beside `griff_core::Score`; the fence holds here. +/// This handle exists so that a level-2 parse has a public result at all — +/// which is what lets the fuzz oracle reach the real path — and it exposes +/// nothing of what it wraps. SWG-4A-09's builder is what opens it, into a +/// `Score` and nothing else. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ExactScore(ExactScoreDocument); + +impl ExactScore { + /// The wrapped document, for the crate-internal formatter. + pub(crate) const fn document(&self) -> &ExactScoreDocument { + &self.0 + } +} + +/// Parses a level-2 source. The caller has already proved the header says +/// `swang 2`; this function owns everything after it. +/// +/// # Errors +/// One diagnostic. `SWG0509` for a declared budget breach, `SWG0403` for a +/// missing required word, `SWG0404` for a repeated singleton, `SWG0505` for +/// a non-canonical spelling, `SWG0506` for a canonical-model invariant, and +/// `SWG0401` for everything structural. +pub(crate) fn parse_exact(source: &str) -> Result> { + parse_score(source).map(ExactScore).map_err(|d| vec![d]) +} + +fn parse_score(source: &str) -> Result { + let mut budget = Level2Budget::declared(); + budget.admit_source(source, span_of(0, source.len()))?; + let tokens = lex_level_two(source, body_offset(source), &mut budget)?; + let mut parser = Parser { + source, + tokens, + pos: 0, + eof: span_of(source.len(), source.len()), + budget, + }; + let document = parser.score()?; + parser.expect_end()?; + Ok(document) +} + +/// The first byte after the header line, matching level 1's own arithmetic. +fn body_offset(source: &str) -> usize { + source + .as_bytes() + .iter() + .take(HEADER_WINDOW) + .position(|&b| b == b'\n') + .map_or(source.len(), |lf| lf.saturating_add(1)) +} + +/// The `score` words this grammar defines but this slice does not yet parse +/// (`exact-score-text.md` §6.4b). Named so the refusal can say which task +/// owns them instead of claiming they are not words at all. +const DEFERRED_SCORE_WORDS: &[&str] = &["master_bar", "track", "source", "loss"]; + +struct Parser<'a> { + source: &'a str, + tokens: Vec, + pos: usize, + eof: Span, + budget: Level2Budget, +} + +impl Parser<'_> { + fn next(&mut self) -> Option { + let token = self.tokens.get(self.pos).copied(); + if token.is_some() { + self.pos = self.pos.saturating_add(1); + } + token + } + + fn peek(&self) -> Option { + self.tokens.get(self.pos).copied() + } + + fn text(&self, token: Level2Token) -> &str { + token.text_in(self.source) + } + + fn unexpected_end(&self) -> Diagnostic { + Diagnostic { + code: "SWG0401", + span: self.eof, + message: "unexpected end of input".to_owned(), + } + } + + /// A structural refusal at a token: `SWG0401`, the class level 1 already + /// uses for an unexpected token or a violated shape. + const fn structural(token: Level2Token, message: String) -> Diagnostic { + Diagnostic { + code: "SWG0401", + span: token.span, + message, + } + } + + fn expect_kind( + &mut self, + kind: Level2TokenKind, + what: &str, + ) -> Result { + let token = self.next().ok_or_else(|| self.unexpected_end())?; + if token.kind == kind { + Ok(token) + } else { + let found = self.text(token).to_owned(); + Err(Self::structural( + token, + format!("expected {what}, found `{found}`"), + )) + } + } + + /// `score { … }` — the one level-2 root (spec §5.7). + fn score(&mut self) -> Result { + let root = self.expect_kind(Level2TokenKind::Word, "a root construct")?; + if self.text(root) != "score" { + let found = self.text(root).to_owned(); + return Err(Self::structural( + root, + format!("expected the `score` root, found `{found}`; level 2 admits no other root"), + )); + } + let open = self.expect_kind(Level2TokenKind::OpenBrace, "`{`")?; + self.budget.enter_block(open.span)?; + let ppqn = self.score_body(root.span)?; + self.budget.leave_block(); + Ok(ExactScoreDocument { + ppqn, + master_bars: Vec::new(), + tracks: Vec::new(), + source: None, + loss: Vec::new(), + }) + } + + /// The `score` body up to its closing brace, returning the one value + /// this slice reads. + fn score_body(&mut self, root: Span) -> Result { + let mut ppqn: Option = None; + loop { + let token = self.next().ok_or_else(|| self.unexpected_end())?; + match token.kind { + Level2TokenKind::CloseBrace => break, + Level2TokenKind::Word => { + let word = self.text(token).to_owned(); + if word != "ppqn" { + return Err(Self::unknown_word(token, &word)); + } + if ppqn.is_some() { + return Err(Diagnostic { + code: "SWG0404", + span: token.span, + message: "`score` takes one `ppqn` word".to_owned(), + }); + } + ppqn = Some(self.ppqn_value()?); + } + Level2TokenKind::Number | Level2TokenKind::OpenBrace => { + let found = self.text(token).to_owned(); + return Err(Self::structural( + token, + format!("expected a word, found `{found}`"), + )); + } + } + } + ppqn.ok_or_else(|| Diagnostic { + code: "SWG0403", + span: root, + message: "`score` requires a `ppqn` word".to_owned(), + }) + } + + /// A word `score` does not take here, said honestly: a word the grammar + /// defines but this slice has not implemented is not the same failure as + /// a word nobody has ever defined, and the message says which it is. + fn unknown_word(token: Level2Token, word: &str) -> Diagnostic { + let message = if DEFERRED_SCORE_WORDS.contains(&word) { + format!( + "`score` takes a `{word}` word, but this build does not parse \ + it yet (SWG-4A-08); it is refused rather than ignored" + ) + } else { + format!("`score` does not take a `{word}` word") + }; + Diagnostic { + code: "SWG0401", + span: token.span, + message, + } + } + + /// `ppqn ` — the one scalar this slice reads. + /// + /// The scalar layer proper is SWG-4A-07's. The two registry refusals + /// here are the ones reachable from this single word, and they are + /// implemented rather than deferred because the alternative is to accept + /// `ppqn 0960` and `ppqn 0` — text §6.6 declares invalid — into the + /// accepted set of a level that has not frozen. + fn ppqn_value(&mut self) -> Result { + let token = self.expect_kind(Level2TokenKind::Number, "a `ppqn` value")?; + let text = self.text(token); + if text.len() > 1 && text.starts_with('0') { + return Err(Diagnostic { + code: "SWG0505", + span: token.span, + message: format!("`{text}` has a leading zero; the canonical spelling has none"), + }); + } + let value: u16 = text.parse().map_err(|_| Diagnostic { + code: "SWG0401", + span: token.span, + message: format!("`{text}` does not fit the `ppqn` field"), + })?; + if value == 0 { + return Err(Diagnostic { + code: "SWG0506", + span: token.span, + message: "`ppqn` is the tick resolution and cannot be zero".to_owned(), + }); + } + Ok(value) + } + + /// One document holds one root: anything after the root's closing brace + /// is a second document trying to arrive inside the first. + fn expect_end(&self) -> Result<(), Diagnostic> { + self.peek().map_or(Ok(()), |token| { + let found = self.text(token).to_owned(); + Err(Self::structural( + token, + format!("expected the end of the document, found `{found}`"), + )) + }) + } +} diff --git a/swang/src/syntax/parser/v2/lexer.rs b/swang/src/syntax/parser/v2/lexer.rs new file mode 100644 index 0000000..608a851 --- /dev/null +++ b/swang/src/syntax/parser/v2/lexer.rs @@ -0,0 +1,135 @@ +//! The level-2 lexer and the token it retains. +//! +//! Level 1's [`Token`] owns a `String` per lexeme. That is not the level-2 +//! storage representation, and saying so is an obligation rather than a +//! preference: spec §5.11 declares `MAX_TOKENS = 4_000_000`, and the +//! derivation behind that number budgets **at most 12 bytes per retained +//! token on `wasm32`**. A token owning a `String` costs 12 bytes for the +//! `String` alone on a 32-bit target, before its kind, its span, and the +//! heap block each lexeme would allocate. +//! +//! So a level-2 token is a kind and a [`Span`], and the lexeme is sliced +//! back out of the source on demand ([`Level2Token::text_in`]). The bound is +//! asserted at compile time rather than in a host test, because the claim is +//! about `wasm32` and a runtime assertion on `x86_64` proves `x86_64`. +//! +//! [`Token`]: crate::syntax::token::Token + +use std::iter::Peekable; +use std::mem::size_of; +use std::str::CharIndices; + +use crate::syntax::diagnostic::Diagnostic; +use crate::syntax::limits::Level2Budget; +use crate::syntax::span::{span_of, Span}; + +/// One retained level-2 lexeme: what it is, and where it is. Never what it +/// says — that is [`Level2Token::text_in`]'s job, and the reason this type +/// fits the budget. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct Level2Token { + /// Which lexeme class this is. + pub(crate) kind: Level2TokenKind, + /// The full source range of the lexeme. + pub(crate) span: Span, +} + +/// The lexeme classes the level-2 grammar reads. Strings, and with them the +/// escape policy and `SWG0508`, are SWG-4A-07's. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Level2TokenKind { + /// `[A-Za-z_][A-Za-z0-9_]*` + Word, + /// A run of ASCII digits. Its spelling laws belong to the construct that + /// reads it, not to the lexer. + Number, + /// `{` + OpenBrace, + /// `}` + CloseBrace, +} + +/// The retained-token budget from spec §5.11, asserted where every target +/// this crate builds for must satisfy it — `wasm32` included. +/// +/// The preregistered SWG-INF-06 falsification probe is *adding owned lexeme +/// text to the level-2 token*: a `String` field takes the type to 24 bytes +/// on `wasm32`, and this assertion is what refuses to compile. +const _RETAINED_TOKEN_FITS_THE_BUDGET: () = assert!( + size_of::() <= 12, + "a retained level-2 token must fit spec §5.11's 12-byte wasm32 budget; \ + owning lexeme text is what breaks this" +); + +impl Level2Token { + /// The lexeme, sliced out of the source this token was lexed from. + pub(crate) fn text_in<'a>(&self, source: &'a str) -> &'a str { + source + .get(self.span.start as usize..self.span.end as usize) + .unwrap_or_default() + } +} + +/// Lexes `source` from byte `from` on, spending one token of `budget` for +/// every token retained. +/// +/// Whitespace is ASCII only, as at level 1: the determinism law (spec §1.2) +/// keeps Unicode classification out of anything semantics can observe. +/// +/// # Errors +/// `SWG0401` for a character the level-2 grammar does not read, and +/// `SWG0509` when the token allowance is spent — the breach is typed and +/// reaches the caller rather than being discovered by the allocator. +pub(crate) fn lex_level_two( + source: &str, + from: usize, + budget: &mut Level2Budget, +) -> Result, Diagnostic> { + let tail = source.get(from..).unwrap_or_default(); + let mut tokens = Vec::new(); + let mut chars = tail.char_indices().peekable(); + while let Some((at, c)) = chars.next() { + let start = from.saturating_add(at); + let kind = match c { + ' ' | '\t' | '\r' | '\n' => continue, + '{' => Level2TokenKind::OpenBrace, + '}' => Level2TokenKind::CloseBrace, + 'A'..='Z' | 'a'..='z' | '_' => Level2TokenKind::Word, + '0'..='9' => Level2TokenKind::Number, + other => { + return Err(Diagnostic { + code: "SWG0401", + span: span_of(start, start.saturating_add(other.len_utf8())), + message: format!("unexpected character {other:?}"), + }) + } + }; + let end = match kind { + Level2TokenKind::Word => run(tail, &mut chars, |ch| { + ch.is_ascii_alphanumeric() || ch == '_' + }), + Level2TokenKind::Number => run(tail, &mut chars, |ch| ch.is_ascii_digit()), + Level2TokenKind::OpenBrace | Level2TokenKind::CloseBrace => at.saturating_add(1), + }; + let span = span_of(start, from.saturating_add(end)); + // Admitted before it is retained: a token that would exceed the + // allowance is never stored, so the refusal costs one token's worth + // of memory rather than the whole tail of the file. + budget.admit_token(span)?; + tokens.push(Level2Token { kind, span }); + } + Ok(tokens) +} + +/// Consumes characters while `keep` holds; returns the end byte offset +/// relative to `tail`. +fn run(tail: &str, chars: &mut Peekable>, keep: impl Fn(char) -> bool) -> usize { + while let Some(&(_, c)) = chars.peek() { + if keep(c) { + chars.next(); + } else { + break; + } + } + chars.peek().map_or(tail.len(), |&(next, _)| next) +} diff --git a/swang/src/syntax/tests.rs b/swang/src/syntax/tests.rs index 12c879f..15f69be 100644 --- a/swang/src/syntax/tests.rs +++ b/swang/src/syntax/tests.rs @@ -914,6 +914,7 @@ mod level_two_budget { mod level_two_token_storage { use std::mem::size_of; + use crate::syntax::limits::Level2Budget; use crate::syntax::parser::v2::lexer::{lex_level_two, Level2Token, Level2TokenKind}; use crate::syntax::span::span_of; @@ -999,8 +1000,8 @@ mod level_two_token_storage { ); } - fn budget_for(source: &str) -> crate::syntax::limits::Level2Budget { - let budget = crate::syntax::limits::Level2Budget::declared(); + fn budget_for(source: &str) -> Level2Budget { + let budget = Level2Budget::declared(); budget .admit_source(source, span_of(0, source.len())) .expect("this fixture is far under the source cap"); From 0a59d669ed5b0ced88c290ae7773e14711f4b956 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 16:30:30 +0000 Subject: [PATCH 03/14] =?UTF-8?q?test(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20the=20depth=20axis=20is=20wired=20but=20unfalsifiab?= =?UTF-8?q?le?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A falsification probe found it the honest way. Deleting `enter_block` from the level-2 parser was caught by **nothing**: the minimal score has exactly one block, so nesting can never approach `MAX_NESTING_DEPTH` and no end-to-end breach can witness the accounting the backlog asks for ("structural-depth accounting during parsing"). Two witnesses that observe the counter directly, on the real parse function with the real budget rather than a reconstruction of it. The first is the one that can fail: a refusal *inside* the block returns before `leave_block`, so a depth of one proves the block was entered — where a balanced parse returns the counter to zero whether it was entered or not, and would have passed either way. RED: `E0603: function parse_score is private` — the witness needs the real entry to take the caller's budget, and it does not yet. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/src/syntax/tests.rs | 52 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/swang/src/syntax/tests.rs b/swang/src/syntax/tests.rs index 15f69be..8d7ec58 100644 --- a/swang/src/syntax/tests.rs +++ b/swang/src/syntax/tests.rs @@ -909,6 +909,58 @@ mod level_two_budget { } } +/// SWG-4A-06: the depth axis is wired, and the wiring is falsifiable. +/// +/// The minimal score has exactly one block, so nesting can never approach +/// `MAX_NESTING_DEPTH` and no end-to-end breach can witness the accounting. +/// A falsification probe found that out the honest way: deleting +/// `enter_block` from the parser was caught by nothing at all. So the +/// witness observes the counter directly, on the real parse function with +/// the real budget — not on a reconstruction of it. +mod level_two_depth_accounting { + use crate::syntax::limits::Level2Budget; + use crate::syntax::parser::v2::parse_score; + use crate::syntax::span::span_of; + + fn budget_for(source: &str) -> Level2Budget { + let budget = Level2Budget::declared(); + budget + .admit_source(source, span_of(0, source.len())) + .expect("far under the source cap"); + budget + } + + #[test] + fn the_root_block_is_entered_on_the_caller_s_budget() { + // A refusal *inside* the block is what makes this observable: the + // parse returns before `leave_block`, so a depth of one is proof the + // block was entered, where a balanced parse would have returned the + // counter to zero either way. + let source = "swang 2 + +score { + nope 1 +} +"; + let mut budget = budget_for(source); + let refusal = parse_score(source, &mut budget).expect_err("`nope` is not a `score` word"); + assert_eq!(refusal.code, "SWG0401"); + assert_eq!( + budget.depth(), + 1, + "the root block was entered on the budget the caller owns" + ); + } + + #[test] + fn a_balanced_parse_returns_the_depth_it_borrowed() { + let source = "swang 2\n\nscore {\n ppqn 960\n}\n".replace("\\n", "\n"); + let mut budget = budget_for(&source); + parse_score(&source, &mut budget).expect("the minimal score"); + assert_eq!(budget.depth(), 0, "what was entered was left"); + } +} + /// SWG-4A-06's inherited SWG-INF-06 obligation: the level-2 token proves the /// `MAX_TOKENS` heap derivation instead of asserting it in prose. mod level_two_token_storage { From 76b77d0d639e95c993d70fb77dbf31a08073bd73 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 16:31:27 +0000 Subject: [PATCH 04/14] =?UTF-8?q?fix(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20the=20parse=20spends=20a=20budget=20its=20caller=20?= =?UTF-8?q?owns?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `parse_score` takes `&mut Level2Budget` instead of constructing one it then drops. `parse_exact` is unchanged in behaviour: it builds the declared budget and hands it in, so the production path spends exactly what it spent before. What changes is that the accounting is now observable on the real function. The counters can be read after a parse — including after a refusal, which is the case that matters: a depth still held at one is proof the block was entered, where a balanced parse returns the counter to zero whether it was entered or not. This is not a side channel. There is no second entry that can succeed without consuming the same budget, and the budget cannot be weakened by the caller: `Level2ResourceLimits` has private fields and `declared()` as its only production constructor, which SWG-INF-06 sealed for exactly this. P26 the parser never enters the block it opened SURVIVED @ 08a4be8 → CAUGHT @ this commit 282 swang tests green, fmt clean, clippy -D warnings exit 0, wasm32 clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/src/syntax/parser/v2.rs | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/swang/src/syntax/parser/v2.rs b/swang/src/syntax/parser/v2.rs index c584bb0..a5b9bc1 100644 --- a/swang/src/syntax/parser/v2.rs +++ b/swang/src/syntax/parser/v2.rs @@ -77,13 +77,28 @@ impl ExactScore { /// a non-canonical spelling, `SWG0506` for a canonical-model invariant, and /// `SWG0401` for everything structural. pub(crate) fn parse_exact(source: &str) -> Result> { - parse_score(source).map(ExactScore).map_err(|d| vec![d]) + let mut budget = Level2Budget::declared(); + parse_score(source, &mut budget) + .map(ExactScore) + .map_err(|d| vec![d]) } -fn parse_score(source: &str) -> Result { - let mut budget = Level2Budget::declared(); +/// The parse itself, spending a budget the caller owns. +/// +/// The caller owning it is what makes the accounting observable: the +/// counters can be read after the parse — including after a refusal, where +/// a depth still held is proof the block was entered. The budget is not an +/// argument a caller may weaken, because `Level2ResourceLimits` has private +/// fields and `declared()` as its only production constructor. +/// +/// # Errors +/// Exactly [`parse_exact`]'s, unwrapped from the vector. +pub(crate) fn parse_score( + source: &str, + budget: &mut Level2Budget, +) -> Result { budget.admit_source(source, span_of(0, source.len()))?; - let tokens = lex_level_two(source, body_offset(source), &mut budget)?; + let tokens = lex_level_two(source, body_offset(source), budget)?; let mut parser = Parser { source, tokens, @@ -116,7 +131,7 @@ struct Parser<'a> { tokens: Vec, pos: usize, eof: Span, - budget: Level2Budget, + budget: &'a mut Level2Budget, } impl Parser<'_> { From 0b40859de7b1149bf81d8e3a87f4faa49ea0c2a8 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 16:32:34 +0000 Subject: [PATCH 05/14] =?UTF-8?q?docs(swang):=20SWG-4A-06=20closure=20?= =?UTF-8?q?=E2=80=94=20the=20slice=20boundary=20and=20what=204A-07/08=20in?= =?UTF-8?q?herit?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The backlog entry records what this task accepts (`score { ppqn }`, a complete level-2 program rather than a stub, because the structural blocks are `*`/`?` words), what it refuses and who owns each refusal, and the three inherited SWG-INF-06 obligations as discharged rather than claimed. One item is inherited by 4A-08 and was found by falsification rather than assumed: the depth axis cannot breach end-to-end in a grammar with one block. When the structural tree makes nesting reachable from text, the depth cap earns an end-to-end breach of its own. Four decision-log entries. The prior-art survey AGENTS.md requires covers the one genuinely new mechanism, the non-owning token: `rustc_lexer` keeps text in the source buffer behind a kind and a length, `rowan` interns rather than owning per occurrence, `logos` hands the caller a `Span` to slice. The twelve-byte bound is not theirs — it is §5.11's derivation from `MAX_TOKENS` and the cockpit's `wasm32` heap, and the compile-time assertion enforcing it is this task's. Recording a survey is not a licence to claim more inheritance than there is. The other three record the level-1 guard and why it is `SWG0401` and not `SWG0001`; why two of 4A-07's registry codes are implemented one task early for `ppqn` alone; and the survived probe that made the parse spend a budget its caller owns. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- docs/decisions.log.md | 54 ++++++++++++++++++++++++++++++++ docs/swang/foundation-backlog.md | 48 +++++++++++++++++++++++++++- 2 files changed, 101 insertions(+), 1 deletion(-) diff --git a/docs/decisions.log.md b/docs/decisions.log.md index 8e6b082..5449cee 100644 --- a/docs/decisions.log.md +++ b/docs/decisions.log.md @@ -2733,3 +2733,57 @@ Architectural decisions go to [`adr/`](adr/) instead. Guarding against `limits::` in a string would need a Rust parser, which costs more than this boundary is worth; the trade is stated in the test rather than left for someone to discover. + +- 2026-09-02 — In the context of SWG-4A-06's level-2 lexer, facing spec + §5.11's contract that a retained token costs at most twelve bytes on + `wasm32`, we decided to **retain a kind and a span and slice the lexeme + from the source on demand**, to achieve a token that cannot silently grow + a heap allocation per lexeme, accepting that every consumer of a token's + text must carry the source alongside it. Prior art, since the technique is + not this task's invention: `rustc_lexer` emits tokens carrying a kind and + a length and leaves the text in the source buffer; `rust-analyzer`'s + `rowan` keeps green-tree tokens interned rather than owned per + occurrence; `logos` generates lexers whose token payload is a `Span` the + caller slices. What none of them supplies is the *bound* — twelve bytes is + §5.11's own derivation from `MAX_TOKENS` and the cockpit's `wasm32` heap, + and the compile-time assertion that enforces it is this task's. Level 1's + `String`-owning `Token` is untouched: it is frozen, it is not the level-2 + representation, and the two now differ on purpose. + +- 2026-09-02 — In the context of the level-1 parser meeting a `swang 2` + header once the build supports level 2, we decided to **refuse it in the + level-1 entry point with `SWG0401`**, to achieve §5.4's "routes to one of + them and never mixes them", accepting that a caller who reaches for the + frozen entry point with level-2 text gets a structural refusal rather than + a routing hint. Without the guard, `Level::new(2)` succeeded and a level-1 + `Program` carried level 2, so the formatter emitted `swang 2` above a + `pattern` block — a document neither level would read back. `SWG0001` was + rejected for it: this build *does* support level 2, and §5.10 forbids one + number carrying a second meaning. The arm cannot fire for a `swang 1` + source, so Law A is untouched by its existence, and the frozen baseline + proves it rather than the claim resting on inspection. + +- 2026-09-02 — In the context of SWG-4A-06 reading exactly one scalar, we + decided to **implement `SWG0505` and `SWG0506` for `ppqn` alone**, to + achieve a level-2 acceptance set that contains no text §6.6 declares + invalid, accepting that two codes belonging to 4A-07's scalar layer appear + one task early. The alternative was `parse("swang 2\n\nscore { ppqn 0 }")` + returning `Ok` — a permissive fallback in the accepted set of a level that + has not frozen, which 4A-07 would then have to *narrow*. Refusing early is + the cheaper mistake to correct: widening an unfrozen level is routine, + and the two checks are four lines scoped to the one word this slice reads. + +- 2026-09-02 — In the context of SWG-4A-06's depth accounting, facing a + falsification probe that **survived**, we decided to **have the parse + spend a budget its caller owns**, to achieve a wiring that can be + falsified today rather than when 4A-08 makes nesting reachable, accepting + a crate-internal signature that hands the budget in. Deleting + `enter_block` from the level-2 parser was caught by nothing at all: the + minimal score has one block, so no end-to-end breach can witness the + counter. The witness now reads it after a refusal *inside* the block, + where a depth still held is proof of entry — a balanced parse returns to + zero whether the block was entered or not, and would have passed either + way. This is not a side channel: there is no second entry that can succeed + without spending the same budget, and `Level2ResourceLimits` still has + private fields and `declared()` as its only production constructor. + diff --git a/docs/swang/foundation-backlog.md b/docs/swang/foundation-backlog.md index 15335c7..7700bad 100644 --- a/docs/swang/foundation-backlog.md +++ b/docs/swang/foundation-backlog.md @@ -117,7 +117,7 @@ recorded in `decisions.log.md` if reversed. | SWG-4A-03 | Writer: transport and master timeline | code | 4A-01 | | SWG-4A-04 | Writer: tracks, voices, groups, atoms | code | 4A-03 | | SWG-4A-05 | Writer: techniques, positions, evidence, losses *(done)* | code | 4A-04, CORE-01 | -| SWG-4A-06 | Parser skeleton and level/root dispatch | code | 4A-01, INF-04, INF-06 | +| SWG-4A-06 | Parser skeleton and level/root dispatch *(done)* | code | 4A-01, INF-04, INF-06 | | SWG-4A-07 | Parser: exact scalar types | code | 4A-06 | | SWG-4A-08 | Parser: structural tree | code | 4A-07, 4A-02 | | SWG-4A-09 | Checked `ScoreBuilder` | code | 4A-08 | @@ -833,6 +833,52 @@ Acceptance: Preregistered falsification probe: *adding owned lexeme text to the level-2 token* **must be CAUGHT**. +**Delivered.** `LANGUAGE_LEVEL` is 2. `syntax/document.rs` routes on the +level and holds no grammar of its own; `parser/v2.rs` owns the level-2 +entry, `format/v2.rs` the level-2 canonical text. Level 1's entry point +gained a guard that fires only for a level it does not own, because without +it a `swang 2` header flowed into a level-1 `Program` and the formatter +emitted `swang 2` above a `pattern` block. + +**What this slice accepts, exactly:** `score { ppqn }`. `ppqn` is +`score`'s only `1` word and the structural blocks are `*`/`?`, so the +minimal empty score is a complete level-2 program rather than a stub. + +**What it refuses, and who owns the refusal.** `master_bar`, `track`, +`source` and `loss` are real `score` words (§6.4b) owned by SWG-4A-08; they +are refused here with a message that says so, never skipped, because a +parser that ignores a word it does not implement is how exact text stops +being exact. Two registry codes are implemented, `SWG0505` and `SWG0506`, +and only for `ppqn` — the scalar layer proper stays 4A-07's. The alternative +was to accept `ppqn 0960` and `ppqn 0`, text §6.6 declares invalid, into the +accepted set of a level that has not frozen. + +**The three inherited SWG-INF-06 obligations, discharged:** + +- the budget is constructed in the level-2 entry and nowhere else, and + consulted before the work it bounds — source bytes pre-lex, tokens as each + is retained, depth as the block is entered. `level_two_budget_boundary.rs` + still passes, with `parser/v2.rs` and `parser/v2/lexer.rs` added to its + exempt list one at a time and the dispatcher deliberately left off it; +- `swang_parse` gains the dispatched arm, so a budget breach reaching a + fuzzed input is a typed `SWG0509`. Two deterministic breaches drive the + declared limits through the public path with no scaling: 16 MiB of + whitespace padding a *grammatically valid* score (so it would be accepted + if the byte check were not consulted first), and four million tokens plus + one spelled under the byte cap; +- the retained level-2 token is a kind and a `Span` — twelve bytes, no + `String`, text sliced from the source. The bound is a `const` assertion in + the lexer, so it travels to every target the crate builds for; the + preregistered probe (owned lexeme text) is **CAUGHT-BY-COMPILE** on + `wasm32-unknown-unknown`. + +**Inherited by SWG-4A-08**, found by falsification rather than assumed: the +depth axis is wired but cannot breach end-to-end in a grammar with one +block. Deleting `enter_block` was caught by nothing until a witness read the +counter directly on the real parse. When the structural tree lands, the +depth cap becomes reachable from text and deserves an end-to-end breach of +its own. + ### SWG-4A-07 — Parser: exact scalar types **Kind:** code. **Depends on:** 4A-06 From 3a3e141e2201ee33aa3c02c4f75da086eb8c5f76 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 16:59:24 +0000 Subject: [PATCH 06/14] =?UTF-8?q?test(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20law=203=20at=20level=202,=20and=20the=20value=20it?= =?UTF-8?q?=20parsed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit, on 0b40859: the fuzz oracle's `Document::Score` arm checked only the root variant, so "a formatter that changes `ppqn` still passes this oracle". Verified by probe rather than by reading — replacing the interpolated `ppqn` with the literal 960 in `format_exact` **SURVIVED** the whole suite. Two reasons it survived, and both are fixed: * the fuzz arm asserted nothing about the value, where level 1's arm asserts `assert_eq!(a, b)`. It now does the same; * every level-2 fixture used `ppqn 960`, so a formatter emitting the literal 960 round-tripped through fixtures that could not tell the difference. The new case uses 1, 480 and 65535 — values the canonical text cannot supply by accident — and asserts the parsed documents are equal, not merely the same variant. This is the law-2/law-3 gap in miniature. A formatter emitting a constant *is* its own fixed point, so `format(parse(format(x))) == format(x)` held throughout; what failed was `parse(format(ast)) == ast`, which nothing asserted at level 2. P33 the formatter emits a constant ppqn, dropping the parsed value SURVIVED @ 0b40859 → CAUGHT @ this commit Also stated, for CodeRabbit's second observation — that a malformed parse can leave nesting state on a reused budget: **one budget, one parse**. A refusal returns with what it had spent, depth included, deliberately, since that is the evidence the block was entered; `parse_exact` constructs a fresh budget every time, so no production path reuses one. A future caller wanting to bound a *sequence* of parses is declaring a different bound than §5.11's per-source one, and should say so rather than inherit it by accident. The contract is now in `parse_score`'s doc instead of being relied upon. 283 swang tests green, fmt clean, clippy -D warnings exit 0, wasm32 clean, fuzz crate compile-checked. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- fuzz/fuzz_targets/swang_parse.rs | 4 +++- swang/src/syntax/parser/v2.rs | 8 ++++++++ swang/tests/level_two_dispatch.rs | 23 +++++++++++++++++++++++ 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/fuzz/fuzz_targets/swang_parse.rs b/fuzz/fuzz_targets/swang_parse.rs index 0a5b846..a50396c 100644 --- a/fuzz/fuzz_targets/swang_parse.rs +++ b/fuzz/fuzz_targets/swang_parse.rs @@ -103,7 +103,9 @@ fuzz_target!(|source: &str| { (Document::Pattern(a), Document::Pattern(b)) => { assert_eq!(a, b, "parse(format(ast)) == ast (law 3)"); } - (Document::Score(_), Document::Score(_)) => {} + (Document::Score(a), Document::Score(b)) => { + assert_eq!(a, b, "parse(format(ast)) == ast (law 3)"); + } _ => panic!("the canonical text of a document keeps its root"), } } diff --git a/swang/src/syntax/parser/v2.rs b/swang/src/syntax/parser/v2.rs index a5b9bc1..382309f 100644 --- a/swang/src/syntax/parser/v2.rs +++ b/swang/src/syntax/parser/v2.rs @@ -91,6 +91,14 @@ pub(crate) fn parse_exact(source: &str) -> Result> { /// argument a caller may weaken, because `Level2ResourceLimits` has private /// fields and `declared()` as its only production constructor. /// +/// **One budget, one parse.** A refusal returns with whatever the parse had +/// spent, depth included — deliberately, since that is the evidence the +/// block was entered — so a budget carried into a second parse would start +/// it part-spent. [`parse_exact`] constructs a fresh one every time, which +/// is why no production path can reuse one; a future caller that wants to +/// bound a *sequence* of parses is declaring a different bound than §5.11's +/// per-source one and should say so explicitly rather than by accident. +/// /// # Errors /// Exactly [`parse_exact`]'s, unwrapped from the vector. pub(crate) fn parse_score( diff --git a/swang/tests/level_two_dispatch.rs b/swang/tests/level_two_dispatch.rs index 935a014..b6a64af 100644 --- a/swang/tests/level_two_dispatch.rs +++ b/swang/tests/level_two_dispatch.rs @@ -332,11 +332,34 @@ fn the_level_two_formatter_preserves_the_level_and_is_its_own_fixed_point() { ); let reparsed = parse_document(&canonical).expect("canonical text reparses"); assert_eq!(format_document(&reparsed), canonical, "law 2: fixed point"); + assert_eq!(reparsed, document, "law 3: and it is the same document"); let Document::Score(_) = reparsed else { panic!("canonical level-2 text is still a score"); }; } +#[test] +fn a_level_two_round_trip_preserves_the_value_it_parsed() { + // Law 3, `parse(format(ast)) == ast`, which the fixed-point law does not + // imply: a formatter emitting a *constant* `ppqn` is its own fixed point + // and reparses to something that is not what it was given. A probe found + // that hole the honest way — replacing the interpolated `ppqn` with the + // literal 960 was caught by nothing, because every fixture used 960. + // + // So the values here are deliberately not the one the other fixtures use. + for ppqn in ["1", "480", "65535"] { + let source = format!("swang 2\n\nscore {{\n ppqn {ppqn}\n}}\n"); + let document = parse_document(&source).expect("a valid minimal score"); + let canonical = format_document(&document); + assert_eq!(canonical, source, "this fixture is already canonical"); + let reparsed = parse_document(&canonical).expect("canonical text reparses"); + assert_eq!( + reparsed, document, + "parse(format(ast)) == ast: the value survives the round trip" + ); + } +} + #[test] fn canonical_level_two_text_is_the_one_spelling() { let document = parse_document("swang 2\n\nscore {\nppqn 960\n}\n").expect("accepted"); From 55f989f81c0a60a3db168d660609986248f43497 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 17:02:21 +0000 Subject: [PATCH 07/14] =?UTF-8?q?test(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20three=20Codex=20findings,=20verified=20then=20pinne?= =?UTF-8?q?d?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All three checked against primary sources before writing a line, and all three hold. **P1 — a level-1 AST can carry level 2.** `ast::v1::Level::new` validates against `LANGUAGE_LEVEL`, so raising the constant silently made `Level::new(2)` succeed. `Program`'s fields are `pub`, so a caller can build one and `format` renders `swang 2` above a `pattern` root — text both the new v1 guard and `parse_document` refuse. `parse(format(ast)) == ast` is therefore broken for an AST anyone can construct. The parser guard I added closed the parse path and left the construction path open; this closes it. **P2 — the source map is in the acceptance contract and was skipped.** `docs/swang/foundation-backlog.md:800-802`: "the skeleton ships complete: a typed root enum, the source map, deterministic diagnostics, resource budgets, and formatter dispatch." `Parsed` is generic precisely so level 2 can use it. Without it a caller cannot locate a parsed level-2 construct and 4A-09's builder would have to reparse to point anywhere. **P2 — the source-byte breach points at the whole document.** Spec §5.11's breach-location table is explicit: "source bytes → the level/header span — no body token has been admitted". I passed `0..source.len()`, which highlights sixteen megabytes and calls it a location — and I quoted that same table in the PR body while violating it. The fourth finding, comparing level-2 ASTs in the fuzz round trip, is the one CodeRabbit raised independently and is already fixed at 3a3e141. RED, for the intended reasons: `E0432` on `parse_document_with_source_map`, `E0599` on `AstId::Score` and `FieldKind::Ppqn`. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/tests/level_two_dispatch.rs | 87 ++++++++++++++++++++++++++++++- 1 file changed, 85 insertions(+), 2 deletions(-) diff --git a/swang/tests/level_two_dispatch.rs b/swang/tests/level_two_dispatch.rs index b6a64af..1f673c0 100644 --- a/swang/tests/level_two_dispatch.rs +++ b/swang/tests/level_two_dispatch.rs @@ -33,8 +33,8 @@ )] use griff_swang::syntax::{ - format, format_document, header_level, parse, parse_document, Diagnostic, Document, - LANGUAGE_LEVEL, + format, format_document, header_level, parse, parse_document, parse_document_with_source_map, + AstId, Diagnostic, Document, FieldKind, FieldRef, Level, Span, LANGUAGE_LEVEL, }; /// The one level-1 program every dispatch test measures level 1 against: @@ -366,6 +366,81 @@ fn canonical_level_two_text_is_the_one_spelling() { assert_eq!(format_document(&document), MINIMAL_SCORE); } +// -- the level a constructed AST may carry ---------------------------------- + +#[test] +fn a_level_one_ast_cannot_carry_level_two() { + // The parser guard closed the *parse* path; this closes the + // *construction* path, which `Program`'s public fields leave open. + // `ast::v1::Level` validated against `LANGUAGE_LEVEL`, so raising the + // constant silently made `Level::new(2)` succeed — and a caller could + // then build a `Program` the formatter renders as `swang 2` above a + // `pattern` root. Both entry points refuse that text, so + // `parse(format(ast)) == ast` would fail for an AST anyone can build. + Level::new(1).expect("level 1 is the level this AST spells"); + for other in [0, 2, 3, LANGUAGE_LEVEL, u32::MAX] { + assert!( + other == 1 || Level::new(other).is_err(), + "the level-1 AST spells no level but 1; `Level::new({other})` must \ + not succeed merely because the build understands that level" + ); + } +} + +// -- the source map the skeleton ships with --------------------------------- + +#[test] +fn a_level_two_parse_carries_a_source_map() { + // The SWG-4A-06 contract: "the skeleton ships complete: a typed root + // enum, the source map, deterministic diagnostics, resource budgets, and + // formatter dispatch." Without it a caller cannot locate a successfully + // parsed level-2 construct, and 4A-09's builder diagnostics would have + // to reparse to point anywhere. + let parsed = parse_document_with_source_map(MINIMAL_SCORE).expect("accepted"); + let Document::Score(_) = parsed.value else { + panic!("it is a score"); + }; + let map = &parsed.source_map; + let root = map + .node_span(AstId::Score(0)) + .expect("the score block has a location"); + assert_eq!( + MINIMAL_SCORE.get(root.start as usize..root.end as usize), + Some("score {\n ppqn 960\n}"), + "the node span is the construct, from its keyword to its brace" + ); + let ppqn = map + .field_span(FieldRef::new(AstId::Score(0), FieldKind::Ppqn)) + .expect("`ppqn` has a location"); + assert_eq!( + MINIMAL_SCORE.get(ppqn.start as usize..ppqn.end as usize), + Some("960"), + "the field span is the value, not the word that introduces it" + ); +} + +#[test] +fn the_dispatched_map_and_the_dispatched_parse_agree() { + // One parser, two entry points: the mapped one is the plain one with the + // map dropped, so the two cannot drift in what they accept. + for source in [MINIMAL_SCORE, LEVEL_ONE] { + let mapped = parse_document_with_source_map(source).expect("accepted"); + let plain = parse_document(source).expect("accepted"); + assert_eq!(mapped.value, plain); + } + for source in [ + "swang 2\n\nscore {\n}\n", + "swang 3\n", + "swang 2\n\npattern x {\n}\n", + ] { + assert_eq!( + parse_document_with_source_map(source).err(), + parse_document(source).err(), + "and they refuse identically: {source:?}" + ); + } +} + // ── the inherited INF-06 obligations ──────────────────────────────────────── #[test] @@ -386,6 +461,14 @@ fn the_source_byte_budget_breaches_end_to_end_before_any_success() { "the breach names its axis and the declared limit: {}", first.message ); + // Spec §5.11's breach-location table: "source bytes -> the level/header + // span — no body token has been admitted". Pointing at `0..len` would + // highlight sixteen megabytes and call it a location. + assert_eq!( + first.span, + Span { start: 6, end: 7 }, + "the breach points at the level digits, not at the whole document" + ); } #[test] From 6d1775dab9e148897543294b6d2c23f7ec93954f Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 17:05:08 +0000 Subject: [PATCH 08/14] =?UTF-8?q?fix(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20the=20AST's=20level,=20the=20breach=20span,=20the?= =?UTF-8?q?=20map?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three Codex findings, all verified against primary sources first. **P1 — the level-1 AST now spells level 1.** `ast::v1::Level::new` compared against `LANGUAGE_LEVEL`, so raising the constant silently admitted `Level::new(2)`; `Program`'s fields are `pub`, so the mixed value was constructible and `format` rendered `swang 2` above a `pattern` root. Both entry points refuse that text, so `parse(format(ast)) == ast` failed for an AST anyone could build. The guard I added last round closed the parse path and left this one open — a narrower fix than the defect. The value is now unconstructible rather than merely unreachable. **P2 — the source-byte breach points at the level digits.** Spec §5.11: "source bytes → the level/header span — no body token has been admitted". It was `0..source.len()`, which highlights sixteen megabytes and calls it a location. Doubly worth fixing because the PR body quoted that table while the code contradicted it. **P2 — the source map the skeleton contract asked for.** `AstId::Score` and `FieldKind::Ppqn` join the enums additively; the parser records the root as its keyword through its closing brace and `ppqn` as the value rather than the word introducing it, the convention level 1's map already follows. `parse_document_with_source_map` dispatches like its unmapped twin, and each level contributes its own level's map, because a location belongs to the grammar that read it. `parse_exact` is now the mapped parser with the map dropped, so the two cannot drift in what they accept. P34 the AST validates against LANGUAGE_LEVEL again → CAUGHT P35 the breach points at the whole document again → CAUGHT P36 the parser records no score node → CAUGHT P37 the parser records no ppqn field → CAUGHT 286 swang tests green, fmt clean, clippy -D warnings exit 0, wasm32 clean, fuzz crate compile-checked. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/src/syntax.rs | 2 +- swang/src/syntax/ast/v1.rs | 18 ++++++-- swang/src/syntax/document.rs | 55 +++++++++++++++++----- swang/src/syntax/parser/v2.rs | 84 ++++++++++++++++++++++++++++------ swang/src/syntax/source_map.rs | 4 ++ swang/src/syntax/tests.rs | 8 +++- 6 files changed, 141 insertions(+), 30 deletions(-) diff --git a/swang/src/syntax.rs b/swang/src/syntax.rs index 58701f8..b668cdf 100644 --- a/swang/src/syntax.rs +++ b/swang/src/syntax.rs @@ -70,7 +70,7 @@ pub use ast::v1::{ MapRhythm, PatternDef, Program, Prune, StrategyName, StrategyPolicy, StringLiteral, Unit, }; pub use diagnostic::Diagnostic; -pub use document::{format_document, parse_document, Document}; +pub use document::{format_document, parse_document, parse_document_with_source_map, Document}; pub use format::v1::format; pub use header::{header_level, LANGUAGE_LEVEL}; pub use parser::v1::{parse, parse_with_source_map}; diff --git a/swang/src/syntax/ast/v1.rs b/swang/src/syntax/ast/v1.rs index 8a8ac0c..bbc1f45 100644 --- a/swang/src/syntax/ast/v1.rs +++ b/swang/src/syntax/ast/v1.rs @@ -73,13 +73,25 @@ impl Error for AstError {} #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Level(u32); +/// The one language level this AST spells (spec §5.4: each released level +/// owns its own parser, formatter and tree). +const LEVEL_ONE: u32 = 1; + impl Level { - /// Validates the level against this build's supported range. + /// Validates the level against the one level this AST spells. + /// + /// Level 1's AST carries level 1 and nothing else. Validating against + /// `LANGUAGE_LEVEL` instead would mean that raising the constant — which + /// SWG-4A-06 did — silently admits a `Program` whose header says + /// `swang 2` above a `pattern` root: text the formatter emits happily + /// and both entry points refuse, so `parse(format(ast)) == ast` fails + /// for an AST a caller can build from public fields. The mixed value is + /// unconstructible instead of merely unreachable. /// /// # Errors - /// [`AstError::UnsupportedLevel`] for zero or a newer level. + /// [`AstError::UnsupportedLevel`] for any level but 1. pub const fn new(level: u32) -> Result { - if level == 0 || level > LANGUAGE_LEVEL { + if level != LEVEL_ONE { return Err(AstError::UnsupportedLevel { level }); } Ok(Self(level)) diff --git a/swang/src/syntax/document.rs b/swang/src/syntax/document.rs index 0837dd8..7d48667 100644 --- a/swang/src/syntax/document.rs +++ b/swang/src/syntax/document.rs @@ -24,6 +24,7 @@ use super::format::v2::format_exact; use super::header::{header_level, LANGUAGE_LEVEL}; use super::parser; use super::parser::v2::ExactScore; +use super::source_map::Parsed; use super::span::span_of; /// A parsed Swang document, tagged by the root its level admits. @@ -49,17 +50,49 @@ pub fn parse_document(source: &str) -> Result> { match level { 1 => parser::v1::parse(source).map(Document::Pattern), 2 => parser::v2::parse_exact(source).map(Document::Score), - // Unreachable: the pre-parser already refused anything outside - // `1..=LANGUAGE_LEVEL`. Defense in depth, so that raising the - // constant without adding an arm is a refusal rather than a panic - // or, worse, a silent fall-through to the wrong grammar. - other => Err(vec![Diagnostic { - code: "SWG0001", - span: span_of(6, 6), - message: format!( - "language level {other} has no parser in this build (1..={LANGUAGE_LEVEL})" - ), - }]), + other => Err(vec![unsupported_level(other)]), + } +} + +/// Unreachable: the pre-parser already refused anything outside +/// `1..=LANGUAGE_LEVEL`. Defense in depth, so that raising the constant +/// without adding an arm is a refusal rather than a panic or, worse, a +/// silent fall-through to the wrong grammar. +fn unsupported_level(level: u32) -> Diagnostic { + Diagnostic { + code: "SWG0001", + span: span_of(6, 6), + message: format!( + "language level {level} has no parser in this build (1..={LANGUAGE_LEVEL})" + ), + } +} + +/// [`parse_document`], additionally returning the [`SourceMap`] side table. +/// +/// One router: [`parse_document`] is this function with the map dropped, so +/// the two cannot drift in what they accept or how they refuse. Each level +/// contributes its own level's map — level 1 the `Program`/`Pattern` ids it +/// already recorded, level 2 the `Score` root and its `Ppqn` field — because +/// a map is a location table, and a location belongs to the grammar that +/// read it. +/// +/// # Errors +/// Exactly [`parse_document`]'s. +/// +/// [`SourceMap`]: super::SourceMap +pub fn parse_document_with_source_map(source: &str) -> Result, Vec> { + let level = header_level(source).map_err(|d| vec![d])?; + match level { + 1 => parser::v1::parse_with_source_map(source).map(|parsed| Parsed { + value: Document::Pattern(parsed.value), + source_map: parsed.source_map, + }), + 2 => parser::v2::parse_exact_with_source_map(source).map(|parsed| Parsed { + value: Document::Score(parsed.value), + source_map: parsed.source_map, + }), + other => Err(vec![unsupported_level(other)]), } } diff --git a/swang/src/syntax/parser/v2.rs b/swang/src/syntax/parser/v2.rs index 382309f..058caa6 100644 --- a/swang/src/syntax/parser/v2.rs +++ b/swang/src/syntax/parser/v2.rs @@ -47,6 +47,7 @@ use crate::syntax::ast::v2::ExactScoreDocument; use crate::syntax::diagnostic::Diagnostic; use crate::syntax::header::HEADER_WINDOW; use crate::syntax::limits::Level2Budget; +use crate::syntax::source_map::{AstId, FieldKind, Parsed, SourceMap}; use crate::syntax::span::{span_of, Span}; /// A parsed level-2 exact score. @@ -77,9 +78,27 @@ impl ExactScore { /// a non-canonical spelling, `SWG0506` for a canonical-model invariant, and /// `SWG0401` for everything structural. pub(crate) fn parse_exact(source: &str) -> Result> { + parse_exact_with_source_map(source).map(|parsed| parsed.value) +} + +/// [`parse_exact`], additionally returning the [`SourceMap`] side table. +/// +/// One parser: [`parse_exact`] is this function with the map dropped, so the +/// two cannot drift in what they accept or how they refuse — the same shape +/// level 1 uses for the same reason. +/// +/// # Errors +/// Exactly [`parse_exact`]'s. +pub(crate) fn parse_exact_with_source_map( + source: &str, +) -> Result, Vec> { let mut budget = Level2Budget::declared(); - parse_score(source, &mut budget) - .map(ExactScore) + let mut map = SourceMap::default(); + parse_score(source, &mut budget, &mut map) + .map(|document| Parsed { + value: ExactScore(document), + source_map: map, + }) .map_err(|d| vec![d]) } @@ -104,8 +123,9 @@ pub(crate) fn parse_exact(source: &str) -> Result> { pub(crate) fn parse_score( source: &str, budget: &mut Level2Budget, + map: &mut SourceMap, ) -> Result { - budget.admit_source(source, span_of(0, source.len()))?; + budget.admit_source(source, level_span(source))?; let tokens = lex_level_two(source, body_offset(source), budget)?; let mut parser = Parser { source, @@ -113,12 +133,37 @@ pub(crate) fn parse_score( pos: 0, eof: span_of(source.len(), source.len()), budget, + map, }; let document = parser.score()?; parser.expect_end()?; Ok(document) } +/// The header's level digits: everything after `swang ` up to the line +/// break, trimmed. +/// +/// Spec §5.11's breach-location table sends a source-byte breach here — +/// "the level/header span — no body token has been admitted" — because at +/// that point nothing else has a location. `0..source.len()` would +/// highlight the whole oversized document and call that a location. +fn level_span(source: &str) -> Span { + let line_end = source + .as_bytes() + .iter() + .take(HEADER_WINDOW) + .position(|&b| b == b'\n') + .unwrap_or(source.len()); + let line = source.get(..line_end).unwrap_or_default(); + let end = line.trim_end().len(); + let start = HEADER.len().min(end); + span_of(start, end) +} + +/// The frozen §1.1 header prefix, whose length is where the level digits +/// begin. +const HEADER: &str = "swang "; + /// The first byte after the header line, matching level 1's own arithmetic. fn body_offset(source: &str) -> usize { source @@ -134,12 +179,17 @@ fn body_offset(source: &str) -> usize { /// owns them instead of claiming they are not words at all. const DEFERRED_SCORE_WORDS: &[&str] = &["master_bar", "track", "source", "loss"]; +/// The one `score` root a level-2 document holds (§5.7). Numbered like level +/// 1's ids so the map has one shape across levels. +const SCORE: AstId = AstId::Score(0); + struct Parser<'a> { source: &'a str, tokens: Vec, pos: usize, eof: Span, budget: &'a mut Level2Budget, + map: &'a mut SourceMap, } impl Parser<'_> { @@ -206,8 +256,13 @@ impl Parser<'_> { } let open = self.expect_kind(Level2TokenKind::OpenBrace, "`{`")?; self.budget.enter_block(open.span)?; - let ppqn = self.score_body(root.span)?; + let (ppqn, close) = self.score_body(root.span)?; self.budget.leave_block(); + // The construct is its keyword through its closing brace, and the + // field is the value rather than the word that introduces it — the + // convention level 1's map already follows. + self.map + .insert_node(SCORE, span_of(root.span.start as usize, close.end as usize)); Ok(ExactScoreDocument { ppqn, master_bars: Vec::new(), @@ -219,12 +274,12 @@ impl Parser<'_> { /// The `score` body up to its closing brace, returning the one value /// this slice reads. - fn score_body(&mut self, root: Span) -> Result { + fn score_body(&mut self, root: Span) -> Result<(u16, Span), Diagnostic> { let mut ppqn: Option = None; - loop { + let close = loop { let token = self.next().ok_or_else(|| self.unexpected_end())?; match token.kind { - Level2TokenKind::CloseBrace => break, + Level2TokenKind::CloseBrace => break token.span, Level2TokenKind::Word => { let word = self.text(token).to_owned(); if word != "ppqn" { @@ -237,7 +292,9 @@ impl Parser<'_> { message: "`score` takes one `ppqn` word".to_owned(), }); } - ppqn = Some(self.ppqn_value()?); + let (value, at) = self.ppqn_value()?; + self.map.insert_field(SCORE, FieldKind::Ppqn, at); + ppqn = Some(value); } Level2TokenKind::Number | Level2TokenKind::OpenBrace => { let found = self.text(token).to_owned(); @@ -247,12 +304,13 @@ impl Parser<'_> { )); } } - } - ppqn.ok_or_else(|| Diagnostic { + }; + let value = ppqn.ok_or_else(|| Diagnostic { code: "SWG0403", span: root, message: "`score` requires a `ppqn` word".to_owned(), - }) + })?; + Ok((value, close)) } /// A word `score` does not take here, said honestly: a word the grammar @@ -281,7 +339,7 @@ impl Parser<'_> { /// implemented rather than deferred because the alternative is to accept /// `ppqn 0960` and `ppqn 0` — text §6.6 declares invalid — into the /// accepted set of a level that has not frozen. - fn ppqn_value(&mut self) -> Result { + fn ppqn_value(&mut self) -> Result<(u16, Span), Diagnostic> { let token = self.expect_kind(Level2TokenKind::Number, "a `ppqn` value")?; let text = self.text(token); if text.len() > 1 && text.starts_with('0') { @@ -303,7 +361,7 @@ impl Parser<'_> { message: "`ppqn` is the tick resolution and cannot be zero".to_owned(), }); } - Ok(value) + Ok((value, token.span)) } /// One document holds one root: anything after the root's closing brace diff --git a/swang/src/syntax/source_map.rs b/swang/src/syntax/source_map.rs index 54fdef7..318ee48 100644 --- a/swang/src/syntax/source_map.rs +++ b/swang/src/syntax/source_map.rs @@ -69,6 +69,8 @@ pub enum AstId { Generate(u32), /// An `export …` pipeline step. Export(u32), + /// A level-2 `score { … }` root (SWG-4A-06). + Score(u32), } /// Which value of a construct a location belongs to. @@ -117,6 +119,8 @@ pub enum FieldKind { Format, /// The quoted `export` path literal. Path, + /// A level-2 `score { ppqn … }` tick resolution (SWG-4A-06). + Ppqn, } /// One value's address: which construct, and which of its words. diff --git a/swang/src/syntax/tests.rs b/swang/src/syntax/tests.rs index 8d7ec58..7555c28 100644 --- a/swang/src/syntax/tests.rs +++ b/swang/src/syntax/tests.rs @@ -920,6 +920,7 @@ mod level_two_budget { mod level_two_depth_accounting { use crate::syntax::limits::Level2Budget; use crate::syntax::parser::v2::parse_score; + use crate::syntax::source_map::SourceMap; use crate::syntax::span::span_of; fn budget_for(source: &str) -> Level2Budget { @@ -943,7 +944,9 @@ score { } "; let mut budget = budget_for(source); - let refusal = parse_score(source, &mut budget).expect_err("`nope` is not a `score` word"); + let mut map = SourceMap::default(); + let refusal = + parse_score(source, &mut budget, &mut map).expect_err("`nope` is not a `score` word"); assert_eq!(refusal.code, "SWG0401"); assert_eq!( budget.depth(), @@ -956,7 +959,8 @@ score { fn a_balanced_parse_returns_the_depth_it_borrowed() { let source = "swang 2\n\nscore {\n ppqn 960\n}\n".replace("\\n", "\n"); let mut budget = budget_for(&source); - parse_score(&source, &mut budget).expect("the minimal score"); + let mut map = SourceMap::default(); + parse_score(&source, &mut budget, &mut map).expect("the minimal score"); assert_eq!(budget.depth(), 0, "what was entered was left"); } } From 5645ee16c3d72b02c32a13ac7673e5100e26e1fb Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:21:59 +0000 Subject: [PATCH 09/14] =?UTF-8?q?test(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20a=20refusal=20that=20lists=20what=20it=20refuses?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Codex, on 6d1775d: the previous fix pinned `ast::v1::Level::new` to level 1 but left its refusal printing the *build's* supported range. With `LANGUAGE_LEVEL` raised to 2 the public error now reads language level 2 is not supported (1..=2) — it refuses a level and, in the same breath, lists that level as supported. Nothing in the suite noticed, because every existing assertion checks the *verdict* (`is_err`) and none reads the message. RED: for every level the constructor refuses, the message must name that level and must not present a supported range containing it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/tests/level_two_dispatch.rs | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/swang/tests/level_two_dispatch.rs b/swang/tests/level_two_dispatch.rs index 1f673c0..7c5abbb 100644 --- a/swang/tests/level_two_dispatch.rs +++ b/swang/tests/level_two_dispatch.rs @@ -387,6 +387,35 @@ fn a_level_one_ast_cannot_carry_level_two() { } } +#[test] +fn the_level_one_ast_refusal_names_the_level_it_spells() { + // Codex, on 6d1775d: pinning the constructor to level 1 left its refusal + // still printing the *build's* range. With `LANGUAGE_LEVEL` at 2 that + // renders "language level 2 is not supported (1..=2)" — a message that + // refuses a level while listing it among the supported ones. A public + // error contradicting itself is a defect on its own: it tells the caller + // the call it just lost should have succeeded, and sends them looking for + // the bug anywhere but where it is. + for level in [0_u32, 2, 3, LANGUAGE_LEVEL, u32::MAX] { + if level == 1 { + continue; + } + let refusal = Level::new(level) + .expect_err("the level-1 AST spells no level but 1") + .to_string(); + assert!( + refusal.contains(&level.to_string()), + "the refusal must name the level it rejected; got {refusal:?}" + ); + assert!( + !refusal.contains(&format!("1..={LANGUAGE_LEVEL}")), + "the level-1 AST's refusal must describe the level *it* spells, \ + not the build's supported range — a range that now contains the \ + level just refused; got {refusal:?}" + ); + } +} + // -- the source map the skeleton ships with --------------------------------- #[test] From 3ac4d2f3732243e22334b57d03d71eb1a6726c3d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:25:39 +0000 Subject: [PATCH 10/14] =?UTF-8?q?fix(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20the=20level-1=20AST's=20refusal=20says=20level=201?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The constructor was already right; only what it said about itself was wrong. `AstError::UnsupportedLevel`'s `Display` interpolated the build's `LANGUAGE_LEVEL`, so raising that constant to 2 turned the refusal of level 2 into "language level 2 is not supported (1..=2)". The two doc comments carried the same stale claim: the variant said "newer than LANGUAGE_LEVEL" and the type said "at most LANGUAGE_LEVEL", when the constraint this AST enforces — and has to enforce, per spec §5.4 — is level 1 and nothing else. The message now names the constraint the code has: the level-1 AST spells level 1; it cannot carry level {level} `LANGUAGE_LEVEL` is no longer imported here at all, which is the point: the level-1 tree does not consult the build's range to decide what it can spell. The two doc comments still link to it, now by path, to say exactly that — a build understanding a level says nothing about which tree carries it. Scope: message and prose only. The verdicts are unchanged, and the existing `a_level_one_ast_cannot_carry_level_two` still pins them. The doc-comment half of the change is prose and is not machine-pinned; the message half is, by the RED above. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/src/syntax/ast/v1.rs | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/swang/src/syntax/ast/v1.rs b/swang/src/syntax/ast/v1.rs index bbc1f45..0faada8 100644 --- a/swang/src/syntax/ast/v1.rs +++ b/swang/src/syntax/ast/v1.rs @@ -5,7 +5,6 @@ use std::fmt; use griff_pattern::{DensityBps, Traversal}; -use crate::syntax::header::LANGUAGE_LEVEL; use crate::TailPolicy; /// Why an AST value refused to exist. @@ -36,7 +35,10 @@ pub enum AstError { }, /// A unit part is zero — no note value has a zero side. ZeroUnitPart, - /// Level zero, or newer than [`LANGUAGE_LEVEL`]. + /// Any level but 1 — this is the level-1 AST, and it spells one level + /// (spec §5.4). Deliberately not the build-wide range that + /// [`LANGUAGE_LEVEL`](crate::syntax::LANGUAGE_LEVEL) reports: a build + /// understanding a level says nothing about which tree carries it. UnsupportedLevel { /// The rejected level. level: u32, @@ -60,7 +62,7 @@ impl fmt::Display for AstError { Self::ZeroUnitPart => write!(f, "a unit part is zero"), Self::UnsupportedLevel { level } => write!( f, - "language level {level} is not supported (1..={LANGUAGE_LEVEL})" + "the level-1 AST spells level 1; it cannot carry level {level}" ), } } @@ -68,8 +70,10 @@ impl fmt::Display for AstError { impl Error for AstError {} -/// A pinned language level, valid by construction: nonzero and at most -/// [`LANGUAGE_LEVEL`]. +/// A pinned language level, valid by construction: level 1, the one this +/// AST spells — not every level +/// [`LANGUAGE_LEVEL`](crate::syntax::LANGUAGE_LEVEL) admits, for the reason +/// [`Level::new`] gives. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Level(u32); From fd8370ce63e1574ead6a2e21ae190eb745e5a67c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 20:26:31 +0000 Subject: [PATCH 11/14] =?UTF-8?q?docs(swang):=20SWG-4A-06=20=E2=80=94=20wh?= =?UTF-8?q?y=20the=20level-1=20AST=20stops=20quoting=20the=20build=20range?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the decision behind the message fix: the level-1 tree asking the build what it may spell is the same error as the constructor bug, one layer out. `SWG0001` in the header stays the one place the build-wide range is named, because there the range is the subject. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- docs/decisions.log.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/decisions.log.md b/docs/decisions.log.md index 5449cee..dfcafb1 100644 --- a/docs/decisions.log.md +++ b/docs/decisions.log.md @@ -2787,3 +2787,17 @@ Architectural decisions go to [`adr/`](adr/) instead. without spending the same budget, and `Level2ResourceLimits` still has private fields and `declared()` as its only production constructor. + +- 2026-09-02 — In the context of `ast::v1::Level::new` being pinned to level + 1 while `LANGUAGE_LEVEL` rose to 2, we decided to **stop the level-1 AST's + refusal from quoting the build's supported range**, to achieve a public + error that describes the constraint the code actually enforces, accepting + that two neighbouring types now word "unsupported level" differently on + purpose. The message interpolated `LANGUAGE_LEVEL`, so refusing level 2 + read `language level 2 is not supported (1..=2)` — a sentence that lists + the level it is refusing, and sends the reader hunting for a bug anywhere + but where it is. The two doc comments carried the same stale claim. This + is the same error the constructor bug was, one layer out: the level-1 tree + asking the *build* what it may spell. It may spell level 1 (§5.4), and the + header's own `SWG0001` remains the place where the build-wide range is + named, because there the range is the subject. From 2e739340d1aebcc016c7a4fd44f293cc845db383 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 3 Sep 2026 05:27:01 +0000 Subject: [PATCH 12/14] =?UTF-8?q?test(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20the=20router=20that=20claimed=20to=20be=20one=20rou?= =?UTF-8?q?ter?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `parse_document_with_source_map`'s doc says `parse_document` "is this function with the map dropped, so the two cannot drift in what they accept or how they refuse". No such delegation exists: `document.rs` holds two independent dispatches, each calling `header_level` for itself. The sentence is true one layer down — `v1::parse` and `v2::parse_exact` really are their map-bearing twin with the map dropped — and appears to have travelled up to a module where nothing implements it. Two probes say the *behaviour* is well covered, so this is prose overclaiming rather than a live bug: P40 the plain router alone diverges on the invalid level-1 path CAUGHT — by the Law A refusal test, which guards level 1 only P41 the plain router alone diverges on the invalid level-2 path CAUGHT — 10 tests, including the sampling test below But both probes are gross divergences. What no test can observe is the structural claim itself: that there is one decision point. A behavioural test only samples inputs, and two routers agreeing on every input a suite happens to try is what a second router looks like until it isn't. RED: the level is read exactly once in `document.rs`. Lexical, because the claim is structural — the same instrument `level_two_budget_boundary.rs` and the retained-token size assertion already use for claims of this kind. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/tests/level_two_dispatch.rs | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/swang/tests/level_two_dispatch.rs b/swang/tests/level_two_dispatch.rs index 7c5abbb..816927c 100644 --- a/swang/tests/level_two_dispatch.rs +++ b/swang/tests/level_two_dispatch.rs @@ -448,6 +448,32 @@ fn a_level_two_parse_carries_a_source_map() { ); } +#[test] +fn the_router_decides_the_level_once() { + // `parse_document_with_source_map`'s doc says `parse_document` "is this + // function with the map dropped, so the two cannot drift in what they + // accept or how they refuse". That sentence describes a delegation, and + // for a while it described one that did not exist: the module held two + // independent dispatches, each reading the level for itself, and the + // property was maintained by hand rather than established by + // construction — while one layer down, `v1::parse` and `v2::parse_exact` + // really are their map-bearing twin with the map dropped, which is + // presumably where the sentence came from. + // + // This is lexical because the claim is structural. The test below can + // only sample inputs, and two routers agreeing on every input a suite + // happens to try is exactly what a second router looks like right up + // until it doesn't. One read of the level means one function owns the + // decision, and any other entry point has to go through it. + let shipped = include_str!("../src/syntax/document.rs"); + assert_eq!( + shipped.matches("header_level(").count(), + 1, + "the level is read once, by the one router; a second reader is a \ + second dispatch, free to drift from the first" + ); +} + #[test] fn the_dispatched_map_and_the_dispatched_parse_agree() { // One parser, two entry points: the mapped one is the plain one with the From 7c3c9cdedb99ebf3d6aa35f9d1987eaf8b8fbe44 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 3 Sep 2026 05:31:30 +0000 Subject: [PATCH 13/14] =?UTF-8?q?fix(swang):=20SWG-4A-06=20review=20?= =?UTF-8?q?=E2=80=94=20one=20router,=20by=20construction?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `parse_document` now is what its neighbour's doc always said it was: `parse_document_with_source_map` with the map dropped. The duplicate `match level` and the second `header_level` call are gone, so the level is decided once and the two entry points cannot drift because there are no longer two things to keep in step. P42 the second router is reintroduced, behaviour identical CAUGHT — and only by `the_router_decides_the_level_once` That probe is the argument for a lexical test. With the duplicate restored the suite ran 287 passed / 1 failed under `--no-fail-fast`: every behavioural test agreed, because a freshly duplicated router does agree. It agrees until someone edits one copy. `v2::parse_exact` was the casualty and is deleted rather than kept alive for symmetry: the router took its only caller, and `-D dead-code` is right that a `pub(crate)` wrapper with no one behind it is not an API. Level 1 keeps `v1::parse`, which has callers. The doc links that pointed at `parse_exact` now point at `parse_exact_with_source_map`, and `format/v2.rs` states its two laws over the level-2 parse rather than over a function that no longer exists. The sampling test keeps its job and gains an honest description: it samples the agreement, and the structural test is what makes that agreement more than a coincidence this corpus happens to confirm. 288 swang tests green, fmt clean, clippy -D warnings exit 0, wasm32 clean, fuzz crate compile-checked, no new rustdoc warnings. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/src/syntax/document.rs | 57 ++++++++++++++++--------------- swang/src/syntax/format/v2.rs | 4 +-- swang/src/syntax/parser/v2.rs | 27 ++++++--------- swang/tests/level_two_dispatch.rs | 12 ++++--- 4 files changed, 50 insertions(+), 50 deletions(-) diff --git a/swang/src/syntax/document.rs b/swang/src/syntax/document.rs index 7d48667..0aae778 100644 --- a/swang/src/syntax/document.rs +++ b/swang/src/syntax/document.rs @@ -40,16 +40,34 @@ pub enum Document { Score(ExactScore), } -/// Parses a source at whichever level its header pins. +/// Parses a source at whichever level its header pins, returning the +/// [`SourceMap`] side table alongside it. +/// +/// **The one router.** Every level decision in this build is this call: +/// [`parse_document`] is this function with the map dropped, so the two +/// cannot drift in what they accept or how they refuse — not by discipline, +/// but because there is only one of them to keep in step. Each level then +/// contributes its own level's map — level 1 the `Program`/`Pattern` ids it +/// already recorded, level 2 the `Score` root and its `Ppqn` field — because +/// a map is a location table, and a location belongs to the grammar that +/// read it. /// /// # Errors /// The frozen §1.1 pre-parser's diagnostics for a header this build cannot /// read, and otherwise exactly the chosen level's own. -pub fn parse_document(source: &str) -> Result> { +/// +/// [`SourceMap`]: super::SourceMap +pub fn parse_document_with_source_map(source: &str) -> Result, Vec> { let level = header_level(source).map_err(|d| vec![d])?; match level { - 1 => parser::v1::parse(source).map(Document::Pattern), - 2 => parser::v2::parse_exact(source).map(Document::Score), + 1 => parser::v1::parse_with_source_map(source).map(|parsed| Parsed { + value: Document::Pattern(parsed.value), + source_map: parsed.source_map, + }), + 2 => parser::v2::parse_exact_with_source_map(source).map(|parsed| Parsed { + value: Document::Score(parsed.value), + source_map: parsed.source_map, + }), other => Err(vec![unsupported_level(other)]), } } @@ -68,32 +86,17 @@ fn unsupported_level(level: u32) -> Diagnostic { } } -/// [`parse_document`], additionally returning the [`SourceMap`] side table. +/// [`parse_document_with_source_map`] with the map dropped. /// -/// One router: [`parse_document`] is this function with the map dropped, so -/// the two cannot drift in what they accept or how they refuse. Each level -/// contributes its own level's map — level 1 the `Program`/`Pattern` ids it -/// already recorded, level 2 the `Score` root and its `Ppqn` field — because -/// a map is a location table, and a location belongs to the grammar that -/// read it. +/// The shape level 1's own pair already has — `v1::parse` is written this +/// way too — so the map costs a caller who does not want it exactly what it +/// costs them one layer down, and dispatch cannot be changed for one entry +/// point without being changed for the other. /// /// # Errors -/// Exactly [`parse_document`]'s. -/// -/// [`SourceMap`]: super::SourceMap -pub fn parse_document_with_source_map(source: &str) -> Result, Vec> { - let level = header_level(source).map_err(|d| vec![d])?; - match level { - 1 => parser::v1::parse_with_source_map(source).map(|parsed| Parsed { - value: Document::Pattern(parsed.value), - source_map: parsed.source_map, - }), - 2 => parser::v2::parse_exact_with_source_map(source).map(|parsed| Parsed { - value: Document::Score(parsed.value), - source_map: parsed.source_map, - }), - other => Err(vec![unsupported_level(other)]), - } +/// Exactly [`parse_document_with_source_map`]'s. +pub fn parse_document(source: &str) -> Result> { + parse_document_with_source_map(source).map(|parsed| parsed.value) } /// Emits the canonical text of a document, at its own level. diff --git a/swang/src/syntax/format/v2.rs b/swang/src/syntax/format/v2.rs index a54af4c..49f983e 100644 --- a/swang/src/syntax/format/v2.rs +++ b/swang/src/syntax/format/v2.rs @@ -14,8 +14,8 @@ use crate::syntax::parser::v2::ExactScore; /// Emits the one canonical text for `score`. /// -/// `format_exact(parse_exact(t))` is idempotent and -/// `parse_exact(format_exact(s))` recovers the same score. +/// `format_exact(parse(t))` is idempotent and `parse(format_exact(s))` +/// recovers the same score, for the level-2 parse. pub(crate) fn format_exact(score: &ExactScore) -> String { // Destructured with no `..` on purpose: when SWG-4A-08 starts filling // the structural slots, this function stops compiling until it learns to diff --git a/swang/src/syntax/parser/v2.rs b/swang/src/syntax/parser/v2.rs index 058caa6..17bbe67 100644 --- a/swang/src/syntax/parser/v2.rs +++ b/swang/src/syntax/parser/v2.rs @@ -69,26 +69,20 @@ impl ExactScore { } } -/// Parses a level-2 source. The caller has already proved the header says -/// `swang 2`; this function owns everything after it. +/// Parses a level-2 source, returning the [`SourceMap`] side table with it. +/// The caller has already proved the header says `swang 2`; this function +/// owns everything after it. +/// +/// The level-2 entry point, singular. Level 1 additionally offers `v1::parse` +/// as its map-dropping wrapper because callers of the frozen level want one; +/// level 2's only caller is the router, which takes the map, so a wrapper +/// here would be a second entry with no one behind it. /// /// # Errors /// One diagnostic. `SWG0509` for a declared budget breach, `SWG0403` for a /// missing required word, `SWG0404` for a repeated singleton, `SWG0505` for /// a non-canonical spelling, `SWG0506` for a canonical-model invariant, and /// `SWG0401` for everything structural. -pub(crate) fn parse_exact(source: &str) -> Result> { - parse_exact_with_source_map(source).map(|parsed| parsed.value) -} - -/// [`parse_exact`], additionally returning the [`SourceMap`] side table. -/// -/// One parser: [`parse_exact`] is this function with the map dropped, so the -/// two cannot drift in what they accept or how they refuse — the same shape -/// level 1 uses for the same reason. -/// -/// # Errors -/// Exactly [`parse_exact`]'s. pub(crate) fn parse_exact_with_source_map( source: &str, ) -> Result, Vec> { @@ -113,13 +107,14 @@ pub(crate) fn parse_exact_with_source_map( /// **One budget, one parse.** A refusal returns with whatever the parse had /// spent, depth included — deliberately, since that is the evidence the /// block was entered — so a budget carried into a second parse would start -/// it part-spent. [`parse_exact`] constructs a fresh one every time, which +/// it part-spent. [`parse_exact_with_source_map`] constructs a fresh one +/// every time, which /// is why no production path can reuse one; a future caller that wants to /// bound a *sequence* of parses is declaring a different bound than §5.11's /// per-source one and should say so explicitly rather than by accident. /// /// # Errors -/// Exactly [`parse_exact`]'s, unwrapped from the vector. +/// Exactly [`parse_exact_with_source_map`]'s, unwrapped from the vector. pub(crate) fn parse_score( source: &str, budget: &mut Level2Budget, diff --git a/swang/tests/level_two_dispatch.rs b/swang/tests/level_two_dispatch.rs index 816927c..c202dc8 100644 --- a/swang/tests/level_two_dispatch.rs +++ b/swang/tests/level_two_dispatch.rs @@ -456,9 +456,9 @@ fn the_router_decides_the_level_once() { // for a while it described one that did not exist: the module held two // independent dispatches, each reading the level for itself, and the // property was maintained by hand rather than established by - // construction — while one layer down, `v1::parse` and `v2::parse_exact` - // really are their map-bearing twin with the map dropped, which is - // presumably where the sentence came from. + // construction — while one layer down, `v1::parse` really is its + // map-bearing twin with the map dropped, which is presumably where the + // sentence came from. // // This is lexical because the claim is structural. The test below can // only sample inputs, and two routers agreeing on every input a suite @@ -476,8 +476,10 @@ fn the_router_decides_the_level_once() { #[test] fn the_dispatched_map_and_the_dispatched_parse_agree() { - // One parser, two entry points: the mapped one is the plain one with the - // map dropped, so the two cannot drift in what they accept. + // One parser, two entry points: the plain one is the mapped one with the + // map dropped, so the two cannot drift in what they accept. This samples + // that agreement; `the_router_decides_the_level_once` is what makes it + // structural rather than a coincidence this corpus happens to confirm. for source in [MINIMAL_SCORE, LEVEL_ONE] { let mapped = parse_document_with_source_map(source).expect("accepted"); let plain = parse_document(source).expect("accepted"); From 08285d59075ac765981776ab7e95da51b909e937 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 3 Sep 2026 05:31:42 +0000 Subject: [PATCH 14/14] =?UTF-8?q?docs(swang):=20SWG-4A-06=20=E2=80=94=20on?= =?UTF-8?q?e=20level=20decision,=20and=20why=20the=20witness=20is=20lexica?= =?UTF-8?q?l?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records why the duplicate router was invisible to 287 behavioural tests: a freshly copied router agrees with its original, so only the structure distinguishes them. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- docs/decisions.log.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/decisions.log.md b/docs/decisions.log.md index dfcafb1..d3dfedc 100644 --- a/docs/decisions.log.md +++ b/docs/decisions.log.md @@ -2801,3 +2801,17 @@ Architectural decisions go to [`adr/`](adr/) instead. asking the *build* what it may spell. It may spell level 1 (§5.4), and the header's own `SWG0001` remains the place where the build-wide range is named, because there the range is the subject. + +- 2026-09-03 — In the context of `syntax/document.rs` documenting a + delegation it did not implement, we decided to **make `parse_document` + literally `parse_document_with_source_map` with the map dropped**, to + achieve one level decision per build rather than two kept in step by hand, + accepting that a caller wanting no map still pays for one — the price + `v1::parse` already pays a layer down. The module held two `match level` + blocks while its doc said the two "cannot drift"; the property was real + but maintained by discipline. Reintroducing the duplicate is invisible to + behaviour — 287 tests agreed with it — because a fresh copy of a router + does agree, and agrees right up until someone edits one copy. So the + witness is lexical: the level is read once in that file. `v2::parse_exact` + lost its only caller and is deleted rather than kept for symmetry, since a + `pub(crate)` wrapper with nobody behind it is not an API.