SWG-INF-04: replace ProgramSpans with a full SourceMap

docs/decisions.log.md

@@ -2477,3 +2477,33 @@ Architectural decisions go to [`adr/`](adr/) instead.
   reach the evaluator at all: `mod ast` is private to `syntax`, so naming
   the type from outside is `E0603`. Accepting that the bullet stays open in
   the backlog until 4A-09 closes it.
+
+- 2026-08-29 — In the context of SWG-INF-04, facing the question of what an
+  `AstId` promises, we decided that it is a **parse-local structural
+  handle** and against any persistence guarantee, to achieve a source map
+  useful to diagnostics and editors today without pre-deciding a question
+  Phase 4C owns, accepting that an edit which inserts, deletes or reorders
+  repeated nodes may renumber every id after it. What is guaranteed: for two
+  sources that parse to equal ASTs, the maps have the same node-key set and
+  the same field-key set, though the spans differ. That is deterministic
+  under whitespace and under §3.2's legal word reordering, which is the
+  stability an editor actually needs to re-anchor after a reformat. What is
+  not guaranteed, stated so nobody has to guess: identity across
+  AST-changing edits, patch identity, a serialized form, a semantic-hash
+  input, or anything about Phase 4C selector identity. Pretending otherwise
+  would make 4C's real problem look solved by a side table that never
+  addressed it.
+
+- 2026-08-29 — In the context of SWG-INF-04's design, facing three pieces of
+  prior art, we decided to adopt two and refuse one, to achieve a source map
+  that is boring in the ways that matter. From `rustc`: byte-offset
+  locations, with line and column resolved only at render time, so nothing
+  stores a line number as semantic state. From `rust-analyzer`'s `AstIdMap`:
+  the separation between structural identity and position-dependent
+  location — the side-table architecture itself. From `rowan`'s
+  `SyntaxNodePtr`: the idea is sound prior art for transient source
+  pointers, but a lossless CST is **not** adopted, because SWG-UI-07 owns
+  that admission gate and it requires three demonstrated needs that have not
+  been demonstrated. The borrowed architecture stops short of
+  `rust-analyzer`'s incremental-IDE identity guarantees, which is the
+  distinction the entry above exists to make explicit.

docs/swang/foundation-backlog.md

@@ -107,7 +107,7 @@ recorded in `decisions.log.md` if reversed.
 | SWG-INF-01 | Sync the S16 status block with reality *(done)* | docs | — |
 | SWG-INF-02 | Language level 2 admission contract *(done)* | docs | INF-03 |
 | SWG-INF-03 | Split `syntax.rs` without behaviour change *(done)* | code | INF-01 |
-| SWG-INF-04 | Replace `ProgramSpans` with a `SourceMap` | code | INF-03 |
+| SWG-INF-04 | Replace `ProgramSpans` with a `SourceMap` *(done)* | code | INF-03 |
 | SWG-INF-05 | Deterministic multi-error recovery | code | INF-04 |
 | SWG-INF-06 | Parser resource gate and differential harness | code | INF-02, INF-03 |
 | SWG-4A-01 | Normative exact-score-text grammar *(done)* | docs | INF-02 |
@@ -288,40 +288,78 @@ than a description of something already coded: no `v2` module, no
 `Level2Root`, no `GrammarVersion`, no dispatch stub, no `Parsed<T>`, no
 `SourceMap`, no recovery, no limits, no public token API.
 
-### SWG-INF-04 — Replace `ProgramSpans` with a `SourceMap`
+### SWG-INF-04 — Replace `ProgramSpans` with a `SourceMap` *(done)*
 
 **Kind:** code. **Depends on:** INF-03
 
 Four spans cannot locate a diagnostic in an exact score text, cannot anchor
-a patch selector, and cannot drive an editor action. Shape:
+a patch selector, and cannot drive an editor action. As built:
 
 ```text
 struct SourceMap {
-    nodes:  BTreeMap<AstId, Span>,
-    fields: BTreeMap<FieldRef, Span>,
+    nodes:  BTreeMap<AstId, Span>,     // private
+    fields: BTreeMap<FieldRef, Span>,  // private
 }
 
-enum AstId    { Program(u32), Pattern(u32), PipelineStep(u32), Generate(u32) }
-enum FieldRef { Node(AstId), Named { node: AstId, field: FieldKind } }
+enum AstId    { Program(u32), Pattern(u32), Fractalize(u32), Linearize(u32),
+                MapRhythm(u32), Generate(u32), Export(u32) }
+struct FieldRef { node: AstId, field: FieldKind }
 ```
 
-Requirements:
+Two departures from this entry's original sketch, both deliberate. `AstId`
+gets one variant per construct rather than a shared `PipelineStep`, so a
+field's owner is named rather than inferred from an ordinal. And `FieldRef`
+has no `Node` variant: `SourceMap::node_span` already owns that relation,
+and two ways to ask one question is one too many. Both enums are
+`#[non_exhaustive]` so level 2 appends variants without breaking a match.
+
+`FieldKind` is not unique on its own — `Seed` names both the pruning and the
+generation seed, and the owning `AstId` tells them apart. A variant per
+(construct, word) pair would grow quadratically and say nothing the pair
+does not already say.
+
+Requirements, all met:
 
 - spans stay out of AST equality — `parse(format(ast)) == ast` still holds;
-- the formatter can consume an AST with no source map at all;
+- the formatter consumes an AST with no source map at all, proved by
+  building a `Program` in Rust with no source text anywhere and reparsing
+  its output;
 - the parser returns `Parsed<T> { value, source_map }`;
-- a diagnostic points at the value the user must change, not at the
-  statement containing it;
-- every semantically significant level-1 field has a span;
+- a diagnostic points at the value, not the statement containing it;
+- every semantically significant level-1 field has a span — seven nodes and
+  eighteen fields in the §3.1 reference, fifteen when pruning and `corpus`
+  are absent;
 - every span lies on a UTF-8 boundary inside the source.
 
 Acceptance:
 
-- the four existing span tests pass through the new model;
-- a witness test enumerates every level-1 AST field and asserts a span for
-  each — adding a field without a span fails to compile or fails the test;
-- reordering words in the source moves the owning span with the value;
-- the spec §3.5 formatter laws are unchanged.
+- the existing span tests pass through the new model. There were **two**,
+  not the four this entry claimed, and they are carried across as
+  characterization rather than rewritten;
+- a witness enumerates every level-1 AST field by exhaustive destructuring
+  with no `..`, so a new AST field without a location classification stops
+  the suite compiling;
+- reordering words moves the owning span with the value while the node and
+  field key sets stay identical — the useful stability guarantee, stated in
+  the decision log;
+- the spec §3.5 formatter laws are unchanged, and so are the four
+  diagnostic locations §3.5 released. The map now knows `bars`,
+  `candidates` and `strategy`; that is editor capability, not permission to
+  move a diagnostic somebody's tooling already parses;
+- eleven mutations, none survived. Three of them were run first against
+  the suite as it stood at closure and survived it — a node span stretched
+  back to byte 0, and two swapped arms of the evaluator's flaw-to-location
+  match — which is how the two review witnesses earned their place rather
+  than merely occupying it.
+
+`ProgramSpans` and `parse_with_spans` are removed rather than wrapped: a
+compatibility shim would have kept the four-field model alive indefinitely,
+which is the thing this task exists to end.
+
+What this task did **not** do, and must not be read as having done: no
+level-2 parsing, no level/root dispatch, no multi-error recovery, no
+resource limits, no token API, no persistent identity. `LANGUAGE_LEVEL`
+stays 1 and a `swang 2` source still takes the unsupported-level path.
 
 ### SWG-INF-05 — Deterministic multi-error recovery
 
@@ -1304,9 +1342,10 @@ INF-01 status sync                    (done)
                 │   surface over it, not another slice of it
                 │
                 └─→ 4A-02 → INF-04 → INF-06 → 4A-06 parser skeleton
-                      (done)   ↑
-                               next — INF-04 can now design SourceMap,
-                               AstId and FieldRef against a real v2 shape
+                      (done)   (done)   ↑
+                                        next — level 2's input bounds must
+                                        be declared before its first
+                                        accepted program (§5.11)
   -> 4A-02..4A-09 writer / parser / builder
   -> 4A-10..4A-14 dump / verify / laws / fuzz
   -> 4B corpus acceptance

swang/src/eval.rs

@@ -28,17 +28,17 @@ use crate::pattern_compile::{
     compile_pattern_flaws, PatternFlaw, PatternPlan, RhythmPatternArgs, TailChoice, TraversalChoice,
 };
 use crate::syntax::{
-    self, Diagnostic, ExportFormat, PatternDef, Program, ProgramSpans, Span, StrategyName,
-    StrategyPolicy,
+    self, AstId, Diagnostic, ExportFormat, FieldKind, FieldRef, PatternDef, Program, SourceMap,
+    Span, StrategyName, StrategyPolicy,
 };
 
-/// A statically-checked program: the parsed AST plus the source spans a
+/// A statically-checked program: the parsed AST plus the source locations a
 /// frontend renders diagnostics at. Text in, structure out — nothing
 /// resolved, nothing run.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub struct CompiledProgram {
     program: Program,
-    spans: ProgramSpans,
+    source_map: SourceMap,
 }
 
 impl CompiledProgram {
@@ -48,10 +48,15 @@ impl CompiledProgram {
         &self.program
     }
 
-    /// The source-span table.
+    /// The source-location side table (SWG-INF-04).
+    ///
+    /// The whole map, not a four-word projection of it: a frontend that
+    /// wants to underline `bars` no longer needs a new parser field to do
+    /// it. What this does **not** license is moving a diagnostic §3.5
+    /// already released — see [`flaw_to_diagnostic`].
     #[must_use]
-    pub const fn spans(&self) -> &ProgramSpans {
-        &self.spans
+    pub const fn source_map(&self) -> &SourceMap {
+        &self.source_map
     }
 
     /// The seed-score path the program declares (`generate { source … }`).
@@ -170,8 +175,11 @@ impl EvaluationResult {
 /// # Errors
 /// The parser's span diagnostics (`SWG0001`–`SWG0404`), never empty on `Err`.
 pub fn compile_program(source: &str) -> Result<CompiledProgram, Vec<Diagnostic>> {
-    let (program, spans) = syntax::parse_with_spans(source)?;
-    Ok(CompiledProgram { program, spans })
+    let parsed = syntax::parse_with_source_map(source)?;
+    Ok(CompiledProgram {
+        program: parsed.value,
+        source_map: parsed.source_map,
+    })
 }
 
 /// Runs a compiled program's pattern pipeline up to `map_rhythm`.
@@ -190,7 +198,7 @@ pub fn expand_program(
     let args = rhythm_args(pattern);
     let bars = clamp_bars(pattern.generate.bars);
     compile_pattern_flaws(&args, source_score, bars)
-        .map_err(|flaw| vec![flaw_to_diagnostic(flaw, &compiled.spans)])
+        .map_err(|flaw| vec![flaw_to_diagnostic(flaw, &compiled.source_map)])
 }
 
 /// Runs a compiled program end to end against resolved inputs.
@@ -238,7 +246,11 @@ pub fn evaluate_program(
     .map_err(|e| {
         vec![EvalDiagnostic {
             code: "SWG0310",
-            location: DiagLocation::Span(compiled.spans.source),
+            location: DiagLocation::Span(released(
+                &compiled.source_map,
+                AstId::Generate(0),
+                FieldKind::Source,
+            )),
             message: format!("the source score cannot seed generation: {e:?}"),
         }]
     })?;
@@ -328,30 +340,50 @@ const fn strategy_kind(name: StrategyName) -> GenerationStrategy {
     }
 }
 
+/// One of the four locations §3.5 released, resolved from the map.
+///
+/// The map is total over a well-formed program — the source-map contract
+/// suite proves all eighteen level-1 fields are present — so the fallback
+/// is unreachable. It exists rather than a panic because a location bug
+/// should degrade to a worse message, not to a crash in a frontend.
+fn released(map: &SourceMap, node: AstId, field: FieldKind) -> Span {
+    map.field_span(FieldRef::new(node, field))
+        .or_else(|| map.node_span(AstId::Program(0)))
+        .unwrap_or(Span { start: 0, end: 0 })
+}
+
 /// Maps a pattern-compilation flaw to a layered [`EvalDiagnostic`] (spec
 /// §1.5): structural breaches keep their `NodePath`, score-borne facts sit at
 /// the `source` word, time-domain flaws at the value that must change.
-fn flaw_to_diagnostic(flaw: PatternFlaw, spans: &ProgramSpans) -> EvalDiagnostic {
+///
+/// SWG-INF-04 widened what the parser can locate; it deliberately did not
+/// widen what this function points at. Each arm resolves exactly the word
+/// §3.5 already names, and the richer map is editor capability rather than
+/// permission to move a released diagnostic.
+fn flaw_to_diagnostic(flaw: PatternFlaw, map: &SourceMap) -> EvalDiagnostic {
     let at = |span: Span, code: &'static str, message: String| EvalDiagnostic {
         code,
         location: DiagLocation::Span(span),
         message,
     };
+    let kernel = released(map, AstId::Pattern(0), FieldKind::Kernel);
+    let unit = released(map, AstId::MapRhythm(0), FieldKind::Unit);
+    let source = released(map, AstId::Generate(0), FieldKind::Source);
     match flaw {
-        PatternFlaw::Kernel(d) | PatternFlaw::Density(d) => at(spans.kernel, d.code, d.message),
-        PatternFlaw::Unit(d) => at(spans.unit, d.code, d.message),
-        PatternFlaw::Score(d) => at(spans.source, d.code, d.message),
+        PatternFlaw::Kernel(d) | PatternFlaw::Density(d) => at(kernel, d.code, d.message),
+        PatternFlaw::Unit(d) => at(unit, d.code, d.message),
+        PatternFlaw::Score(d) => at(source, d.code, d.message),
         PatternFlaw::Budget(e) => budget_diagnostic(&e),
-        PatternFlaw::Lower(e) => lower_diagnostic(&e, spans),
+        PatternFlaw::Lower(e) => lower_diagnostic(&e, map),
         PatternFlaw::SilentExpansion => at(
-            spans.kernel,
+            kernel,
             "SWG0306",
             "the expansion produced no onsets — nothing to generate (change the kernel, \
              depth, density, or rhythm seed)"
                 .to_owned(),
         ),
         PatternFlaw::SilentWindow { used } => at(
-            spans.kernel,
+            kernel,
             "SWG0306",
             format!(
                 "the first {used} template(s) the bars window rotates over are all silent — \
@@ -392,27 +424,29 @@ fn budget_diagnostic(e: &griff_pattern::PatternError) -> EvalDiagnostic {
 }
 
 /// A time-domain lowering flaw at the value that must change.
-fn lower_diagnostic(e: &crate::LowerError, spans: &ProgramSpans) -> EvalDiagnostic {
+fn lower_diagnostic(e: &crate::LowerError, map: &SourceMap) -> EvalDiagnostic {
+    let unit_span = released(map, AstId::MapRhythm(0), FieldKind::Unit);
+    let tail_span = released(map, AstId::MapRhythm(0), FieldKind::Tail);
     match e {
         crate::LowerError::UnitDoesNotDivideBar { bar_duration, unit } => EvalDiagnostic {
             code: "SWG0301",
-            location: DiagLocation::Span(spans.unit),
+            location: DiagLocation::Span(unit_span),
             message: format!(
                 "unit {} does not divide the {}-tick bar exactly",
                 unit.0, bar_duration.0
             ),
         },
         crate::LowerError::ZeroUnit => EvalDiagnostic {
             code: "SWG0301",
-            location: DiagLocation::Span(spans.unit),
+            location: DiagLocation::Span(unit_span),
             message: "the rhythm unit is zero ticks".to_owned(),
         },
         crate::LowerError::IncompleteFinalBar {
             have_slots,
             slots_per_bar,
         } => EvalDiagnostic {
             code: "SWG0302",
-            location: DiagLocation::Span(spans.tail),
+            location: DiagLocation::Span(tail_span),
             message: format!(
                 "the final bar holds {have_slots} of {slots_per_bar} slots; a rest_pad tail \
                  pads it with timed rests"

swang/src/syntax.rs

@@ -59,6 +59,7 @@ mod format;
 mod header;
 mod lexer;
 mod parser;
+mod source_map;
 mod span;
 mod token;
 
@@ -69,7 +70,8 @@ pub use ast::v1::{
 pub use diagnostic::Diagnostic;
 pub use format::v1::format;
 pub use header::{header_level, LANGUAGE_LEVEL};
-pub use parser::v1::{parse, parse_with_spans, ProgramSpans};
+pub use parser::v1::{parse, parse_with_source_map};
+pub use source_map::{AstId, FieldKind, FieldRef, Parsed, SourceMap};
 pub use span::Span;
 #[cfg(test)]
 #[allow(