From e3852cc629ea49d4b1c1f73502a48005af6c9824 Mon Sep 17 00:00:00 2001 From: Roger-luo Date: Thu, 16 Jul 2026 23:47:52 -0400 Subject: [PATCH] docs(design): document traits 2 refactor --- docs/design/tableau-data-structure.md | 329 ++++++++++++ .../traits-2-configuration-and-hashing.md | 508 ++++++++++++++++++ docs/design/word-data-structures.md | 357 ++++++++++++ 3 files changed, 1194 insertions(+) create mode 100644 docs/design/tableau-data-structure.md create mode 100644 docs/design/traits-2-configuration-and-hashing.md create mode 100644 docs/design/word-data-structures.md diff --git a/docs/design/tableau-data-structure.md b/docs/design/tableau-data-structure.md new file mode 100644 index 00000000..7b283359 --- /dev/null +++ b/docs/design/tableau-data-structure.md @@ -0,0 +1,329 @@ +# Contiguous tableau data structure + +Status: design sketch + +## Purpose + +The tableau should be a specialized bit-matrix data structure. It should not +be represented as `Vec>`, and its internal rows should not +affect the shared PPVM trait system. + +This design separates: + +- the logical stabilizer-tableau model; +- its physical, contiguous memory layout; +- orientation changes used to accelerate different operations; and +- structural hashing used when a tableau is a classical-mixture key. + +Gate, noise, measurement, reset, and `Indexable` traits observe tableau +behavior. They do not expose matrix blocks, row values, strides, alignment, or +orientation. + +## Goals + +- Store the X/Z tableau bits in contiguous, aligned, bit-packed memory. +- Make one- and two-qubit column operations efficient. +- Permit temporary transposition for row-oriented elimination and collapse. +- Keep phases and optional loss state independently addressable and hashable. +- Hash the logical tableau independently of its current physical orientation. +- Avoid parameterizing the trait system by a row type or Pauli-word storage. +- Leave room for SIMD-width and padding changes without changing public + behavioral traits. + +## Non-goals + +- Reusing the standalone `PauliWord` representation for tableau rows. +- Exposing borrowed tableau rows as the primary public interface. +- Standardizing a general-purpose matrix trait in `ppvm-traits-2`. +- Maintaining both row-major and column-major copies before benchmarks show + that the memory and synchronization cost is worthwhile. +- Deciding in this document whether PPVM should store a forward or inverse + tableau. Orientation and inversion are independent design choices. + +## Logical model + +For `n` qubits, a stabilizer/destabilizer tableau has `2n` generators. Each +generator contains an X bit and a Z bit for every qubit, plus a phase: + +```text + qubit + 0 1 2 ... n-1 +generator 0 x/z x/z x/z +generator 1 x/z x/z x/z + ... +generator 2n-1 x/z x/z x/z + +phase one value per generator +``` + +The logical state consists of: + +- an X matrix of shape `(2n, n)`; +- a Z matrix of shape `(2n, n)`; +- a phase plane of length `2n`; and +- for lossy variants, a loss plane of length `n` or another explicitly defined + logical shape. + +This model does not imply a physical row object. + +## Physical storage + +The first prototype should use one aligned contiguous allocation for the bit +planes, divided by computed offsets: + +```rust +pub struct TableauData { + blocks: AlignedVec, + x_offset: usize, + z_offset: usize, + phase_offset: usize, + loss_offset: Option, + major_stride: usize, + n_qubits: usize, + orientation: Orientation, +} +``` + +`Block` is an internal implementation choice such as `u64` or a SIMD-width +block. It is not an associated type of a public tableau trait. Offsets and +strides account for alignment and padding, while logical accessors enforce the +actual `(2n, n)` dimensions. + +Keeping all planes in one allocation improves cloning and locality and avoids +one allocation per generator. It also lets a mixture branch copy a single +contiguous region. If benchmarks favor separate aligned allocations for X and +Z, that change remains internal to `TableauData`. + +Padding must either be kept zero or excluded from equality and hashing. Zeroed +padding is preferable because it permits bulk comparison and hashing of +canonical ranges. + +## Column-major orientation + +The default orientation should make the generator dimension contiguous for a +fixed qubit. In other words, the X and Z planes are column-major with respect +to the logical `(generator, qubit)` matrix: + +```text +qubit 0: X bits for generators 0..2n, then padding +qubit 1: X bits for generators 0..2n, then padding +... + +qubit 0: Z bits for generators 0..2n, then padding +qubit 1: Z bits for generators 0..2n, then padding +... +``` + +This layout makes a selected qubit column contiguous. Single-qubit gates can +load the X and Z columns, update them with bitwise operations, and update the +affected phase bits. Two-qubit gates operate on two pairs of contiguous +columns. Measurement can scan the selected anticommutation column without +stepping across separately allocated row objects. + +Stim uses aligned SIMD bit tables and documents its tableau layout as +column-major, with output-observable iteration following contiguous memory. It +also provides an explicit quadrant transpose and a transposition guard for +operations that need the opposite orientation: + +- [Stim `Tableau`](https://github.com/quantumlib/Stim/blob/main/src/stim/stabilizers/tableau.h) +- [Stim `TableauSimulator`](https://github.com/quantumlib/Stim/blob/main/src/stim/simulators/tableau_simulator.h) +- [Stim `simd_bit_table`](https://github.com/quantumlib/Stim/blob/main/src/stim/mem/simd_bit_table.h) + +Column-major storage is only one part of Stim's performance strategy. PPVM +should benchmark its own gate, measurement, and sampling workloads instead of +assuming the same total performance from layout alone. + +## Temporary transposition + +Row multiplication and elimination need long generator rows and therefore +prefer the opposite orientation. The initial design should transpose the X/Z +quadrants temporarily instead of storing two permanent copies: + +```rust +pub enum Orientation { + ColumnMajor, + RowMajor, +} + +pub struct TransposedTableau<'a> { + tableau: &'a mut Tableau, +} +``` + +Creating the guard transposes the bit matrices and marks the physical +orientation. Dropping it restores the canonical column-major orientation: + +```rust +impl Drop for TransposedTableau<'_> { + fn drop(&mut self) { + self.tableau.restore_column_major(); + } +} +``` + +Operations that require row-major access receive the guard, making the +orientation precondition explicit. Public methods return with the tableau in +canonical orientation. Panic safety must be preserved by the guard's `Drop` +implementation. + +Transposition is a physical reordering of the same logical bits. It does not +invalidate structural hashes or change equality. Hashing should occur only +through logical access or while the tableau is in its public canonical state. + +Maintaining both orientations with dirty flags remains a future alternative +for workloads that switch frequently enough to amortize the doubled storage. + +## Tableau API boundary + +The public API exposes logical operations instead of storage slices: + +```rust +impl Tableau { + pub fn n_qubits(&self) -> usize; + + pub fn h(&mut self, qubit: usize); + pub fn cnot(&mut self, control: usize, target: usize); + pub fn measure_z(&mut self, qubit: usize) -> bool; + + pub fn x_bit(&self, generator: usize, qubit: usize) -> bool; + pub fn z_bit(&self, generator: usize, qubit: usize) -> bool; + pub fn phase(&self, generator: usize) -> Phase; +} +``` + +Logical bit accessors are useful for tests, serialization, debugging, and +interoperability. They are not intended as the hot gate implementation path. +Bulk import/export methods may use a canonical serialized representation +without exposing the in-memory layout. + +There should be no `stabilizers_mut() -> &mut [Row]` or equivalent escape hatch +that bypasses hash invalidation and orientation invariants. Specialized +internal row operations operate through `TableauData` or a transposition guard. + +## Gate access patterns + +The implementation should categorize mutations by physical access and hash +effect: + +| Operation | Preferred access | X/Z changed | Phase changed | +| --- | --- | --- | --- | +| Pauli `X`, `Y`, `Z` | column | no | possibly | +| `H`, `S` | one column pair | yes | possibly | +| `CNOT`, `CZ` | two column pairs | yes | possibly | +| Find measurement pivot | column | no | no | +| Row multiplication | transposed row | yes | possibly | +| Collapse/elimination | column scan + transposed rows | yes | yes | +| Physical transpose | bulk matrix | logically no | no | + +This table describes logical mutations. A gate may determine that no phase bit +changed and preserve the phase cache in that special case, but conservative +component invalidation is correct for the first implementation. + +## Structural hashing + +A tableau used in a classical mixture is an `Indexable` key. It owns its own +hasher and cache representations; neither is inherited from `PauliWord`. + +The structural hash is composed from independent logical components: + +```text +tableau hash = combine(xz hash, phase hash[, loss hash]) +``` + +The X and Z planes share an `xz_hash` cache because most Clifford mutations +update them together. The phase plane has a separate cache so Pauli +conjugations and sign changes do not force a matrix rehash. Loss should use a +third cache if it is independently mutable and part of key identity. + +```rust +pub struct Tableau { + data: TableauData, + xz_hash: XzHashCache, + phase_hash: PhaseHashCache, + loss_hash: Option, + rng: SmallRng, +} +``` + +The cache types above are concrete choices made by the tableau author. The +shared `Indexable::HashCache` associated type may name a small aggregate or +another representation chosen by the implementation. + +Equality and hashing include logical qubit count, generator order, all logical +X/Z bits, phases, and loss state when present. They exclude: + +- RNG state; +- allocation capacity; +- alignment padding; +- cache values and validity flags; and +- current physical orientation. + +The component invalidation rules are: + +| Mutation | X/Z cache | Phase cache | Loss cache | +| --- | --- | --- | --- | +| Pauli `X`, `Y`, `Z` | preserve | invalidate if changed | preserve | +| Direct phase change | preserve | invalidate | preserve | +| `H`, `S`, `CNOT`, `CZ` | invalidate | invalidate if changed | preserve | +| Row multiplication | invalidate | invalidate if changed | preserve | +| Mark or clear loss | preserve | preserve | invalidate | +| Physical transpose | preserve | preserve | preserve | +| RNG update | preserve | preserve | preserve | + +The current `ppvm-tableau-sum` split between `word_fingerprint` and +`phase_loss_hash` is evidence that component hashing matters. The new tableau +owns these components directly and separates loss as well when the logical +model permits it. + +## Cloning and mixture use + +Classical-mixture branching can clone tableaus frequently. Contiguous backing +storage makes cloning a bulk memory copy. A clone may copy valid hash caches +because it initially has identical logical contents. Subsequent mutation of +the branch invalidates only the affected components. + +Copy-on-write backing storage may be evaluated later, but it should not be part +of the initial design: most branches are mutated immediately, which may turn +reference counting and deferred copying into overhead. + +An indexable tableau must not be structurally mutated while stored as a map +key. Mixture storage removes or clones a key before applying gates and inserts +the resulting tableau under its updated hash. + +## Sampling implications + +Column-major X/Z planes make fixed-qubit queries and bit-parallel gate updates +efficient. They are also compatible with scanning many generators during a +measurement. Temporary transposition makes elimination and row products +contiguous when required. + +This layout should be evaluated separately from higher-level sampling +algorithms. Stim also uses an inverse tableau and reference-frame sampling; +those algorithmic choices are not consequences of column-major storage and do +not belong in the PPVM trait system. + +## Prototype validation + +The prototype should include: + +- property tests comparing gates and measurements with the existing tableau; +- round-trip tests for column-major -> row-major -> column-major transpose; +- equality and hash tests across physical orientations; +- tests proving phase-only changes preserve the X/Z hash cache; +- tests proving padding never affects equality or hashing; +- benchmarks for one- and two-qubit Clifford gates; +- benchmarks for deterministic and random measurement paths; +- benchmarks for clone-and-mutate mixture branching; and +- benchmarks comparing permanent row-major, permanent column-major, and + temporary-transpose variants on representative circuits. + +## Open questions + +1. What block width and alignment should the first implementation use? +2. Should phases occupy one or two bits per generator in the tableau model? +3. Should loss be stored per qubit, per generator, or outside the core tableau? +4. Which operations should receive a transposition guard versus performing + column-strided work directly? +5. Does a dual-orientation representation outperform temporary transposition + for PPVM's measurement-heavy workloads? +6. Should PPVM ultimately store a forward or inverse tableau? diff --git a/docs/design/traits-2-configuration-and-hashing.md b/docs/design/traits-2-configuration-and-hashing.md new file mode 100644 index 00000000..5d242a2a --- /dev/null +++ b/docs/design/traits-2-configuration-and-hashing.md @@ -0,0 +1,508 @@ +# `ppvm-traits-2`: type composition, indexable values, and cached hashing + +Status: design sketch + +## Motivation + +The current `ppvm_traits::Config` bundles choices from several unrelated +layers: + +- coefficient type; +- packed Pauli storage; +- Pauli-word representation; +- key hasher; +- truncation strategy; and +- concrete map implementation. + +This makes the foundational configuration specific to `PauliSum`, even though +the gate and noise traits are shared by other algorithms. In particular, a map +and a truncation strategy are not properties of quantum data. They are choices +made by a particular algorithm. + +The second trait-system experiment should separate: + +1. the coefficient type, passed directly as a generic parameter; +2. concrete quantum-data representations; +3. the hashing contract of values that can be used as keys; and +4. algorithm-specific storage and policy choices. + +The redesign should remain compile-time generic. It should not introduce +runtime trait objects or runtime dispatch for these choices. + +This proposal was compared against the current definitions in +`ppvm-traits/src/config.rs`, `traits/map.rs`, `traits/strategy.rs`, and +`traits/word_trait.rs`; `ppvm-pauli-sum/src/sum/data.rs`; the concrete word +types in `ppvm-pauli-word`; and `ppvm-tableau-sum`'s `EntryStore`, `VecStorage`, +and `MapStorage`. Existing names are retained below unless the abstraction or +responsibility changes. + +## Type-composition layers + +### Coefficient type + +There is no algorithm-agnostic `Config` trait. Algorithms use the coefficient +type directly: + +```rust +pub struct SomeAlgorithm { + // ... +} +``` + +A trait containing only `type Coeff` adds indirection without providing useful +composition. Maps, policies, Pauli-word storage, Pauli-word +implementations, and hashers are also selected independently rather than being +collected into a replacement global configuration trait. + +### Representation types + +There is no separate global storage configuration, and representation storage +does not appear as an associated type. A concrete value encapsulates its own +fields; generic algorithms use behavioral methods instead of naming or +inspecting the backing memory. + +`Word` is the common concept for an indexed algebraic monomial. The old +Pauli-word operations are not Pauli-specific: every supported word has a site +alphabet, an indexed extent, site access and mutation, and a weight: + +```rust +pub trait Word { + type Site; + + fn n_sites(&self) -> usize; + fn get(&self, index: usize) -> Self::Site; + fn set(&mut self, index: usize, site: Self::Site); + fn weight(&self) -> usize; +} +``` + +`Site` selects the operator alphabet without introducing `PauliWord` or +`FermionWord` subtraits. For example, the relevant concrete types may be: + +```rust +pub enum Pauli { + I, + X, + Y, + Z, +} + +pub enum LossySite { + Present(S), + Lost, +} + +pub struct FermionSite { + pub mode: usize, + pub action: FermionAction, +} +``` + +Thus an ordinary packed Pauli word implements `Word`, a lossy +wrapper implements `Word>`, and a future ordered +fermionic product can implement `Word`. A fermionic word's +index denotes factor order; `FermionSite` carries the physical mode. For a +dense Pauli word, the index is the qubit and `n_sites()` is its width. +`weight()` is the number of non-identity factors according to the concrete site +alphabet; an ordered representation that stores no explicit identities may +therefore have `weight() == n_sites()`. Implementations of `set()` preserve +their representation invariants and invalidate the affected hash components. + +The concrete `WithLoss` family provides loss mutation and +`loss_weight()` as inherent methods. Loss channels and loss-specific +truncation specialize directly on `WithLoss`; there is no one-implementation +`LossyPauliWord` capability trait. + +Concrete packed Pauli, lossy, phased, and hash-cache layouts are described in +[`word-data-structures.md`](word-data-structures.md). None of those layouts is +visible through `Word`. + +A tableau is an independent concrete representation. It does not contain a +public row type, implement `Word`, or select an associated word +implementation. Its X/Z matrices, phases, orientation, and contiguous backing +allocation are private implementation details described in +[`tableau-data-structure.md`](tableau-data-structure.md). + +### Behavioral traits + +Shared traits describe operations, not representation layout. Clifford gates +need no coefficient parameter. Operations that consume numeric parameters use +the coefficient type directly: + +```rust +pub trait Clifford { + fn h(&mut self, qubit: usize); + fn cnot(&mut self, control: usize, target: usize); + // ... +} + +pub trait RotationOne { + fn rx(&mut self, qubit: usize, theta: C); + // ... +} + +pub trait PauliError { + fn pauli_error(&mut self, qubit: usize, probabilities: [C; 3]); +} +``` + +The same concrete tableau may implement a numeric trait for every supported +coefficient type without storing that coefficient type. Measurement and reset +traits likewise expose behavior and result types without exposing tableau +rows, packed matrix blocks, or matrix orientation. + +There is no shared `TableauStorage` trait in the first design. If multiple +tableau implementations are later useful, each concrete type can implement +the same behavioral traits. A storage abstraction should only be introduced +after two implementations demonstrate a common interface. + +### Algorithm and storage parameters + +An algorithm should take its independent choices as direct type parameters. +An associated-type bundle is not useful merely because it replaces two type +parameters with one. In particular, there is no `PauliSumAlgorithm` trait that +bundles a term map with a policy: storage layout and policy are orthogonal +choices. + +The reusable sparse-sum shape is: + +```rust +pub struct OperatorSum +where + C: Coefficient, + W: Word + Indexable, + S: SumStorage, + P: Policy, +{ + storage: S, + policy: P, + n_sites: usize, +} +``` + +Here `C`, `W`, `S`, and `P` respectively mean coefficient domain, algebraic +word, concrete sparse-sum storage engine, and algorithm policy. Each +parameter has an independent meaning. Propagation methods select their algebra +through the site type, for example `W: Word` or +`W: Word`. + +`Policy` is the proposed name for the current `Strategy` concept. It retains +the current responsibilities: predicting initial capacity and truncating the +sum. Existing concrete strategies become policies without otherwise changing +their meaning; `NoStrategy` and `CombinedStrategy` become `NoPolicy` and +`CombinedPolicy`, while `MaxPauliWeight` and `CoefficientThreshold` keep their +established names: + +```rust +pub trait Policy: Default + Clone + Copy +where + W: Word + Indexable, + C: Coefficient, +{ + fn capacity(&self, n_sites: usize) -> usize; + + fn truncate(&self, map: &mut M) + where + M: ACMap; +} +``` + +`Policy` and its concrete implementations belong to the sparse-sum crate. This +removes the current split where the `Strategy` trait lives in `ppvm-traits` but +its concrete strategies live in `ppvm-pauli-sum`; the policy is not an +algorithm-agnostic `ppvm-traits-2` concern. + +`ACMap` remains the name of the associative coefficient-map capability already +implemented by `HashMap`, `IndexMap`, and `DashMap`. Its generic signature can +be simplified after `PauliStorage` and the separately supplied build hasher are +removed, but coefficient accumulation, iteration, insertion, retention, and +consumption are the same concept. `ACMap` moves with the sparse-sum engine (the +existing `ppvm-pauli-sum` initially) rather than being renamed or kept in the +algorithm-agnostic trait crate. Its existing capability names—such as +`ACMapBase`, `ACMapIter`, `ACMapAddAssign`, `ACMapInsert`, `ACMapRetain`, and +`ACMapConsume`—should also remain unless implementation work shows that two +capabilities should actually be merged or split. + +A `SumStorage` is a new abstraction extracted from the fields currently owned +directly by `PauliSum`: its maps and reusable workspace. It is an actual value, +not a marker configuration: + +```rust +pub trait SumStorage: Clone +where + W: Word + Indexable, + C: Coefficient, +{ + type Map: ACMap; + + fn data(&self) -> &Self::Map; + fn data_mut(&mut self) -> &mut Self::Map; + + fn map_insert(&mut self, f: F) + where + F: Fn(&W, &mut C) -> Option<(W, C)>; + + fn map_insert_multiple(&mut self, f: F) + where + F: Fn(&W, &mut C) -> Option>; + + fn map_add(&mut self, f: F) + where + F: Fn(&W, &C) -> (W, C); + + fn consume(&mut self); +} +``` + +The exact closure bounds and support for multiple produced terms remain an +implementation detail for the prototype. The important boundary is that the +trait preserves the current semantic operation names without exposing physical +auxiliary maps or scratch buffers. + +The generalized engine may be named `OperatorSum`, but the Pauli specialization +retains the existing domain-facing `PauliSum` name. This is a new internal +generalization, not a requirement to rename Pauli call sites. + +A classical tableau mixture follows the same principle and takes its +`EntryStore` directly rather than introducing a one-associated-type +`TableauMixtureAlgorithm` bundle: + +```rust +pub struct GeneralizedTableauSum +where + C: Coefficient, + T: Indexable, + S: EntryStore, +{ + entries: S, +} +``` + +`GeneralizedTableauSum` and `EntryStore` retain their current names because the +proposal does not change their underlying roles. `GeneralizedTableauSum` and +`OperatorSum` are both sparse linear combinations of indexable keys, so they +may eventually share an implementation. This iteration deliberately keeps them +separate: their mutation, branching, normalization, and storage requirements +have not yet been reduced to a proven common interface. The next design +iteration should look for the smallest useful common factor and merge only that +factor, rather than assuming that the two complete algorithms are identical. + +Every keyed store must use the build hasher associated with its key type. + +### Compatibility with current names + +The redesign is not a vocabulary reset. The following names are retained or +changed according to whether their underlying responsibility changes: + +| Current implementation | Proposal | Rationale | +| --- | --- | --- | +| `Config` | removed | The bundle itself is removed; this is not a rename. | +| `PauliWordTrait` | `Word` plus `Indexable` where used as a key | Word operations are generalized through `Word::Site`; hashing becomes a separate capability. | +| `n_qubits`, `get`, `set`, `weight` | `n_sites`, `get`, `set`, `weight` | Only the Pauli-specific extent name changes; the other operation names stay. | +| concrete `PauliWord` | `PauliWord` | The packed X/Z word is the same domain concept. | +| concrete `LossyPauliWord` | `LossyPauliWord` alias over `WithLoss` | The public concept stays; only the implementation becomes a generic wrapper. | +| `PhasedPauliWord` | `PhasedPauliWord` alias over `Phased` | The public Pauli name stays; the implementation becomes a word-generic wrapper. | +| `rehash` | `invalidate_hash` for lazy modes | Recalculation changes from eager mutation-time work to lazy demand-time work. | +| `Strategy` | `Policy` | Intentional terminology change requested for this redesign. | +| `ACMap` | `ACMap` | The associative coefficient map has the same role. | +| `PauliSum::data`, `map_insert`, `map_add` | same method names on `SumStorage` | These semantic operations already match the proposed boundary. | +| `PauliSum` map pair and `scratch` fields | `SumStorage` | A new abstraction is extracted from currently unnamed storage state. | +| `PauliSum` | `PauliSum` over generalized `OperatorSum` machinery | Pauli-facing code keeps its established name; `OperatorSum` names the new cross-algebra engine. | +| `GeneralizedTableauSum` | `GeneralizedTableauSum` | The classical mixture algorithm remains the same concept. | +| `EntryStore`, `VecStorage`, `MapStorage` | unchanged | The proposal uses the existing storage boundary and implementations. | +| `BuildHasher`, `HashFinalize` | unchanged | The hasher concepts remain, but ownership moves from `Config` to each indexable key type. | +| `PauliStorage` | removed | Packed backing storage becomes private to the concrete word representation. | + +Names such as `Word`, `Indexable`, `SumStorage`, `WithLoss`, and +`OperatorSum` are therefore new because they denote abstractions that do not +exist in the current implementation, not because the existing API is being +renamed wholesale. + +### Trait admission rule + +A proposed trait belongs in the design only when generic code consumes it and +there are multiple meaningful implementation families, or when it is an +established behavioral boundary implemented by different backends. A trait is +not justified merely to name inherent methods on one generic struct. + +This rule keeps: + +- `Indexable`, consumed by keyed stores and implemented by hash-enabled words + and tableaus; +- `Word`, consumed by the shared sparse-sum engine and propagation algorithms; +- gate and noise traits, implemented across propagation and tableau backends; +- `SumStorage` and `ACMap`, implemented by genuinely different storage engines + and collections; +- `EntryStore`, already implemented by `VecStorage` and `MapStorage`; and +- `Policy`, implemented by independent capacity and truncation behaviors. + +It rejects the removed global `Config`, `PauliSumAlgorithm`, +`TableauMixtureAlgorithm`, and `TableauStorage` traits, as well as word +subtraits named `PauliWord`, `FermionWord`, or `LossyPauliWord`. Their +distinctions are expressed by `Word::Site` or by concrete wrapper types instead +of one-alphabet subtraits; the concrete `PauliWord` and `LossyPauliWord` type +names remain available. + +### Sparse-sum branch staging + +A propagation rule can turn one term into multiple terms. For example, a +Pauli rotation may produce: + +```text +c P -> c cos(theta) P + c sin(theta) P' +``` + +The existing entry can be updated while the active map is traversed, but a new +key cannot generally be inserted into that same map during mutable iteration. +New terms must therefore be staged and merged after the traversal. New keys +may collide with each other or with existing keys, so the merge accumulates +their coefficients. + +Only one staging mechanism is required for correctness. The current engine +uses two because they serve different performance paths: + +- an auxiliary map supports whole-map rewrites, combines output collisions as + they are produced, and retains its allocation across operations; and +- a reusable `Vec<(W, C)>` scratch buffer stages additional terms when existing + entries remain in place, avoiding an auxiliary-map insertion followed by a + second map probe during the merge. + +Conceptually: + +```text +whole-map rewrite: active map -> auxiliary map -> swap +in-place branching: active map + scratch buffer -> merge into active map +``` + +Neither physical mechanism is part of the public storage contract. A default +`SumStorage` implementation may privately contain both maps and the reusable +vector, matching the current `PauliSum` layout. A simpler backend may implement +both `map_insert` and `map_add` using only an auxiliary map; another backend +may use per-thread buffers. Whether retaining both mechanisms is worthwhile is +a benchmark decision, not a type-system requirement. + +## Indexable values + +Hash-enabled `Word` values and `Tableau` can both be expensive, mostly-stable +map keys. Their hashing contract should be expressed independently of any map. + +The tentative common capability is: + +```rust +pub trait Indexable: Clone + Eq + Hash { + type BuildHasher: std::hash::BuildHasher + Clone + Default + HashFinalize; + type HashCache; + + fn invalidate_hash(&mut self); +} +``` + +The important points are: + +- `BuildHasher` retains the current associated-type name and moves from + `Config` to the key type; +- the build hasher is associated with the key type, not with a configuration + bundle; +- the hash cache is an associated type, not a framework-provided concrete + `HashCache` structure; +- the author of the concrete data type owns the cache layout and invalidation + implementation; and +- equality and hashing cover only structural key identity, never cache fields + or incidental runtime state. + +For example, a word implementation may choose `u64`, `Option`, an atomic +representation, or another type. `ppvm-traits-2` does not prescribe the +fields that store it. + +### Lazy hashing and interior mutability + +Rust's `Hash::hash` receives `&self`. A cache that is first populated from +inside `Hash::hash` therefore requires interior mutability. Possible concrete +representations include: + +- `Cell` plus `Cell` for single-threaded keys; +- atomics for keys that must remain `Sync`; or +- a plain `u64` plus `bool` when cache preparation is eager or explicitly + performed through `&mut self` before hashing. + +This is deliberately a representation-level decision. A cache using interior +mutability will generally prevent the containing value from being `Copy`. + +Multiple concurrent readers may compute the same invalid hash more than once; +that is acceptable as long as they publish the same structural hash safely. +Structural mutation still requires `&mut self`. + +### Key mutation invariant + +An indexable value must not be structurally mutated while it is stored as a map +key. Cache invalidation makes a value correct for its next insertion or lookup; +it cannot make in-place mutation of an existing map key valid. + +Structural fields should therefore be private in the new representations. +Mutation must go through operations that invalidate the affected cache, or +through mutation guards that invalidate on completion. + +## Concrete word hashing + +Every concrete `Word` owns its `BuildHasher`, cache representation, structural +hash algorithm, and invalidation logic through `Indexable`. Pauli words hash +their X/Z content, lossy words compose Pauli and loss components, and future +fermion words hash their ordered factors. Factor order is part of fermionic +identity. + +The trait layer does not expose packed storage to support hashing. Concrete +implementations hash their private fields and may apply hasher-specific +finalization internally. Detailed layouts and component invalidation rules are +in [`word-data-structures.md`](word-data-structures.md). + +## Tableau indexability + +A tableau may itself be used as a key by a classical-mixture algorithm, so the +concrete tableau implements `Indexable` directly and owns a tableau-specific +hasher and cache representation. This does not imply that a tableau is a +`Word`; they only share the `Indexable` key capability. + +The tableau's structural hash is composed from its logical X/Z matrix, phase +plane, and optional loss plane. It excludes RNG, padding, cache state, and +physical matrix orientation. Separate component caches allow phase-only +changes to avoid rehashing the X/Z matrix. Physical transposition is a layout +change, not a logical mutation, and does not invalidate the structural hash. + +The concrete memory layout, component invalidation table, and canonical hash +order live in [`tableau-data-structure.md`](tableau-data-structure.md) so they +do not leak into the shared trait-system design. + +## Expected generic composition + +The intended composition is explicit: + +```rust +OperatorSum +Tableau +GeneralizedTableauSum +``` + +Domain-specific aliases or wrappers can preserve `PauliSum` and introduce +`FermionSum` without rebuilding a monolithic configuration trait. + +## Non-goals for the first prototype + +- Migrating the existing crates to `ppvm-traits-2` immediately. +- Merging `GeneralizedTableauSum` and `OperatorSum` in this iteration; only a + smaller proven common factor should be considered in the next iteration. +- Defining one collection interface shared by all algorithms. +- Requiring every sparse-sum storage backend to physically contain both an + auxiliary map and a scratch buffer. +- Selecting a single cache representation for every key type. +- Preserving `Copy` at the expense of correct lazy caching. +- Adding runtime dispatch for storage, hashing, or algorithm policies. + +## Open design questions + +1. What is the minimal method surface of `Indexable` beyond the associated + `BuildHasher` and `HashCache` types? Cache mechanics should remain owned by + the concrete type, but algorithms may need a common invalidation operation. +2. Which cache representations must support `Send` and `Sync` in the first + prototype? +3. Do benchmarks justify retaining both the auxiliary-map and vector-staging + fast paths in the default sparse-sum storage backend? diff --git a/docs/design/word-data-structures.md b/docs/design/word-data-structures.md new file mode 100644 index 00000000..5ece8e5c --- /dev/null +++ b/docs/design/word-data-structures.md @@ -0,0 +1,357 @@ +# Word data structures + +Status: design sketch + +## Purpose + +This document describes concrete data structures for standalone algebraic +words. The shared trait design is specified in +[`traits-2-configuration-and-hashing.md`](traits-2-configuration-and-hashing.md). + +The trait-level `Word` is representation-free but defines the common indexed +product operations: + +```rust +pub trait Word { + type Site; + + fn n_sites(&self) -> usize; + fn get(&self, index: usize) -> Self::Site; + fn set(&mut self, index: usize, site: Self::Site); + fn weight(&self) -> usize; +} +``` + +There is no `Word::Storage` associated type. Packed arrays, loss masks, ordered +factors, hash fields, and validity flags are private fields of concrete types. +Generic propagation and collection code uses behavioral traits and never names +the backing memory. + +`Word` does not extend `Indexable`. A word used as an `ACMap` key implements +both traits, while a mutable intermediate such as `Phased<_, NoHash>` can still +implement `Word` without pretending to be a valid map key. This separation +replaces the current `REHASH = false` use case with an explicit non-key mode. + +This document initially focuses on: + +- packed Pauli words; +- lossy Pauli words; +- phased-word composition; and +- lazy structural hash caches. + +Fermion-word storage will receive a separate design when fermionic propagation +is implemented. It will use the same `Word` interface with a different `Site`: +the word index records product order, while the fermionic site value records +the physical mode and creation or annihilation action. + +`weight()` counts non-identity factors according to the selected site type. +For a representation that stores only non-identity factors, it may equal +`n_sites()`. `set()` is the shared structural mutation boundary and must +preserve concrete invariants and invalidate the affected hash components. + +## Logical Pauli model + +An ordinary Pauli word is a fixed-width tensor product over `I`, `X`, `Y`, and +`Z`. Each site is represented by two logical bits: + +| Pauli | X bit | Z bit | +| --- | --- | --- | +| `I` | 0 | 0 | +| `X` | 1 | 0 | +| `Z` | 0 | 1 | +| `Y` | 1 | 1 | + +A lossy Pauli word adds a `Lost` state. Loss is exclusive with the four Pauli +states; a lost site has canonical bits `(x, z, lost) = (0, 0, 1)`. + +The exact Pauli alphabet and its lossy extension are site types rather than +word subtraits: + +```rust +pub enum Pauli { + I, + X, + Y, + Z, +} + +pub enum LossySite { + Present(S), + Lost, +} +``` + +An ordinary word implements `Word`. `WithLoss` implements +`Word>` and returns `Lost` for marked sites. This keeps +the word interface independent of the chosen operator alphabet. + +## `PauliWord` packed representation + +The initial packed implementation stores parallel fixed-size X and Z arrays: + +```rust +pub struct PauliWord { + xbits: BitArray, + zbits: BitArray, + nqubits: usize, + hash_cache: M, + _phantom: PhantomData, +} +``` + +`A` is an implementation parameter such as `[u8; N]` or `[usize; N]`. It is +not exposed through `Word`. The implementation validates that `nqubits` fits +the arrays and ignores or canonicalizes unused high bits. + +The structural identity is: + +```text +(nqubits, logical X bits, logical Z bits) +``` + +Equality and hashing exclude unused capacity, cache contents, cache validity, +and the `PhantomData` marker. + +All fields are private. Mutations go through methods that preserve unused-bit +invariants and invalidate the hash when a logical Pauli site changes. + +## Lossy Pauli word + +Loss is best modeled as an orthogonal wrapper around a Pauli word: + +```rust +pub struct WithLoss +where + W: Word, +{ + word: W, + lbits: BitArray, + loss_hash: M, +} +``` + +`A` is the packed bit-array backing selected by the concrete type. The public +alias retains the current `LossyPauliWord` name: + +```rust +pub type LossyPauliWord = WithLoss, A>; +``` + +`WithLoss` is a justified new implementation name because the proposal changes +the current standalone lossy representation into a generic wrapper. The +established domain-facing alias does not need to change. + +Neither `A` nor the underlying word's array type appears in `Word`. + +### Canonical loss invariant + +A lost site must contain identity in the underlying Pauli word: + +```text +lost[q] = 1 => word[q] = I +``` + +`set_lost(q)` first sets the underlying word to `I`, then sets the loss bit. +`set(q, LossySite::Present(p))` clears the loss bit and then writes `p`. This +prevents multiple physical encodings from representing the same logical lossy +word. + +The structural identity is: + +```text +(underlying Pauli word, logical loss bits) +``` + +`weight()` counts `X`, `Y`, `Z`, and `Lost`; `loss_weight()` counts only lost +sites. + +### Loss-specific behavior + +Generic lossy Pauli propagation sees `LossySite::Lost` through `Word` and +preserves or skips the site according to the operation's semantics. Operations +that create, clear, or count loss use inherent `WithLoss` methods: + +```rust +impl WithLoss +where + W: Word, +{ + pub fn is_lost(&self, qubit: usize) -> bool; + pub fn set_lost(&mut self, qubit: usize); + pub fn clear_loss(&mut self, qubit: usize); + pub fn loss_weight(&self) -> usize; +} +``` + +Loss channels and maximum-loss-weight truncation specialize directly on the +generic wrapper family: + +```rust +impl LossChannel + for OperatorSum, S, P> +where + W: Word, +{ + // ... +} +``` + +There are no traits named `PauliWord`, `LossyPauliWord`, or `FermionWord`. +`PauliWord` and `LossyPauliWord` remain concrete domain type names. Algorithms +select the algebra through `Word::Site`; loss-only operations remain inherent +to `WithLoss`. + +## Phased words + +Phase is another orthogonal wrapper: + +```rust +pub struct Phased +where + W: Word, + M: HashMode, +{ + word: W, + phase: Phase, + hash_state: M::State, +} +``` + +`Phased` is the generalized wrapper implementation. Pauli-facing code retains +the established name through an alias: + +```rust +pub type PhasedPauliWord = Phased; +``` + +For Pauli use, `Phase` represents `+1`, `+i`, `-1`, and `-i`. The wrapper may +also be useful for other word algebras, but algebra-specific multiplication is +implemented only under the appropriate specialized word bound. + +Loss and phase compose without a new combined representation: + +```rust +PhasedPauliWord, A>, Mode> +``` + +The wrapper fields are private so phase mutation cannot bypass component-cache +invalidation. + +## Hash ownership + +Every hash-enabled word implements `Indexable` and selects: + +- a build hasher; +- an `Indexable::HashCache` associated type; +- concrete cache fields; and +- the algorithm that hashes its private structural data. + +The associated cache type does not require every implementation to use the +same representation. Examples include: + +- a plain unsigned integer with a separate validity flag for eager or explicit + recomputation; +- `Cell` plus `Cell` for lazy single-threaded hashing; +- atomics for a `Sync` key; and +- `()` for a mode that deliberately provides no cache. + +Because `Hash::hash` receives `&self`, a cache populated lazily from that method +requires interior mutability. Such a representation will generally prevent +the containing word from being `Copy`. + +## Component hashes + +Hash composition follows the logical wrappers: + +```text +packed Pauli hash = hash(nqubits, X bits, Z bits) +lossy hash = combine(Pauli hash, loss hash) +phased hash = combine(inner-word hash, phase hash) +phased lossy hash = combine(Pauli hash, loss hash, phase hash) +``` + +`combine` must be ordered and domain-separated. It must not be an +unqualified XOR of arbitrary component digests. + +The phase has only four values, so `CompositeHash` can normally compute its +contribution from a small table or mixer without another cache. `CachedHash` +is justified only if profiling identifies a caller that benefits from caching +the combined phased value. + +Loss masks may be large, so `WithLoss` can cache the loss component separately +from the Pauli component. A loss-only mutation then avoids rehashing X/Z. + +## Invalidation rules + +| Mutation | Pauli component | Loss component | Phase component | +| --- | --- | --- | --- | +| Change ordinary Pauli site | invalidate | preserve | preserve | +| Mark identity site lost | preserve | invalidate | preserve | +| Mark nonidentity site lost | invalidate | invalidate | preserve | +| Clear loss to identity | preserve | invalidate | preserve | +| Replace loss with Pauli | invalidate if nonidentity | invalidate | preserve | +| Change phase | preserve | preserve | recompute or invalidate | + +Constructors compute caches eagerly or mark them invalid according to the +selected cache mode. Cloning copies a valid cache because the clone initially +has identical structural contents. + +## Hash modes + +Wrappers that are sometimes used only as mutable intermediate values should be +generic over hash behavior: + +```rust +pub trait HashMode { + type State: Clone; +} + +pub struct NoHash; +pub struct CompositeHash; +pub struct CachedHash(PhantomData); +``` + +- `NoHash` stores no wrapper cache and does not implement `Hash` or + `Indexable` for the wrapper. +- `CompositeHash` implements `Hash` by combining cached inner components on + demand, with no combined cache. +- `CachedHash` stores a lazily or eagerly maintained combined cache. + +The concrete policy protocol should be finalized while implementing the first +two modes. It should not expose cache state to propagation algorithms. + +## Ordering and serialization + +`Eq`, `Ord`, `Hash`, display, parsing, and serialization must agree on logical +identity: + +- Pauli sites compare in a documented order. +- Loss participates after the underlying Pauli content or through an explicit + `LossySite` ordering. +- Phase participates only when the phased wrapper's identity includes it. +- Unused bits and cache state never participate. + +Serialization uses logical symbols and lengths, not raw native-word memory, so +it remains stable across storage widths and platforms. + +## Prototype validation + +The prototype should include: + +- round-trip parsing tests for ordinary and lossy symbols; +- property tests comparing packed operations with a simple symbol vector; +- tests enforcing `lost => underlying I` after every mutator; +- equality/hash agreement tests for all wrappers and modes; +- tests showing loss-only changes preserve the Pauli hash component; +- tests showing phase-only changes preserve Pauli and loss components; +- tests proving unused high bits do not affect identity; +- `Send`/`Sync` assertions for cache modes intended for concurrent maps; and +- benchmarks comparing eager, lazy, composite, and uncached hashing. + +## Open questions + +1. Should `WithLoss` permit a loss-mask storage width different from its inner + word's packed width? +2. Does any real phased-word caller benefit from a combined `CachedHash` mode? +3. Does `Indexable::HashCache` have a generic consumer, or should the cache + type also remain entirely private like word storage?