From 7bc09ed57b6a0dcfb0563532df72fe2eda52c2ae Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Wed, 16 Sep 2026 14:12:42 -0700 Subject: [PATCH 01/13] Document the integrations crate split --- ...-09-16-split-integrations-crates-design.md | 662 ++++++++++++++++++ 1 file changed, 662 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md diff --git a/docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md new file mode 100644 index 000000000..7331d662f --- /dev/null +++ b/docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md @@ -0,0 +1,662 @@ +# Split Integrations into Dedicated Rust and JavaScript Crates + +**Date:** 2026-09-16 +**Status:** Proposed +**Scope:** Move every concrete integration implementation out of +`trusted-server-core` and `trusted-server-js` into two statically compiled, +directory-discovered workspace crates + +## Summary + +Trusted Server will separate its concrete integrations from its neutral server +and browser runtimes. + +The workspace gains two crates: + +- `trusted-server-integrations`, containing every concrete Rust integration. +- `trusted-server-integrations-js`, containing every integration-specific + browser module, browser asset, and JavaScript test. + +Each crate discovers integrations from its own directories at build time. The +Rust and JavaScript inventories are independent because the repository has +Rust-only and JavaScript-only integrations. A Rust integration may declare that +it requires a same-named JavaScript module, and that reference is checked at +build time. + +`trusted-server-core` keeps only the neutral contracts and runtime machinery +needed to execute registrations. `trusted-server-js` keeps only the neutral +browser runtime. Adapters and the CLI use the statically linked +`trusted-server-integrations` crate as the composition root. + +This is a packaging and dependency-direction change. It is not the provider, +permission, configuration, or externally loaded integration system proposed by +PR #1084. + +## Context + +On `main` at `6cae7f5da`, concrete integrations live inside +`crates/trusted-server-core/src/integrations`. That directory contains the +neutral registry alongside fifteen concrete implementation units: + +- `adserver_mock` +- `aps` +- `datadome` +- `didomi` +- `google_tag_manager` +- `gpt` +- `gpt_diagnostics` +- `js_asset_proxy` +- `lockr` +- `nextjs` +- `osano` +- `permutive` +- `prebid` +- `sourcepoint` +- `testlight` + +The ordinary builder table registers twelve of those units. APS and Prebid +register separately from the compiled auction plan, while `adserver_mock` +registers the current mock mediator. That difference in construction does not +justify leaving those implementations in core. + +Integration browser code lives beside browser core under +`crates/trusted-server-js/lib/src/integrations`. The current build discovers +directories containing `index.ts`, emits one IIFE per discovered entry point, +and embeds those bundles and hashes into the `trusted-server-js` Rust crate. +The directory also contains APS renderer code that browser core imports +directly even though APS has no current `index.ts` entry point. + +This layout has four problems: + +1. Core contains both the integration contract and every implementation. +2. Rust registration, deploy validation, and migration guards maintain + separate concrete inventories. +3. Browser core and integration code have dependencies in both directions. +4. Adding an in-tree integration requires editing central hand-maintained + lists instead of adding a self-contained directory. + +PR #1084 identifies these problems but combines their solution with external +vendor crates, runtime registration injection, provider capabilities, +configuration changes, permission work, and adapter composition changes. This +specification takes only the shared-crate and build-time discovery decisions. + +## Goals + +1. Move all fifteen concrete Rust implementation units into one + `trusted-server-integrations` crate. +2. Move all integration-specific TypeScript, JavaScript assets, fixtures, and + JavaScript tests into one `trusted-server-integrations-js` crate. +3. Give every concrete Rust implementation its own directory. +4. Discover the Rust and JavaScript inventories at build time. +5. Remove concrete integration construction and validation tables from core. +6. Preserve current configuration, routes, ordering, auction behavior, wire + formats, and request/response behavior. +7. Make core compile without depending on either integrations crate. +8. Keep the two new crates statically linked workspace components. + +## Meaning of “Concrete Integration” + +For this change, a concrete integration is an implementation currently under +`trusted-server-core/src/integrations`, integration-specific browser code under +`trusted-server-js/lib/src/integrations`, or direct construction and lifecycle +plumbing that imports one of those implementations. + +The extraction does not require renaming or relocating every existing domain +type, compatibility field, comment, or wire type that mentions APS, GPT, +Prebid, or another integration. For example, a serialized auction renderer +descriptor may remain in core when it is part of the existing core wire model. +Such types move only when leaving them in place would create a dependency from +core to a concrete implementation. + +This boundary keeps the requested extraction complete without turning it into +an auction-domain or configuration-model rewrite. + +## Non-Goals + +This specification does not introduce: + +- External integration crates. +- Runtime-loaded integrations or dynamic registration. +- Per-vendor ownership or independent release lifecycles. +- A public plugin SDK. +- Identity, geo, device, permission, demand, or ad-server provider systems. +- Configuration key, schema, or file-format changes. +- New integrations or new integration capabilities. +- A redesign of the auction plan, ranking, mediation, or renderer wire format. +- EdgeZero lifecycle, evidence, store, or adapter changes. +- A general-purpose hook or capability language. +- A reorganization of the CLI audit analyzer or other integration-related code + that is not part of the implementation, validation, or registration paths + being extracted. + +The new cross-crate interfaces exist only to preserve the current statically +compiled application. Designing them so a future external crate could use them +is explicitly deferred until a real external consumer exists. + +## Target Workspace Layout + +```text +crates/ + trusted-server-core/ + src/ + integration/ + mod.rs + registry.rs + + trusted-server-js/ + lib/src/core/ + src/ + + trusted-server-integrations/ + build.rs + Cargo.toml + src/ + lib.rs + adserver_mock/ + integration.toml + mod.rs + aps/ + integration.toml + mod.rs + datadome/ + integration.toml + mod.rs + protection.rs + protection_scope.rs + ... + nextjs/ + integration.toml + mod.rs + html_post_process.rs + rsc.rs + rsc_placeholders.rs + script_rewriter.rs + shared.rs + fixtures/ + + trusted-server-integrations-js/ + build.rs + Cargo.toml + lib/ + package.json + src/ + aps/ + index.ts + render.ts + creative/ + index.ts + ... + datadome/ + index.ts + ... + ... + test/ + integrations/ + fixtures/ + src/ + lib.rs +``` + +Every flat Rust implementation file becomes `/mod.rs`. Existing nested +modules and fixtures stay with their owning integration. The GPT bootstrap +script moves to `trusted-server-integrations-js` and is exported as an +integration asset rather than remaining beside Rust source. + +JavaScript-only `creative` remains valid without a Rust directory. Rust-only +integrations remain valid without a JavaScript directory. + +## Dependency Direction + +The dependency graph is one-way: + +```text +trusted-server-js + ^ + | +trusted-server-integrations-js + ^ + | +trusted-server-integrations ----> trusted-server-core ----> trusted-server-js + ^ ^ + | | + +------------- adapters -----------+ + +--------------- CLI +``` + +The diagram shows logical dependencies. Cargo may deduplicate a shared +`trusted-server-js` dependency; no cycle is permitted. + +The rules are: + +1. Core never depends on `trusted-server-integrations` or + `trusted-server-integrations-js`. +2. Concrete Rust integrations may use public core contracts and domain types. +3. Integration JavaScript may import the explicit browser-core source API. +4. Browser core must not import a concrete integration. +5. Adapters and the CLI compose core with the built-in integrations crate. + +## Neutral Core Contract + +The existing neutral contents of `integrations/registry.rs` move to a singular +`trusted_server_core::integration` module. The singular name distinguishes the +contract from the collection of concrete implementations. + +Core continues to own: + +- `IntegrationRegistration` and its builder. +- `IntegrationRegistry` and registry execution. +- Request filtering, proxy, rewriting, head injection, HTML post-processing, + and other neutral hook traits and contexts. +- Duplicate ID and route detection. +- Neutral script-module metadata used by publishing and HTML injection. +- Test helpers for empty or stub registries. + +`IntegrationRegistry` no longer constructs built-ins. Its production +constructor accepts completed registrations and the existing compiled auction +plan. The fixed `builders()` table, APS/Prebid special construction, and +concrete `IntegrationRegistry::with_plan` assembly move to +`trusted-server-integrations`. + +Core tests that need only registry behavior use neutral stubs. Tests that need +the actual built-in catalog move to or depend on the integrations crate at the +outer composition layer. + +## Rust Directory Discovery + +`trusted-server-integrations/build.rs` scans immediate directories under +`src/`. A Rust integration directory must contain: + +- `mod.rs`. +- `integration.toml`. + +The directory-local manifest is deliberately small: + +```toml +id = "gpt" +order = 80 +javascript = true +``` + +The fields mean: + +- `id` must equal the directory name and use Rust `snake_case`. +- `order` preserves current registration and hook order. Ordering metadata is + owned by the integration rather than a central list. +- `javascript` states whether a same-named directory with `index.ts` must exist + in `trusted-server-integrations-js`. + +No provider, permission, configuration, dependency, or ownership metadata is +added to this manifest. + +The build script generates module declarations and an ordered definition +table. It fails the build for: + +- A malformed or mismatched ID. +- A missing manifest or `mod.rs`. +- Duplicate IDs or order values. +- A JavaScript requirement whose same-named `index.ts` is absent. +- Generated output that would be empty. + +The script emits `cargo:rerun-if-changed` directives for the discovered Rust +directories and for JavaScript directories referenced by a Rust manifest. + +Each generated module exposes one crate-private definition function. The +returned internal definition may carry only the contributions already needed +by current code: + +- Deploy and startup validation. +- An ordinary page registration. +- An auction profile or transport implementation. +- The current mock mediator implementation. +- An optional JavaScript module or asset reference. + +This internal definition is not exported as a plugin contract. APS, Prebid, +and `adserver_mock` use the same generated inventory as every other directory, +even though their existing contributions are different. + +## JavaScript Directory Discovery + +`trusted-server-integrations-js` owns its Node project, tests, build pipeline, +generated Rust module catalog, and integration assets. + +Its JavaScript build discovers immediate directories containing `index.ts`, +sorts them deterministically, and emits one self-contained IIFE per directory. +Its Cargo build script embeds each bundle and its SHA-256 hash, following the +current `trusted-server-js` mechanism. + +The inventories remain independent: + +- A JavaScript-only directory such as `creative` is built without a Rust + registration. +- A Rust directory with `javascript = false` does not require a browser module. +- A Rust directory with `javascript = true` requires a same-named browser entry + point and receives that generated module at composition time. + +There is no central allowlist spanning the crates. + +`trusted-server-js` builds only the core IIFE and exposes neutral helpers for +combining the core bundle with supplied integration modules. The concatenation +order remains core first, followed by immediate integration modules in registry +order. Deferred and standalone modules remain separately addressable. + +## Static Composition + +`trusted-server-integrations` is the only built-in composition root. It uses +the generated definition table in this order: + +1. Run integration-aware deploy or startup validation as requested by the + caller. +2. Collect the existing auction profile inputs needed to compile the canonical + auction plan. +3. Ask core to compile the plan using those inputs. +4. Build the existing mediator and integration registrations against that plan. +5. Resolve required bundles and assets from + `trusted-server-integrations-js`. +6. Pass completed registrations to core's neutral registry constructor. + +The exact Rust function names are left to the implementation plan, but the +composition path must be shared. The four adapters must not recreate the +generated catalog or call concrete integrations directly. + +At runtime: + +1. An adapter holds the completed core registry and auction state as it does + today. +2. Core invokes neutral registry hooks during request and response processing. +3. Script generation starts with the core bundle and reads enabled integration + bundles from the registry. +4. Deferred and standalone script requests resolve against registry-carried + module metadata rather than a global concrete list in core. + +No runtime directory scanning or dynamic loading occurs. + +## Configuration and CLI Validation + +Configuration keys and serialized shapes do not change. + +Core retains global settings parsing and global validation. Concrete config +types and integration-specific validation move with their implementations. +The generated definition table supplies integration validation to both runtime +startup and the CLI. + +The integration-aware typed app-config wrapper composes: + +- Core secret-field metadata and global validation. +- Generated integration secret-field metadata and validation. + +The CLI uses this composed wrapper for `config validate`, `config diff`, and +`config push`. This removes the concrete validation table and DataDome-specific +secret paths from core without changing the published `trusted-server.toml` +shape. + +Unknown or invalid integration settings continue to fail with the current +error contexts. An integration present in configuration but absent from the +compiled catalog must fail rather than be silently ignored. + +## Required Neutral Lifecycle Hooks + +Moving every implementation exposes two existing reverse dependencies that +must become neutral registry behavior. + +### Response sharing annotation + +DataDome currently communicates a concrete request marker back into core so +core buffers the full response and applies private caching. Replace that marker +with a neutral response-sharing annotation owned by core. DataDome sets the +annotation through its registered hook; core performs the same buffering and +cache behavior without importing a DataDome type. + +This annotation does not introduce a general policy system. It represents only +the existing shared-versus-request-private decision already consumed by core. + +### Request preparation and response finalization + +GPT diagnostics currently has direct preparation calls in every adapter and in +core, plus direct finalization in core. Add neutral registration hooks for +those two lifecycle points. The registry owns any opaque request-scoped state +between them. + +Adapters call the registry preparation operation at the same request boundaries +used today. Core calls finalization on the same response path used today. +Integration-specific bootstrap and script decisions flow through the existing +head-injection and document-state mechanisms instead of concrete fields on +`HtmlProcessorConfig`. + +No other lifecycle stages are added. + +## Auction-Coupled Implementations + +APS, Prebid, and `adserver_mock` move with all other concrete implementations. +Core keeps the generic auction plan, orchestration, request, response, and wire +types. + +The smallest cross-crate inputs needed by the current implementations become +public neutral core APIs: + +- Profile compilation registrations consumed by the existing plan compiler. +- Provider transport and response callbacks consumed by the existing generic + provider path. +- The optional mediator passed to the existing orchestrator. + +The integrations crate supplies only the current built-ins through these APIs. +This work must not change provider configuration, plan semantics, routing, +timeouts, response normalization, ranking, mediation, or telemetry. + +Existing APS-specific serialized types may remain in core where they are part +of the established browser wire contract. The concrete APS parsing, +validation, transport, and rendering implementation moves. + +## Browser APS Boundary + +Browser core currently imports APS renderer parsing and dispatch directly. +That reverse dependency must end when APS moves. + +Browser core will own one narrow bid-renderer registration mechanism: + +- Core parses the existing renderer envelope only far enough to identify its + type and retain its opaque payload. +- An integration module registers the parser and dispatcher for its renderer + type during IIFE initialization. +- The APS browser module registers the existing `aps` renderer implementation. +- Core dispatches a renderer bid through the registered implementation. +- An absent or rejecting renderer continues to fail closed without executing + unvalidated creative code. + +The APS module is included in the immediate script set whenever the compiled +auction plan can emit an APS renderer descriptor. Core is loaded first, so the +registration exists before application code can request and render bids. + +The serialized renderer descriptor, APS validation rules, sandbox flags, +message authentication, timeouts, and render results remain unchanged. This is +not a general creative-renderer redesign; it is the minimum inversion needed +to remove the core-to-APS source import. + +## Error Handling + +Failures remain fail-closed and occur as early as the information permits. + +Build-time failures include malformed directories, invalid manifests, +duplicate discovery metadata, missing JavaScript entry points, JavaScript build +failures, and missing generated bundles. + +Startup or deploy-validation failures include duplicate integration IDs, +duplicate routes, invalid integration configuration, unresolved script assets, +and incompatible auction contributions. + +Runtime hook errors retain the current `Report` contexts +and response behavior. Moving a call behind the registry must not turn an +existing error into a log-and-continue path or a panic. + +Core visibility changes must be narrow. A private helper moves with its +integration when possible. When an integration genuinely needs a core helper, +the implementation exposes the smallest named API and documents it. The change +must not broadly convert core modules or fields to `pub`. + +## Compatibility + +The following are compatibility requirements: + +- Existing `trusted-server.toml` files require no migration. +- Integration IDs and enablement rules do not change. +- Registration and hook order do not change. +- Routes and endpoint behavior do not change. +- Immediate, deferred, and standalone delivery decisions do not change, except + that APS becomes an explicit integration module required by its plan. +- Auction requests, responses, renderer descriptors, and telemetry do not + change. +- Cache privacy and full-buffer decisions do not change. +- All four adapters expose the same routes and behaviors as before. + +Bundle hashes and cache-busting URLs may change because browser core and +integration code are rebuilt in different crates. The server must always emit +URLs matching the newly generated hashes, so old and new artifacts cannot be +confused in cache. No stable bundle hash is part of the compatibility contract. + +## Migration Sequence + +Implementation may use small commits, but the merged workspace must never +contain two active built-in catalogs. + +1. Establish the neutral script-module and lifecycle contracts in core without + changing behavior. +2. Create `trusted-server-integrations-js`, move integration browser sources + and tests, and remove concrete imports from browser core. +3. Create `trusted-server-integrations`, add directory discovery, and move all + fifteen Rust implementation units. +4. Move concrete configuration validation and secret metadata into the + generated catalog. +5. Rewire the CLI and all adapters to the shared composition entry point. +6. Delete the old concrete directory, fixed builder table, validation list, + and concrete migration-guard inventory from core. + +The final change is atomic from an operator's perspective. There is no dual +configuration or deprecation period because the configuration does not change. + +## Testing and Verification + +### Discovery tests + +- Every valid Rust directory appears exactly once in generated output. +- Directory name, manifest ID, and generated module ID match. +- Missing `mod.rs`, missing manifests, duplicate order values, and malformed + IDs fail generation. +- A required JavaScript directory without `index.ts` fails generation. +- JavaScript-only and Rust-only directories are accepted. +- Generated bundle hashes match embedded bytes. + +### Parity tests + +- The generated Rust catalog contains all fifteen current implementation IDs. +- Enabled registration IDs and order match pre-move behavior for representative + settings. +- Immediate, deferred, standalone, and absent module selections match current + behavior. +- Configuration validation accepts and rejects the same fixtures. +- Secret-field metadata remains equivalent. +- Route tables, hook order, and duplicate detection remain equivalent. +- DataDome response privacy and client-tag suppression remain equivalent. +- GPT diagnostics request preparation, bootstrap injection, finalization, and + cache behavior remain equivalent on every adapter path. +- APS and Prebid plan compilation, transport, parsing, and auction results + remain equivalent. +- The APS browser renderer passes its existing validation, sandbox, messaging, + timeout, and rendering tests through the neutral renderer registration. + +### Boundary guards + +Automated source and dependency guards verify that: + +- `trusted-server-core` has no dependency on either integrations crate. +- Browser core contains no import from an integration directory. +- Core has no concrete builder or deploy-validation inventory. +- Adapters and the CLI do not import concrete integration modules. +- The old `trusted-server-core/src/integrations` and + `trusted-server-js/lib/src/integrations` directories no longer exist. + +These guards target implementation coupling. They do not reject existing +domain or wire types merely because a stable type name contains `Aps`, `Gpt`, +or another integration name. + +### Repository gates + +Before handoff, run the full project gates from `AGENTS.md`, including: + +- Rust formatting. +- All target-matched clippy aliases. +- Fastly, Axum, Cloudflare, and Spin tests. +- CLI tests and integration parity tests. +- Native and required WASM compilation for both new crates through their + consumers. +- JavaScript builds, Vitest suites, and formatting for both browser crates. +- Documentation formatting. + +## Risks and Mitigations + +### Hidden reverse dependencies + +Some concrete integrations use core-private helpers. Moving each module may +tempt broad visibility changes. + +Mitigation: inventory each use, move integration-owned helpers outward, and +expose only the smallest unavoidable neutral core API. Treat new public surface +as a reviewed deliverable. + +### Ordering drift + +Filesystem iteration order must not decide runtime hook order. + +Mitigation: require unique directory-local order values, sort generated output, +and pin parity with tests. + +### Divergent validation paths + +Runtime startup and CLI deploy validation could consume different catalogs. + +Mitigation: both use the same generated integration definitions. There is no +secondary validation list. + +### Stale JavaScript artifacts + +Splitting the Node build can accidentally embed a previous bundle. + +Mitigation: retain the current refusal to use stale output after a failed build, +generate hashes from the copied output bytes, and test every embedded hash. + +### APS load-order regression + +Moving APS renderer code out of browser core can leave the renderer unavailable +when an auction response arrives. + +Mitigation: include APS immediately whenever its plan can emit the descriptor, +load core before integrations, and add end-to-end renderer-dispatch tests. + +## Acceptance Criteria + +The design is complete when all of the following are true: + +1. Both new crates are workspace members and statically linked by every runtime + adapter and the CLI where appropriate. +2. All fifteen current concrete Rust implementation units live under + `trusted-server-integrations/src//`. +3. All integration browser sources, assets, fixtures, and tests live under + `trusted-server-integrations-js`. +4. Rust and JavaScript directories are discovered without a central integration + allowlist. +5. Core owns only neutral integration contracts and runtime execution. +6. Browser core imports no concrete integration. +7. APS, Prebid, and `adserver_mock` are not exceptions to the Rust move. +8. The CLI and all adapters use the shared static composition path. +9. Current configuration and runtime behavior pass parity tests. +10. The full repository verification gates pass. + +## Deferred Work + +The following require separate designs and real consumers: + +- External vendor-owned crates. +- Runtime integration injection. +- Independent integration release and compatibility policies. +- Provider capability registration for identity, geo, device, permissions, + demand, or ad servers. +- EdgeZero composition or host-service changes. +- Moving CLI audit detection metadata into integration directories. From 5730688cbdd5414fd24cec1c066d581edfffc24a Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 17 Sep 2026 11:59:21 -0700 Subject: [PATCH 02/13] Revise integrations crate split design --- ...-09-16-split-integrations-crates-design.md | 662 ------------- ...-09-17-split-integrations-crates-design.md | 894 ++++++++++++++++++ 2 files changed, 894 insertions(+), 662 deletions(-) delete mode 100644 docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md create mode 100644 docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md diff --git a/docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md deleted file mode 100644 index 7331d662f..000000000 --- a/docs/superpowers/specs/2026-09-16-split-integrations-crates-design.md +++ /dev/null @@ -1,662 +0,0 @@ -# Split Integrations into Dedicated Rust and JavaScript Crates - -**Date:** 2026-09-16 -**Status:** Proposed -**Scope:** Move every concrete integration implementation out of -`trusted-server-core` and `trusted-server-js` into two statically compiled, -directory-discovered workspace crates - -## Summary - -Trusted Server will separate its concrete integrations from its neutral server -and browser runtimes. - -The workspace gains two crates: - -- `trusted-server-integrations`, containing every concrete Rust integration. -- `trusted-server-integrations-js`, containing every integration-specific - browser module, browser asset, and JavaScript test. - -Each crate discovers integrations from its own directories at build time. The -Rust and JavaScript inventories are independent because the repository has -Rust-only and JavaScript-only integrations. A Rust integration may declare that -it requires a same-named JavaScript module, and that reference is checked at -build time. - -`trusted-server-core` keeps only the neutral contracts and runtime machinery -needed to execute registrations. `trusted-server-js` keeps only the neutral -browser runtime. Adapters and the CLI use the statically linked -`trusted-server-integrations` crate as the composition root. - -This is a packaging and dependency-direction change. It is not the provider, -permission, configuration, or externally loaded integration system proposed by -PR #1084. - -## Context - -On `main` at `6cae7f5da`, concrete integrations live inside -`crates/trusted-server-core/src/integrations`. That directory contains the -neutral registry alongside fifteen concrete implementation units: - -- `adserver_mock` -- `aps` -- `datadome` -- `didomi` -- `google_tag_manager` -- `gpt` -- `gpt_diagnostics` -- `js_asset_proxy` -- `lockr` -- `nextjs` -- `osano` -- `permutive` -- `prebid` -- `sourcepoint` -- `testlight` - -The ordinary builder table registers twelve of those units. APS and Prebid -register separately from the compiled auction plan, while `adserver_mock` -registers the current mock mediator. That difference in construction does not -justify leaving those implementations in core. - -Integration browser code lives beside browser core under -`crates/trusted-server-js/lib/src/integrations`. The current build discovers -directories containing `index.ts`, emits one IIFE per discovered entry point, -and embeds those bundles and hashes into the `trusted-server-js` Rust crate. -The directory also contains APS renderer code that browser core imports -directly even though APS has no current `index.ts` entry point. - -This layout has four problems: - -1. Core contains both the integration contract and every implementation. -2. Rust registration, deploy validation, and migration guards maintain - separate concrete inventories. -3. Browser core and integration code have dependencies in both directions. -4. Adding an in-tree integration requires editing central hand-maintained - lists instead of adding a self-contained directory. - -PR #1084 identifies these problems but combines their solution with external -vendor crates, runtime registration injection, provider capabilities, -configuration changes, permission work, and adapter composition changes. This -specification takes only the shared-crate and build-time discovery decisions. - -## Goals - -1. Move all fifteen concrete Rust implementation units into one - `trusted-server-integrations` crate. -2. Move all integration-specific TypeScript, JavaScript assets, fixtures, and - JavaScript tests into one `trusted-server-integrations-js` crate. -3. Give every concrete Rust implementation its own directory. -4. Discover the Rust and JavaScript inventories at build time. -5. Remove concrete integration construction and validation tables from core. -6. Preserve current configuration, routes, ordering, auction behavior, wire - formats, and request/response behavior. -7. Make core compile without depending on either integrations crate. -8. Keep the two new crates statically linked workspace components. - -## Meaning of “Concrete Integration” - -For this change, a concrete integration is an implementation currently under -`trusted-server-core/src/integrations`, integration-specific browser code under -`trusted-server-js/lib/src/integrations`, or direct construction and lifecycle -plumbing that imports one of those implementations. - -The extraction does not require renaming or relocating every existing domain -type, compatibility field, comment, or wire type that mentions APS, GPT, -Prebid, or another integration. For example, a serialized auction renderer -descriptor may remain in core when it is part of the existing core wire model. -Such types move only when leaving them in place would create a dependency from -core to a concrete implementation. - -This boundary keeps the requested extraction complete without turning it into -an auction-domain or configuration-model rewrite. - -## Non-Goals - -This specification does not introduce: - -- External integration crates. -- Runtime-loaded integrations or dynamic registration. -- Per-vendor ownership or independent release lifecycles. -- A public plugin SDK. -- Identity, geo, device, permission, demand, or ad-server provider systems. -- Configuration key, schema, or file-format changes. -- New integrations or new integration capabilities. -- A redesign of the auction plan, ranking, mediation, or renderer wire format. -- EdgeZero lifecycle, evidence, store, or adapter changes. -- A general-purpose hook or capability language. -- A reorganization of the CLI audit analyzer or other integration-related code - that is not part of the implementation, validation, or registration paths - being extracted. - -The new cross-crate interfaces exist only to preserve the current statically -compiled application. Designing them so a future external crate could use them -is explicitly deferred until a real external consumer exists. - -## Target Workspace Layout - -```text -crates/ - trusted-server-core/ - src/ - integration/ - mod.rs - registry.rs - - trusted-server-js/ - lib/src/core/ - src/ - - trusted-server-integrations/ - build.rs - Cargo.toml - src/ - lib.rs - adserver_mock/ - integration.toml - mod.rs - aps/ - integration.toml - mod.rs - datadome/ - integration.toml - mod.rs - protection.rs - protection_scope.rs - ... - nextjs/ - integration.toml - mod.rs - html_post_process.rs - rsc.rs - rsc_placeholders.rs - script_rewriter.rs - shared.rs - fixtures/ - - trusted-server-integrations-js/ - build.rs - Cargo.toml - lib/ - package.json - src/ - aps/ - index.ts - render.ts - creative/ - index.ts - ... - datadome/ - index.ts - ... - ... - test/ - integrations/ - fixtures/ - src/ - lib.rs -``` - -Every flat Rust implementation file becomes `/mod.rs`. Existing nested -modules and fixtures stay with their owning integration. The GPT bootstrap -script moves to `trusted-server-integrations-js` and is exported as an -integration asset rather than remaining beside Rust source. - -JavaScript-only `creative` remains valid without a Rust directory. Rust-only -integrations remain valid without a JavaScript directory. - -## Dependency Direction - -The dependency graph is one-way: - -```text -trusted-server-js - ^ - | -trusted-server-integrations-js - ^ - | -trusted-server-integrations ----> trusted-server-core ----> trusted-server-js - ^ ^ - | | - +------------- adapters -----------+ - +--------------- CLI -``` - -The diagram shows logical dependencies. Cargo may deduplicate a shared -`trusted-server-js` dependency; no cycle is permitted. - -The rules are: - -1. Core never depends on `trusted-server-integrations` or - `trusted-server-integrations-js`. -2. Concrete Rust integrations may use public core contracts and domain types. -3. Integration JavaScript may import the explicit browser-core source API. -4. Browser core must not import a concrete integration. -5. Adapters and the CLI compose core with the built-in integrations crate. - -## Neutral Core Contract - -The existing neutral contents of `integrations/registry.rs` move to a singular -`trusted_server_core::integration` module. The singular name distinguishes the -contract from the collection of concrete implementations. - -Core continues to own: - -- `IntegrationRegistration` and its builder. -- `IntegrationRegistry` and registry execution. -- Request filtering, proxy, rewriting, head injection, HTML post-processing, - and other neutral hook traits and contexts. -- Duplicate ID and route detection. -- Neutral script-module metadata used by publishing and HTML injection. -- Test helpers for empty or stub registries. - -`IntegrationRegistry` no longer constructs built-ins. Its production -constructor accepts completed registrations and the existing compiled auction -plan. The fixed `builders()` table, APS/Prebid special construction, and -concrete `IntegrationRegistry::with_plan` assembly move to -`trusted-server-integrations`. - -Core tests that need only registry behavior use neutral stubs. Tests that need -the actual built-in catalog move to or depend on the integrations crate at the -outer composition layer. - -## Rust Directory Discovery - -`trusted-server-integrations/build.rs` scans immediate directories under -`src/`. A Rust integration directory must contain: - -- `mod.rs`. -- `integration.toml`. - -The directory-local manifest is deliberately small: - -```toml -id = "gpt" -order = 80 -javascript = true -``` - -The fields mean: - -- `id` must equal the directory name and use Rust `snake_case`. -- `order` preserves current registration and hook order. Ordering metadata is - owned by the integration rather than a central list. -- `javascript` states whether a same-named directory with `index.ts` must exist - in `trusted-server-integrations-js`. - -No provider, permission, configuration, dependency, or ownership metadata is -added to this manifest. - -The build script generates module declarations and an ordered definition -table. It fails the build for: - -- A malformed or mismatched ID. -- A missing manifest or `mod.rs`. -- Duplicate IDs or order values. -- A JavaScript requirement whose same-named `index.ts` is absent. -- Generated output that would be empty. - -The script emits `cargo:rerun-if-changed` directives for the discovered Rust -directories and for JavaScript directories referenced by a Rust manifest. - -Each generated module exposes one crate-private definition function. The -returned internal definition may carry only the contributions already needed -by current code: - -- Deploy and startup validation. -- An ordinary page registration. -- An auction profile or transport implementation. -- The current mock mediator implementation. -- An optional JavaScript module or asset reference. - -This internal definition is not exported as a plugin contract. APS, Prebid, -and `adserver_mock` use the same generated inventory as every other directory, -even though their existing contributions are different. - -## JavaScript Directory Discovery - -`trusted-server-integrations-js` owns its Node project, tests, build pipeline, -generated Rust module catalog, and integration assets. - -Its JavaScript build discovers immediate directories containing `index.ts`, -sorts them deterministically, and emits one self-contained IIFE per directory. -Its Cargo build script embeds each bundle and its SHA-256 hash, following the -current `trusted-server-js` mechanism. - -The inventories remain independent: - -- A JavaScript-only directory such as `creative` is built without a Rust - registration. -- A Rust directory with `javascript = false` does not require a browser module. -- A Rust directory with `javascript = true` requires a same-named browser entry - point and receives that generated module at composition time. - -There is no central allowlist spanning the crates. - -`trusted-server-js` builds only the core IIFE and exposes neutral helpers for -combining the core bundle with supplied integration modules. The concatenation -order remains core first, followed by immediate integration modules in registry -order. Deferred and standalone modules remain separately addressable. - -## Static Composition - -`trusted-server-integrations` is the only built-in composition root. It uses -the generated definition table in this order: - -1. Run integration-aware deploy or startup validation as requested by the - caller. -2. Collect the existing auction profile inputs needed to compile the canonical - auction plan. -3. Ask core to compile the plan using those inputs. -4. Build the existing mediator and integration registrations against that plan. -5. Resolve required bundles and assets from - `trusted-server-integrations-js`. -6. Pass completed registrations to core's neutral registry constructor. - -The exact Rust function names are left to the implementation plan, but the -composition path must be shared. The four adapters must not recreate the -generated catalog or call concrete integrations directly. - -At runtime: - -1. An adapter holds the completed core registry and auction state as it does - today. -2. Core invokes neutral registry hooks during request and response processing. -3. Script generation starts with the core bundle and reads enabled integration - bundles from the registry. -4. Deferred and standalone script requests resolve against registry-carried - module metadata rather than a global concrete list in core. - -No runtime directory scanning or dynamic loading occurs. - -## Configuration and CLI Validation - -Configuration keys and serialized shapes do not change. - -Core retains global settings parsing and global validation. Concrete config -types and integration-specific validation move with their implementations. -The generated definition table supplies integration validation to both runtime -startup and the CLI. - -The integration-aware typed app-config wrapper composes: - -- Core secret-field metadata and global validation. -- Generated integration secret-field metadata and validation. - -The CLI uses this composed wrapper for `config validate`, `config diff`, and -`config push`. This removes the concrete validation table and DataDome-specific -secret paths from core without changing the published `trusted-server.toml` -shape. - -Unknown or invalid integration settings continue to fail with the current -error contexts. An integration present in configuration but absent from the -compiled catalog must fail rather than be silently ignored. - -## Required Neutral Lifecycle Hooks - -Moving every implementation exposes two existing reverse dependencies that -must become neutral registry behavior. - -### Response sharing annotation - -DataDome currently communicates a concrete request marker back into core so -core buffers the full response and applies private caching. Replace that marker -with a neutral response-sharing annotation owned by core. DataDome sets the -annotation through its registered hook; core performs the same buffering and -cache behavior without importing a DataDome type. - -This annotation does not introduce a general policy system. It represents only -the existing shared-versus-request-private decision already consumed by core. - -### Request preparation and response finalization - -GPT diagnostics currently has direct preparation calls in every adapter and in -core, plus direct finalization in core. Add neutral registration hooks for -those two lifecycle points. The registry owns any opaque request-scoped state -between them. - -Adapters call the registry preparation operation at the same request boundaries -used today. Core calls finalization on the same response path used today. -Integration-specific bootstrap and script decisions flow through the existing -head-injection and document-state mechanisms instead of concrete fields on -`HtmlProcessorConfig`. - -No other lifecycle stages are added. - -## Auction-Coupled Implementations - -APS, Prebid, and `adserver_mock` move with all other concrete implementations. -Core keeps the generic auction plan, orchestration, request, response, and wire -types. - -The smallest cross-crate inputs needed by the current implementations become -public neutral core APIs: - -- Profile compilation registrations consumed by the existing plan compiler. -- Provider transport and response callbacks consumed by the existing generic - provider path. -- The optional mediator passed to the existing orchestrator. - -The integrations crate supplies only the current built-ins through these APIs. -This work must not change provider configuration, plan semantics, routing, -timeouts, response normalization, ranking, mediation, or telemetry. - -Existing APS-specific serialized types may remain in core where they are part -of the established browser wire contract. The concrete APS parsing, -validation, transport, and rendering implementation moves. - -## Browser APS Boundary - -Browser core currently imports APS renderer parsing and dispatch directly. -That reverse dependency must end when APS moves. - -Browser core will own one narrow bid-renderer registration mechanism: - -- Core parses the existing renderer envelope only far enough to identify its - type and retain its opaque payload. -- An integration module registers the parser and dispatcher for its renderer - type during IIFE initialization. -- The APS browser module registers the existing `aps` renderer implementation. -- Core dispatches a renderer bid through the registered implementation. -- An absent or rejecting renderer continues to fail closed without executing - unvalidated creative code. - -The APS module is included in the immediate script set whenever the compiled -auction plan can emit an APS renderer descriptor. Core is loaded first, so the -registration exists before application code can request and render bids. - -The serialized renderer descriptor, APS validation rules, sandbox flags, -message authentication, timeouts, and render results remain unchanged. This is -not a general creative-renderer redesign; it is the minimum inversion needed -to remove the core-to-APS source import. - -## Error Handling - -Failures remain fail-closed and occur as early as the information permits. - -Build-time failures include malformed directories, invalid manifests, -duplicate discovery metadata, missing JavaScript entry points, JavaScript build -failures, and missing generated bundles. - -Startup or deploy-validation failures include duplicate integration IDs, -duplicate routes, invalid integration configuration, unresolved script assets, -and incompatible auction contributions. - -Runtime hook errors retain the current `Report` contexts -and response behavior. Moving a call behind the registry must not turn an -existing error into a log-and-continue path or a panic. - -Core visibility changes must be narrow. A private helper moves with its -integration when possible. When an integration genuinely needs a core helper, -the implementation exposes the smallest named API and documents it. The change -must not broadly convert core modules or fields to `pub`. - -## Compatibility - -The following are compatibility requirements: - -- Existing `trusted-server.toml` files require no migration. -- Integration IDs and enablement rules do not change. -- Registration and hook order do not change. -- Routes and endpoint behavior do not change. -- Immediate, deferred, and standalone delivery decisions do not change, except - that APS becomes an explicit integration module required by its plan. -- Auction requests, responses, renderer descriptors, and telemetry do not - change. -- Cache privacy and full-buffer decisions do not change. -- All four adapters expose the same routes and behaviors as before. - -Bundle hashes and cache-busting URLs may change because browser core and -integration code are rebuilt in different crates. The server must always emit -URLs matching the newly generated hashes, so old and new artifacts cannot be -confused in cache. No stable bundle hash is part of the compatibility contract. - -## Migration Sequence - -Implementation may use small commits, but the merged workspace must never -contain two active built-in catalogs. - -1. Establish the neutral script-module and lifecycle contracts in core without - changing behavior. -2. Create `trusted-server-integrations-js`, move integration browser sources - and tests, and remove concrete imports from browser core. -3. Create `trusted-server-integrations`, add directory discovery, and move all - fifteen Rust implementation units. -4. Move concrete configuration validation and secret metadata into the - generated catalog. -5. Rewire the CLI and all adapters to the shared composition entry point. -6. Delete the old concrete directory, fixed builder table, validation list, - and concrete migration-guard inventory from core. - -The final change is atomic from an operator's perspective. There is no dual -configuration or deprecation period because the configuration does not change. - -## Testing and Verification - -### Discovery tests - -- Every valid Rust directory appears exactly once in generated output. -- Directory name, manifest ID, and generated module ID match. -- Missing `mod.rs`, missing manifests, duplicate order values, and malformed - IDs fail generation. -- A required JavaScript directory without `index.ts` fails generation. -- JavaScript-only and Rust-only directories are accepted. -- Generated bundle hashes match embedded bytes. - -### Parity tests - -- The generated Rust catalog contains all fifteen current implementation IDs. -- Enabled registration IDs and order match pre-move behavior for representative - settings. -- Immediate, deferred, standalone, and absent module selections match current - behavior. -- Configuration validation accepts and rejects the same fixtures. -- Secret-field metadata remains equivalent. -- Route tables, hook order, and duplicate detection remain equivalent. -- DataDome response privacy and client-tag suppression remain equivalent. -- GPT diagnostics request preparation, bootstrap injection, finalization, and - cache behavior remain equivalent on every adapter path. -- APS and Prebid plan compilation, transport, parsing, and auction results - remain equivalent. -- The APS browser renderer passes its existing validation, sandbox, messaging, - timeout, and rendering tests through the neutral renderer registration. - -### Boundary guards - -Automated source and dependency guards verify that: - -- `trusted-server-core` has no dependency on either integrations crate. -- Browser core contains no import from an integration directory. -- Core has no concrete builder or deploy-validation inventory. -- Adapters and the CLI do not import concrete integration modules. -- The old `trusted-server-core/src/integrations` and - `trusted-server-js/lib/src/integrations` directories no longer exist. - -These guards target implementation coupling. They do not reject existing -domain or wire types merely because a stable type name contains `Aps`, `Gpt`, -or another integration name. - -### Repository gates - -Before handoff, run the full project gates from `AGENTS.md`, including: - -- Rust formatting. -- All target-matched clippy aliases. -- Fastly, Axum, Cloudflare, and Spin tests. -- CLI tests and integration parity tests. -- Native and required WASM compilation for both new crates through their - consumers. -- JavaScript builds, Vitest suites, and formatting for both browser crates. -- Documentation formatting. - -## Risks and Mitigations - -### Hidden reverse dependencies - -Some concrete integrations use core-private helpers. Moving each module may -tempt broad visibility changes. - -Mitigation: inventory each use, move integration-owned helpers outward, and -expose only the smallest unavoidable neutral core API. Treat new public surface -as a reviewed deliverable. - -### Ordering drift - -Filesystem iteration order must not decide runtime hook order. - -Mitigation: require unique directory-local order values, sort generated output, -and pin parity with tests. - -### Divergent validation paths - -Runtime startup and CLI deploy validation could consume different catalogs. - -Mitigation: both use the same generated integration definitions. There is no -secondary validation list. - -### Stale JavaScript artifacts - -Splitting the Node build can accidentally embed a previous bundle. - -Mitigation: retain the current refusal to use stale output after a failed build, -generate hashes from the copied output bytes, and test every embedded hash. - -### APS load-order regression - -Moving APS renderer code out of browser core can leave the renderer unavailable -when an auction response arrives. - -Mitigation: include APS immediately whenever its plan can emit the descriptor, -load core before integrations, and add end-to-end renderer-dispatch tests. - -## Acceptance Criteria - -The design is complete when all of the following are true: - -1. Both new crates are workspace members and statically linked by every runtime - adapter and the CLI where appropriate. -2. All fifteen current concrete Rust implementation units live under - `trusted-server-integrations/src//`. -3. All integration browser sources, assets, fixtures, and tests live under - `trusted-server-integrations-js`. -4. Rust and JavaScript directories are discovered without a central integration - allowlist. -5. Core owns only neutral integration contracts and runtime execution. -6. Browser core imports no concrete integration. -7. APS, Prebid, and `adserver_mock` are not exceptions to the Rust move. -8. The CLI and all adapters use the shared static composition path. -9. Current configuration and runtime behavior pass parity tests. -10. The full repository verification gates pass. - -## Deferred Work - -The following require separate designs and real consumers: - -- External vendor-owned crates. -- Runtime integration injection. -- Independent integration release and compatibility policies. -- Provider capability registration for identity, geo, device, permissions, - demand, or ad servers. -- EdgeZero composition or host-service changes. -- Moving CLI audit detection metadata into integration directories. diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md new file mode 100644 index 000000000..c6877ad0d --- /dev/null +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -0,0 +1,894 @@ +# Split Integrations into Dedicated Crates and One Ordered Configuration + +**Date:** 2026-09-17 + +**Status:** Proposed + +**Scope:** Move all concrete Rust and browser integrations into two statically +compiled workspace crates, introduce typed multi-capability registration, and +make ordered `[integrations]` configuration the single inventory for concrete +integrations and their auction providers. + +## Summary + +Trusted Server will separate concrete integrations from its neutral Rust and +browser runtimes. + +The workspace gains two crates: + +- `trusted-server-integrations`, containing all concrete Rust integrations. +- `trusted-server-integrations-js`, containing all integration-specific + TypeScript, JavaScript assets, fixtures, tests, and generated bundles. + +`trusted-server-core` will retain neutral integration contracts and execution +engines. `trusted-server-js` will retain the neutral browser runtime. The +adapters and CLI will use `trusted-server-integrations` as the statically linked +application composition root. + +Each Rust integration will expose one definition that may register multiple +typed capabilities. APS, for example, may register proxy, head-injection, +JavaScript, and OpenRTB-profile capabilities. Capabilities are subsystem-owned +types and traits, not a single `IntegrationType` enum and not a discriminator +in operator configuration. + +`[integrations]` will become the only operator inventory for concrete +integrations. An integration will own all of its settings, including any named +auction provider instances. Global `[auction]` settings will continue to own +cross-integration orchestration such as the auction timeout, bidder routing, +creative policy, and mediator selection. + +Runtime order will come exclusively from TOML declaration order. The config +push and config-store representation will preserve that order explicitly; +filesystem discovery order will never affect execution. + +This design intentionally changes the auction configuration introduced by PR +#1016 while preserving that work's compiled-plan and runtime guarantees. It +adopts only the typed-registration portion of PR #1084 and excludes that PR's +external provider ecosystem and unrelated provider systems. + +## Context + +On `main` at `6cae7f5da`, neutral registry machinery and concrete integrations +share `crates/trusted-server-core/src/integrations`. The concrete Rust units +are: + +- `adserver_mock` +- `aps` +- `datadome` +- `didomi` +- `google_tag_manager` +- `gpt` +- `gpt_diagnostics` +- `js_asset_proxy` +- `lockr` +- `nextjs` +- `osano` +- `permutive` +- `prebid` +- `sourcepoint` +- `testlight` + +The ordinary builder table registers twelve units. APS and Prebid are also +constructed from the compiled auction plan, and `adserver_mock` supplies the +current mediator. Core additionally imports concrete APS and Prebid code from +the OpenRTB profile and provider paths, concrete DataDome response state, and +GPT diagnostics lifecycle functions. + +Integration browser code shares a Node project with browser core under +`crates/trusted-server-js/lib/src/integrations`. The build discovers +directories containing `index.ts`, emits an IIFE for each entry point, and +embeds bundles and hashes into the `trusted-server-js` Rust crate. APS renderer +code is imported directly by browser core even though APS does not currently +have its own `index.ts`. + +Configuration is also split by implementation detail. Browser/page settings +use `[integrations.]`, while server auction providers use +`[auction.providers.]` plus `profile = "aps"` or +`profile = "prebid-server"`. One logical APS integration is therefore +configured in two inventories and may be activated implicitly by an auction +plan. `IntegrationSettings` currently uses `HashMap`, so integration +declaration order is discarded before registry construction. + +These conditions produce five related problems: + +1. Core owns both neutral contracts and concrete implementations. +2. Rust registration, auction profiles, deploy validation, and migration guards + maintain separate concrete inventories. +3. Browser core imports integration-specific code. +4. One logical integration can be configured and activated through unrelated + locations. +5. Runtime integration order is not a stable property of the operator + configuration. + +## Relationship to Existing Work + +### PR #1016 + +PR #1016 made auction providers configuration-driven and introduced a single +validated, immutable auction plan shared by adapter backend construction, +runtime dispatch, browser demand, routing, and telemetry. It also separated a +configured provider instance from the OpenRTB profile implementation it uses. + +This design preserves those runtime guarantees: + +- Multiple configured instances may use one integration implementation. +- Qualified provider IDs remain the stable identity shared by bidder routing, + backend correlation, diagnostics, and telemetry. +- One validated plan remains authoritative across every adapter and runtime + consumer. +- The generic OpenRTB transport remains shared. +- Existing routing, timeout, notification, response admission, mediation, and + telemetry attribution behavior remains unchanged. + +This design changes the configuration location and identity spelling. Provider +instances move below their owning integration, and cross-integration references +use a qualified `.` identifier. + +### PR #1084 + +PR #1084 proposes a much broader compile-time provider ecosystem: public +registration for external vendor crates, provider systems for identity, geo, +device, permission signals, demand and ad servers, a permission/jurisdiction +model, client-cycle EC resolution, provider-code governance, adapter and +EdgeZero composition work, and independent vendor ownership expectations. + +This design shares one idea with that proposal: one integration may register +multiple typed capabilities. It does not create the external ecosystem. The +new contracts serve the integrations compiled in this workspace; they are not +a stable third-party SDK or independent release boundary. + +## Goals + +1. Move every concrete Rust integration implementation out of core and into one + `trusted-server-integrations` crate. +2. Move all integration-specific browser sources and artifacts into one + `trusted-server-integrations-js` crate. +3. Give each concrete integration one Rust directory and, when applicable, one + same-named browser directory. +4. Discover Rust and browser integration inventories from directories at build + time. +5. Let one integration register multiple typed capabilities without a global + integration-kind enum. +6. Remove concrete integration construction, auction-profile, validation, and + lifecycle imports from core. +7. Make `[integrations]` the single ordered inventory for concrete integration + configuration, including auction providers. +8. Preserve the runtime behavior and compiled-plan guarantees of PR #1016. +9. Make core compile without depending on either integrations crate. +10. Keep all integrations statically linked; no runtime loading is introduced. + +## Non-Goals + +This design does not introduce: + +- One Cargo crate per vendor. +- External vendor crate injection or adapter-supplied registration. +- Runtime-loaded integrations, dynamic linking, or an ABI. +- A stable public plugin or integration SDK. +- Independent vendor release, compatibility, security-response, or governance + policies. +- Identity, EC, geo, device, or permission-signal provider systems. +- A jurisdiction or permission-policy redesign. +- Client-cycle EC resolution or provider-code allocation. +- EdgeZero lifecycle, host-evidence, store, or adapter changes. +- New auction protocols, bidding behavior, ranking, notification behavior, or + telemetry semantics. +- A reorganization of CLI audit detection that is unrelated to configuration + validation and composition. + +The design adds only capabilities needed to move current implementations. A +future non-OpenRTB provider, external crate, or new provider family requires a +separate design with a real consumer. + +## Terms + +The following terms are distinct: + +- **Integration definition:** the statically discovered code definition for a + stable integration ID such as `aps`. +- **Capability registration:** one typed contribution made by an integration, + such as a proxy, HTML rewriter, JavaScript module, OpenRTB profile, or + mediator. +- **Integration configuration:** the single ordered operator block at + `[integrations.]` that activates and configures the definition. +- **Auction provider instance:** one named endpoint and policy configuration + below an integration, such as `aps.main`. Multiple instances may use the same + integration implementation. +- **Integration registry:** core runtime state containing the enabled page, + request, response, and browser capabilities in configuration order. +- **Auction plan:** core runtime state containing the validated configured + provider instances, routes, and orchestration policy. + +An integration is therefore a container for capabilities; it is not itself a +single capability type. + +## Target Workspace Layout + +```text +crates/ + trusted-server-core/ + src/ + integration/ + mod.rs + registry.rs + + trusted-server-js/ + lib/ + src/core/ + src/ + + trusted-server-integrations/ + build.rs + Cargo.toml + src/ + lib.rs + adserver_mock/ + mod.rs + aps/ + mod.rs + datadome/ + mod.rs + protection.rs + protection_scope.rs + didomi/ + mod.rs + ... + nextjs/ + mod.rs + html_post_process.rs + rsc.rs + rsc_placeholders.rs + script_rewriter.rs + shared.rs + fixtures/ + openrtb/ + mod.rs + + trusted-server-integrations-js/ + build.rs + Cargo.toml + lib/ + package.json + src/integrations/ + aps/ + index.ts + render.ts + creative/ + index.ts + datadome/ + index.ts + ... + test/ + integrations/ + fixtures/ + src/ + lib.rs +``` + +Every current flat Rust integration file becomes `/mod.rs`. Existing +nested modules and fixtures stay with their owner. `openrtb` is a built-in +Rust-only integration that exposes configuration for the current standard +OpenRTB profile without turning the neutral OpenRTB execution engine into a +concrete integration. + +JavaScript-only `creative` remains valid without a Rust directory. Rust-only +integrations remain valid without a browser directory. + +## Dependency Direction + +The Cargo dependency graph is one-way: + +```text +trusted-server-integrations ──→ trusted-server-core ──→ trusted-server-js + │ + └───────────────→ trusted-server-integrations-js + +adapters and CLI ────────────→ trusted-server-integrations +adapters and CLI ────────────→ trusted-server-core +``` + +The rules are: + +1. Core never depends on either integrations crate. +2. Concrete Rust integrations use public neutral core contracts and domain + types. +3. `trusted-server-integrations` links Rust definitions with generated browser + modules from `trusted-server-integrations-js`. +4. Integration TypeScript may use the explicit browser-core API, but browser + core never imports a concrete integration. +5. Every adapter and the CLI uses the same composition and validation entry + points from `trusted-server-integrations`. +6. No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or + another integration module directly. + +## Directory Discovery + +### Rust + +`trusted-server-integrations/build.rs` scans immediate directories under +`src/`. A directory containing `mod.rs` is a concrete integration whose stable +ID is the directory name. IDs must use the existing integration ID grammar and +must be unique. + +The build script generates module declarations and a definition catalog. Its +lexical sorting makes generated source reproducible but has no runtime ordering +meaning. Each module must expose the expected crate-private `definition` +function; failure to do so is a compile error. + +There is no directory-local numeric order. Runtime order belongs to +configuration. + +The build fails for malformed IDs, duplicate normalized IDs, unreadable +directories, or an empty catalog. It emits `cargo:rerun-if-changed` directives +for the discovered directories. + +### JavaScript + +`trusted-server-integrations-js` owns its Node project, integration tests, +build pipeline, generated Rust module catalog, and browser assets. Its build +discovers immediate directories containing `index.ts` and emits one +self-contained IIFE per entry point. Its Cargo build embeds each output and its +SHA-256 hash. + +The generated Rust API exposes typed module identifiers rather than accepting +unchecked strings. A Rust registration referencing a missing browser module +therefore fails compilation. JavaScript-only modules are valid and need no Rust +definition. + +Rust and JavaScript discovery are independent; neither filesystem inventory is +treated as the canonical list for the other. + +## Neutral Core Contracts + +The neutral contents of `integrations/registry.rs` move to singular +`trusted_server_core::integration`. Core continues to own: + +- `IntegrationDefinition` and `IntegrationRegistration` contracts. +- `IntegrationRegistry` and registry execution. +- Proxy, request-filter, attribute-rewriter, script-rewriter, + HTML-post-processor, and head-injector traits and contexts. +- Neutral request-preparation and response-finalization hooks. +- Neutral response-sharing/private-cache annotations. +- Neutral JavaScript module metadata and load modes. +- OpenRTB profile registration contracts consumed by the generic plan and + transport engines. +- Mediator registration contracts consumed by auction orchestration. +- Duplicate route, ID, renderer-type, and capability detection. +- Empty and stub registrations for core tests. + +Core does not own a global `IntegrationType` enum. The builder has typed methods +for each supported contribution, and one definition may supply any compatible +combination. The initial methods correspond only to behavior present in the +repository. + +An APS definition conceptually registers: + +```text +APS +├── proxy capability +├── head-injection capability +├── JavaScript renderer module +└── OpenRTB profile capability +``` + +The OpenRTB profile contract replaces the closed +`CompiledOpenRtbProfile::{Aps, PrebidServer, ...}` dependency. It supplies the +profile-owned operations required by the existing generic engine, including +typed configuration compilation, request specialization, response parsing, +diagnostics, and renderer information. Core invokes the contract without +matching on vendor variants or importing integration types. + +Compilation returns an `Arc`-backed trait object representing one immutable +compiled profile. The auction plan stores that object beside the common +provider settings, and the generic OpenRTB engine calls its typed methods. No +`Any` downcast or vendor-keyed side table is used. + +The standard OpenRTB implementation is registered by the built-in `openrtb` +integration. APS and Prebid register their implementations from their own +directories. `adserver_mock` registers the existing mediator capability. + +No unused generic `AuctionProviderFactory` extension is added. A provider that +cannot use the current OpenRTB engine will define that additional seam in a +future design. + +## Static Catalog and Activation + +Directory discovery establishes what the binary supports. Configuration +establishes what runs. + +At startup or deploy validation: + +1. Parse `[integrations]` into an ordered sequence. +2. Resolve each ID against the generated definition catalog. +3. Ask the owning definition to parse and validate its complete configuration. +4. For each enabled integration, construct its typed capability registrations + in TOML order. +5. Collect integration-owned auction provider instances and profiles. +6. Ask core to compile the single canonical auction plan. +7. Resolve typed browser modules and construct the neutral integration + registry. + +An absent integration is inactive. An explicitly disabled integration is +validated but contributes no runtime capabilities. Configuration cannot +activate APS through an auction plan while omitting `[integrations.aps]`. + +The four adapters and CLI share this path. Runtime and deploy validation cannot +use different catalogs or integration schemas. + +## One Integration Configuration + +### Operator-facing shape + +`[integrations]` is the single concrete integration inventory. No `type` field +is added; the table key resolves the statically compiled definition. + +```toml +[integrations.prebid] +enabled = true +client_side_bidders = ["example-browser"] + +[integrations.prebid.auction.providers.pbs-main] +endpoint = "https://prebid.example.com/openrtb2/auction" +timeout_ms = 1000 +routing = "explicit" +debug = false +test_mode = false +consent_forwarding = "both" + +[integrations.prebid.auction.providers.pbs-main.notifications] +suppress_all = false +suppress_seats = [] + +[integrations.aps] +enabled = true +rendering_mode = "trusted_server" + +[integrations.aps.auction.providers.aps-main] +endpoint = "https://aps.example.com/e/pb/bid" +timeout_ms = 800 +routing = "all_eligible" +account_id = "example-account" +debug = false +allow_script_creatives = false +``` + +The parent integration determines the implementation. Integration-owned +providers therefore do not accept `profile`, `implementation`, or +`profile_config`. Common provider fields and integration-specific profile +fields form one typed provider schema owned by that integration. Common +notification settings may remain in the nested `notifications` table. + +Multiple named provider instances are supported beneath one integration. + +The standard generic path uses the same inventory: + +```toml +[integrations.openrtb] +enabled = true + +[integrations.openrtb.auction.providers.example-direct] +endpoint = "https://bidder.example.com/openrtb" +timeout_ms = 1000 +routing = "explicit" +``` + +### Global auction configuration + +`[auction]` retains settings that coordinate integrations: + +```toml +[auction] +enabled = true +timeout_ms = 2000 +rewrite_creatives = true +sanitize_creatives = false +mediator = "adserver_mock" + +[auction.bidders.example-bidder] +provider = "prebid.pbs-main" +``` + +Provider references use the canonical `.` form. +The qualified value is the provider identity used by the compiled plan, +backend correlation, diagnostics, and telemetry. The local provider name may +repeat under different integrations without collision. + +The mediator selects an enabled integration that registered a mediator +capability. Its settings remain under that integration: + +```toml +[integrations.adserver_mock] +enabled = true +endpoint = "https://adserver.example.com/mediate" +timeout_ms = 500 +``` + +## Ordering Contract + +No manifest field, filename, alphabetic sort, hash-map iteration, or numeric +priority controls runtime order. + +The contract is: + +1. Integration order is the first explicit declaration order of parent + `[integrations.]` tables. +2. Each integration must have an explicit parent table; a nested auction table + cannot implicitly create or position it. +3. Provider order is declaration order under that integration's + `auction.providers` map. +4. Disabled integrations contribute nothing; remaining integrations keep their + relative order. +5. The flattened auction plan orders providers first by owning integration and + then by local provider declaration. +6. Hook and immediate/deferred JavaScript lists retain integration order. +7. Browser output is neutral browser core first, the existing fixed + JavaScript-only `creative` prelude second, and configured integration modules + afterward. + +The TOML parser must capture order directly. `IntegrationSettings` may not use +`HashMap` or another unordered representation. + +### Config-store representation + +JSON object member order is not an ordering contract. Config push therefore +converts the operator tables into an explicit ordered sequence in the signed +blob envelope. Nested provider maps are likewise encoded with explicit +sequence order. Runtime loading reconstructs ordered settings from those +sequences and never infers order from JSON object iteration. + +Conceptually, the stored representation carries: + +```json +{ + "integrations": [ + { + "id": "prebid", + "config": { + "enabled": true, + "auction": { + "providers": [ + { + "id": "pbs-main", + "config": { + "endpoint": "https://prebid.example.com/openrtb2/auction" + } + } + ] + } + } + }, + { "id": "aps", "config": { "enabled": true } } + ] +} +``` + +The exact private Rust types may differ, but the serialized order must be +explicit and covered by compatibility tests across: + +```text +trusted-server.toml + → typed CLI configuration + → signed blob envelope + → config store + → runtime Settings + → registry, JavaScript lists, and AuctionPlan +``` + +## Configuration Ownership and Validation + +Core retains global settings and auction-orchestration validation. Each +integration owns the typed schema and validation for its full configuration, +including its provider instances. + +The generated definition catalog supplies integration parsing, validation, +secret metadata, and capability construction to both runtime startup and the +CLI. `config validate`, `config diff`, and `config push` must use the same +catalog as the adapters. + +Validation fails for: + +- An unknown integration ID. +- Integration configuration not accepted by its owner. +- A nested provider table without an explicit parent integration table. +- Auction providers on an explicitly disabled integration. +- Duplicate local provider IDs or duplicate qualified provider identities. +- A bidder route to an unknown, disabled, or incompatible provider. +- A selected mediator whose integration is absent, disabled, or lacks the + mediator capability. +- Duplicate routes or renderer types. +- A referenced browser module absent from the generated browser catalog. +- An unsupported capability combination. + +A globally disabled auction may retain otherwise valid enabled integration and +provider configuration so operators can prepare configuration before enabling +the auction. + +## Configuration Migration + +This is a deliberate breaking migration. The runtime and CLI do not support +both inventories or define precedence between them. + +Representative mappings are: + +| Previous configuration | New configuration | +| --------------------------------------------------------------- | --------------------------------------------------------- | +| `[auction.providers.pbs-main]` with `profile = "prebid-server"` | `[integrations.prebid.auction.providers.pbs-main]` | +| `[auction.providers.aps-main]` with `profile = "aps"` | `[integrations.aps.auction.providers.aps-main]` | +| A standard profile provider named `example-direct` | `[integrations.openrtb.auction.providers.example-direct]` | +| `[auction.providers..profile_config]` | Flattened into the owning integration's provider table | +| Bidder route `provider = "pbs-main"` | `provider = "prebid.pbs-main"` | + +Old `[auction.providers]`, `profile`, and `profile_config` fields fail with an +actionable message naming the new integration-owned location. A mixed old/new +configuration also fails. There is no silent translation at runtime and no +deprecation interval. + +Examples, integration fixtures, environment-overlay tests, CLI documentation, +and operator guides migrate in the same change. + +## Required Neutral Lifecycle Boundaries + +### Response sharing annotation + +DataDome currently communicates a concrete marker to core so core buffers a +full response and applies private caching. Replace that marker with a neutral +response-sharing annotation owned by core. DataDome sets it through a +registered hook; core preserves the current buffering and cache behavior. + +The annotation represents only the existing shared-versus-request-private +decision. It is not a general policy or permissions system. + +### Request preparation and response finalization + +GPT diagnostics currently has direct preparation calls in adapters and core +and a direct finalization call in core. Add neutral typed hooks for those two +existing lifecycle points. The registry owns opaque request-scoped state +between them. + +Adapters invoke registry preparation at the existing boundaries. Core invokes +finalization on the existing response path. No additional lifecycle stages are +introduced. + +## Browser Composition and APS Renderer + +`trusted-server-js` builds only the neutral core IIFE and exposes a narrow API +for integration registration and module combination. + +`trusted-server-integrations-js` builds integration IIFEs. Immediate modules +are concatenated in the ordering contract above. Deferred and standalone +modules remain separate assets but retain their configuration-relative order +and typed identities. + +Browser core currently imports APS renderer logic directly. Replace that +reverse dependency with one neutral renderer registration mechanism: + +- Core parses the existing renderer envelope far enough to identify its type + and retain its payload. +- The APS browser module registers the parser and dispatcher for the existing + APS renderer type. +- Core dispatches through the registered renderer. +- Missing, duplicate, or rejecting renderers fail closed. + +An enabled APS integration whose provider can emit APS renderer descriptors +includes its immediate APS browser module. The core IIFE and fixed creative +prelude load first, so APS registration is complete before a bid can render. + +The serialized descriptor, validation, sandbox flags, message authentication, +timeouts, and render results do not change. + +## Error Handling + +Failures remain fail-closed and occur as early as the available information +allows. + +Build-time failures include invalid discovered directories, generated catalog +errors, missing typed browser modules, JavaScript compilation failures, and +missing generated bundles. + +CLI or startup failures include retired or mixed configuration shapes, +unknown definitions, invalid integration settings, unresolved qualified +provider references, duplicate routes, incompatible capabilities, invalid +auction plans, and unavailable assets. + +Runtime hook errors retain the current `Report` context and +HTTP behavior. Moving a concrete call behind a registry must not convert an +error into a warning, ignore it, or panic. + +Core visibility changes remain narrow. Helpers move with their integration +when possible. Core exposes a new public item only when a neutral cross-crate +contract requires it. + +## Compatibility Contract + +The change intentionally does not preserve operator configuration or config +blob shape. It does preserve: + +- Integration IDs. +- Existing routes and endpoint behavior. +- Existing integration hook behavior, now ordered by configuration. +- Auction plan compilation semantics after configuration normalization. +- OpenRTB request, response, routing, timeout, notification, and response + admission behavior. +- Auction ranking, mediation, renderer descriptors, and telemetry semantics. +- Existing provider identity fields, with values migrated from local IDs such + as `pbs-main` to qualified IDs such as `prebid.pbs-main`. +- Cache privacy and full-buffer decisions. +- The route and behavioral parity of Fastly, Axum, Cloudflare, and Spin. + +Bundle hashes and cache-busting URLs may change because browser sources are +rebuilt in different crates. The server must emit URLs matching the new +embedded hashes; bundle hashes are not a stable public contract. + +## Migration Sequence + +Implementation may use small commits, but the merged workspace must never have +two active integration or provider inventories. + +1. Add neutral typed capability, lifecycle, renderer, and script-module + contracts to core without changing behavior. +2. Create `trusted-server-integrations-js`, move integration browser sources + and tests, and remove the browser-core APS import. +3. Create `trusted-server-integrations`, add directory discovery, and move all + fifteen current Rust implementation units. +4. Replace closed APS and Prebid profile variants with registered OpenRTB + profile behavior and add the built-in `openrtb` integration. +5. Move all integration-specific configuration, validation, and secret metadata + into integration definitions. +6. Change operator and stored configuration to the ordered integration-owned + provider model. +7. Rewire the CLI and all adapters to the single composition entry point. +8. Remove the old concrete directories, fixed builder/profile tables, + validation lists, and `[auction.providers]` schema. +9. Update examples, fixtures, operator documentation, and migration errors. + +## Testing and Verification + +### Discovery and dependency tests + +- Every valid Rust directory appears exactly once in generated output. +- Invalid or duplicate directory IDs fail generation. +- JavaScript-only and Rust-only directories are accepted. +- A typed Rust reference to an absent browser module fails compilation. +- Embedded bundle hashes match built bytes. +- Core has no dependency on either integrations crate. +- Browser core imports no integration source. +- Adapters and CLI import no concrete integration module. + +### Configuration and ordering tests + +- TOML parent-table order becomes `IntegrationSettings` order. +- Nested provider declaration order is retained. +- TOML-to-envelope-to-runtime round trips preserve both orders byte-for-byte at + the sequence level. +- Config-store loading produces the same registry, JavaScript, and provider + order that the CLI validated. +- Disabled integrations are skipped without reordering enabled neighbors. +- Nested-only, unknown, disabled-with-provider, and mixed old/new configurations + fail with actionable messages. +- Qualified provider references resolve correctly and reject missing or + incompatible targets. +- Multiple provider instances under APS, Prebid, and standard OpenRTB compile + with stable qualified identities. + +### Capability and behavior parity tests + +- The generated catalog contains all current integration IDs plus `openrtb`. +- APS and Prebid register page/browser and auction capabilities without core + importing their types. +- `adserver_mock` registers and is selected through the mediator capability. +- Route tables and duplicate detection retain behavior. +- DataDome privacy and buffering behavior remains unchanged. +- GPT diagnostics preparation, bootstrap injection, finalization, and caching + remain unchanged on every adapter path. +- APS and Prebid request construction, transport, parsing, response admission, + and auction results remain equivalent to PR #1016 behavior. +- Bidder routing, backend naming, notification suppression, telemetry identity, + and mediator behavior remain equivalent. + +### Browser tests + +- Output order is core, creative prelude, and configured integrations. +- Immediate and deferred lists preserve configuration-relative order. +- APS is absent from browser core and registers its renderer from its own IIFE. +- Existing APS validation, sandbox, messaging, timeout, and rendering tests pass + through neutral dispatch. +- Missing or duplicate renderer registrations fail closed. + +### Repository gates + +Before handoff, run every gate required by `AGENTS.md`, including Rust format, +all target-matched clippy aliases, Fastly/Axum/Cloudflare/Spin tests, CLI and +cross-adapter parity tests, required native and WASM builds, JavaScript builds +and Vitest suites for both browser crates, JavaScript formatting, and +documentation formatting. + +## Risks and Mitigations + +### Scope expansion through generic extension points + +Moving concrete implementations can invite abstractions for hypothetical +providers. + +Mitigation: add only capability contracts exercised by current code. External +providers, non-OpenRTB factories, and additional lifecycle stages remain +separate designs. + +### Configuration migration obscures the crate boundary + +Combining packaging and configuration work increases the number of affected +files. + +Mitigation: keep the behavioral invariant explicit: configuration is +normalized into the same core auction plan and registries. Use focused commits +and parity tests around each boundary. + +### Ordering loss across serialization + +TOML order can be lost through unordered Rust maps or JSON objects. + +Mitigation: use ordered in-memory types and explicit sequences in the signed +blob. Test the complete push/store/load path rather than only the TOML parser. + +### Hidden reverse dependencies + +Concrete integrations use core-private helpers and vendor-specific enum arms. + +Mitigation: move owned helpers outward, replace vendor matches with the narrow +typed capability contract, and review every new core public item. + +### Divergent validation paths + +CLI validation and adapter startup could use different catalogs or schemas. + +Mitigation: both call the same generated composition API. No secondary +validation inventory is allowed. + +### Stale or incorrectly ordered browser artifacts + +Splitting the Node build can embed previous output or load APS too late. + +Mitigation: retain stale-output refusal, hash built bytes, load core and the +creative prelude first, and run end-to-end renderer and ordering tests. + +## Acceptance Criteria + +The change is complete when: + +1. Both new crates are workspace members and statically linked by the CLI and + every adapter where required. +2. All fifteen current concrete Rust implementation units live under + `trusted-server-integrations/src//`. +3. The standard provider configuration is supplied by the built-in Rust-only + `openrtb` integration. +4. All integration browser sources, assets, fixtures, and tests live under + `trusted-server-integrations-js`. +5. Rust and JavaScript inventories are independently directory-discovered. +6. Core owns only neutral contracts and execution engines and imports no + concrete integration. +7. One integration can register multiple typed capabilities; APS, Prebid, and + `adserver_mock` are not special construction paths. +8. `[integrations]` is the only concrete integration and auction-provider + inventory. +9. Configuration and provider ordering survive config push and runtime loading + exactly. +10. The old `[auction.providers]` schema is rejected with targeted migration + guidance. +11. The compiled auction plan retains PR #1016 behavior after normalization. +12. Browser core imports no concrete integration, and APS rendering works + through registration. +13. The CLI and all adapters use the same generated composition and validation + path. +14. The full repository verification gates pass. + +## Deferred Work + +The following require separate designs and real consumers: + +- External vendor-owned crates or adapter-supplied registrations. +- Runtime integration loading or a stable integration SDK. +- Independent integration release and compatibility policies. +- Identity, EC, geo, device, and permission-signal providers. +- Permission and jurisdiction policy changes. +- Non-OpenRTB auction provider factories. +- EdgeZero composition and host-service changes. +- Moving CLI audit detection metadata into integration directories. From e97d13bf47e31039a489a968da3919f32a433ea6 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 17 Sep 2026 12:08:02 -0700 Subject: [PATCH 03/13] Clarify integration composition boundaries --- ...-09-17-split-integrations-crates-design.md | 42 ++++++++++++++----- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index c6877ad0d..4ff8466fd 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -295,7 +295,8 @@ The rules are: 3. `trusted-server-integrations` links Rust definitions with generated browser modules from `trusted-server-integrations-js`. 4. Integration TypeScript may use the explicit browser-core API, but browser - core never imports a concrete integration. + core never imports a concrete integration and integration bundles never + embed a private copy of stateful browser-core modules. 5. Every adapter and the CLI uses the same composition and validation entry points from `trusted-server-integrations`. 6. No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or @@ -525,16 +526,20 @@ The contract is: JavaScript-only `creative` prelude second, and configured integration modules afterward. -The TOML parser must capture order directly. `IntegrationSettings` may not use -`HashMap` or another unordered representation. +Trusted Server must capture source-level table order before the EdgeZero +configuration path reduces TOML to a semantic value. This source-aware stage +also enforces the explicit-parent rule. Every entry point that accepts TOML, +including local settings loading and CLI validation or push, uses that stage. +`IntegrationSettings` may not use `HashMap` or another unordered +representation. ### Config-store representation JSON object member order is not an ordering contract. Config push therefore -converts the operator tables into an explicit ordered sequence in the signed -blob envelope. Nested provider maps are likewise encoded with explicit -sequence order. Runtime loading reconstructs ordered settings from those -sequences and never infers order from JSON object iteration. +converts the operator tables into an explicit ordered sequence in the +hash-verified blob envelope. Nested provider maps are likewise encoded with +explicit sequence order. Runtime loading reconstructs ordered settings from +those sequences and never infers order from JSON object iteration. Conceptually, the stored representation carries: @@ -568,7 +573,7 @@ explicit and covered by compatibility tests across: ```text trusted-server.toml → typed CLI configuration - → signed blob envelope + → hash-verified blob envelope → config store → runtime Settings → registry, JavaScript lists, and AuctionPlan @@ -581,9 +586,12 @@ integration owns the typed schema and validation for its full configuration, including its provider instances. The generated definition catalog supplies integration parsing, validation, -secret metadata, and capability construction to both runtime startup and the -CLI. `config validate`, `config diff`, and `config push` must use the same -catalog as the adapters. +secret metadata, pre-resolution handling for conditionally active secrets, and +capability construction to both runtime startup and the CLI. For example, +DataDome's inactive secret references are filtered by its definition before +the shared secret resolver runs; core's config-payload code does not retain a +DataDome-specific JSON path. `config validate`, `config diff`, and +`config push` must use the same catalog as the adapters. Validation fails for: @@ -638,6 +646,11 @@ registered hook; core preserves the current buffering and cache behavior. The annotation represents only the existing shared-versus-request-private decision. It is not a general policy or permissions system. +DataDome's tag-suppression and other integration-private request state stays +owned by DataDome and moves with the implementation. It may use neutral opaque +request/document state, but it is not folded into the response-sharing +annotation or exposed as a core vendor-specific field. + ### Request preparation and response finalization GPT diagnostics currently has direct preparation calls in adapters and core @@ -654,6 +667,13 @@ introduced. `trusted-server-js` builds only the neutral core IIFE and exposes a narrow API for integration registration and module combination. +The core IIFE initializes exactly one stateful registration object on the +Trusted Server browser namespace before any integration IIFE runs. Integration +bundles consume that object through an external runtime shim and type-only +browser-core declarations; their bundler must not inline the stateful registry +implementation. Artifact tests prove that a renderer registered by an +integration IIFE is visible to the already-loaded core IIFE. + `trusted-server-integrations-js` builds integration IIFEs. Immediate modules are concatenated in the ordering contract above. Deferred and standalone modules remain separate assets but retain their configuration-relative order From 9bdc5a9f631ff815da2364df1ddec6457a1d0a90 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 17 Sep 2026 12:09:21 -0700 Subject: [PATCH 04/13] Correct config blob integrity terminology --- .../specs/2026-09-17-split-integrations-crates-design.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index 4ff8466fd..8a4d77782 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -846,8 +846,9 @@ and parity tests around each boundary. TOML order can be lost through unordered Rust maps or JSON objects. -Mitigation: use ordered in-memory types and explicit sequences in the signed -blob. Test the complete push/store/load path rather than only the TOML parser. +Mitigation: use ordered in-memory types and explicit sequences in the +hash-verified blob. Test the complete push/store/load path rather than only the +TOML parser. ### Hidden reverse dependencies From 16c64d4a234e5d2c95a60685eaa459ccf7b2f9a9 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 17 Sep 2026 12:33:40 -0700 Subject: [PATCH 05/13] Resolve integrations design review findings --- ...-09-17-split-integrations-crates-design.md | 352 ++++++++++++++---- 1 file changed, 276 insertions(+), 76 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index 8a4d77782..d8f480e6c 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -39,7 +39,9 @@ creative policy, and mediator selection. Runtime order will come exclusively from TOML declaration order. The config push and config-store representation will preserve that order explicitly; -filesystem discovery order will never affect execution. +filesystem discovery order will never affect execution. For auction providers, +that order is also operational priority: it controls launch and response order, +mediator input order, and equal-price tie-breaking. This design intentionally changes the auction configuration introduced by PR #1016 while preserving that work's compiled-plan and runtime guarantees. It @@ -112,17 +114,23 @@ configured provider instance from the OpenRTB profile implementation it uses. This design preserves those runtime guarantees: - Multiple configured instances may use one integration implementation. -- Qualified provider IDs remain the stable identity shared by bidder routing, - backend correlation, diagnostics, and telemetry. +- One validated provider identity remains shared by bidder routing, backend + correlation, diagnostics, and telemetry; this design changes its serialized + value from a local ID to a qualified ID. - One validated plan remains authoritative across every adapter and runtime consumer. - The generic OpenRTB transport remains shared. - Existing routing, timeout, notification, response admission, mediation, and - telemetry attribution behavior remains unchanged. + telemetry attribution behavior remains unchanged except where provider order + is observable. This design changes the configuration location and identity spelling. Provider instances move below their owning integration, and cross-integration references -use a qualified `.` identifier. +use a qualified `.` identifier. It also intentionally +changes deterministic provider priority from lexical provider-ID order to +operator declaration order. The pricing algorithm is unchanged, but the first +configured provider retains an equal-price tie and later providers receive the +remaining shared auction budget after earlier providers launch. ### PR #1084 @@ -153,7 +161,9 @@ a stable third-party SDK or independent release boundary. lifecycle imports from core. 7. Make `[integrations]` the single ordered inventory for concrete integration configuration, including auction providers. -8. Preserve the runtime behavior and compiled-plan guarantees of PR #1016. +8. Preserve the runtime behavior and compiled-plan guarantees of PR #1016, + except that deterministic provider priority moves from lexical ID order to + configuration order. 9. Make core compile without depending on either integrations crate. 10. Keep all integrations statically linked; no runtime loading is introduced. @@ -170,9 +180,10 @@ This design does not introduce: - Identity, EC, geo, device, or permission-signal provider systems. - A jurisdiction or permission-policy redesign. - Client-cycle EC resolution or provider-code allocation. -- EdgeZero lifecycle, host-evidence, store, or adapter changes. -- New auction protocols, bidding behavior, ranking, notification behavior, or - telemetry semantics. +- Upstream EdgeZero lifecycle, host-evidence, store, or adapter changes. +- New auction protocols, pricing algorithms, notification policies, or + telemetry schemas. Configuration order intentionally replaces lexical + provider-ID order wherever deterministic provider priority is observable. - A reorganization of CLI audit detection that is unrelated to configuration validation and composition. @@ -194,6 +205,10 @@ The following terms are distinct: - **Auction provider instance:** one named endpoint and policy configuration below an integration, such as `aps.main`. Multiple instances may use the same integration implementation. +- **Local provider ID:** the provider name within one integration, such as + `main`. +- **Qualified provider ID:** the strong, globally unique pair of an integration + ID and local provider ID, serialized as `.`. - **Integration registry:** core runtime state containing the enabled page, request, response, and browser capabilities in configuration order. - **Auction plan:** core runtime state containing the validated configured @@ -302,6 +317,48 @@ The rules are: 6. No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or another integration module directly. +### Application composition ownership + +`trusted-server-integrations` is the application composition root, not only a +directory of implementations. It owns: + +- `TrustedServerAppConfig`, the typed operator-facing app-config root used by + the CLI. +- Source-aware TOML structure validation and ordered integration + deserialization. +- Aggregation of core and integration secret metadata. +- Integration-owned preprocessing for conditionally active secrets. +- Catalog-aware validation and capability construction. +- The public runtime entry points that load a config-store blob and return one + composed runtime value. + +That runtime value, conceptually `TrustedServerComposition`, contains the +validated neutral `Settings`, one `Arc`, and one +`IntegrationRegistry`. Adapters consume this value; they do not separately +compile the auction plan or rebuild the integration registry. + +Core retains neutral config-store access, Fastly chunk reconstruction, blob +envelope verification, secret-resolution primitives, global settings types, +and auction-plan compilation. Those helpers accept or return neutral data and +never call the concrete catalog. The integration crate calls them in this +order: + +```text +config-store bytes + → core chunk reconstruction and envelope verification + → integration-owned inactive-secret preprocessing + → aggregated core + integration secret resolution + → catalog-aware config validation + → core AuctionPlan compilation + → core IntegrationRegistry construction from typed registrations + → TrustedServerComposition +``` + +The CLI imports `TrustedServerAppConfig` and its config command wrappers from +`trusted-server-integrations`. Each wrapper performs the source-aware pre-pass +before delegating storage and diff mechanics to EdgeZero's typed CLI functions. +No EdgeZero source change or new host service is required. + ## Directory Discovery ### Rust @@ -372,17 +429,34 @@ APS └── OpenRTB profile capability ``` -The OpenRTB profile contract replaces the closed -`CompiledOpenRtbProfile::{Aps, PrebidServer, ...}` dependency. It supplies the -profile-owned operations required by the existing generic engine, including -typed configuration compilation, request specialization, response parsing, -diagnostics, and renderer information. Core invokes the contract without -matching on vendor variants or importing integration types. - -Compilation returns an `Arc`-backed trait object representing one immutable -compiled profile. The auction plan stores that object beside the common -provider settings, and the generic OpenRTB engine calls its typed methods. No -`Any` downcast or vendor-keyed side table is used. +The OpenRTB profile boundary has three stages: + +1. `OpenRtbProfileDefinition` is the catalog-level capability. It supplies the + stable profile ID, default timeout policy, typed configuration compiler, + endpoint canonicalization and validation, and supported routing policy. +2. `CompiledOpenRtbProfile` is an object-safe, `Send + Sync` immutable profile + stored as an `Arc` in each provider plan. It exposes only neutral routing + facts and prepares one provider exchange from neutral auction input. +3. `PreparedOpenRtbExchange` contains the finalized outbound request plus a + boxed, object-safe response parser bound to that exact provider and request. + The parser owns any request-local APS, Prebid, or standard parsing state and + consumes itself when parsing the response. + +The prepared exchange lets the core engine retain shared backend registration, +transport, deadlines, notification policy, normalized response handling, and +telemetry. The profile owns request specialization, profile-specific headers, +debug capture, response parsing, diagnostics, and renderer descriptors. + +Neutral routing policy replaces checks such as `is_prebid_server`. It expresses +only behaviors the generic router needs, including whether `all_eligible` is +allowed, whether trusted stored-request demand is recognized, and how bidder +parameters are admitted. Endpoint policy likewise replaces string comparisons +against profile IDs. + +No stage returns `Any`, requires a downcast, or indexes a vendor-keyed side +table. Because the response parser is created by the same profile object that +prepares the request, state from one provider instance cannot be supplied to +another accidentally. The standard OpenRTB implementation is registered by the built-in `openrtb` integration. APS and Prebid register their implementations from their own @@ -409,9 +483,20 @@ At startup or deploy validation: 7. Resolve typed browser modules and construct the neutral integration registry. -An absent integration is inactive. An explicitly disabled integration is -validated but contributes no runtime capabilities. Configuration cannot -activate APS through an auction plan while omitting `[integrations.aps]`. +An absent integration is inactive. Every explicit parent integration table must +contain `enabled = true` or `enabled = false`; there is no integration-specific +default. An explicitly disabled integration may retain its settings and provider +instances but contributes no runtime capabilities or providers. Configuration +cannot activate APS through an auction plan while omitting +`[integrations.aps]`, and bidder or mediator references to a disabled +integration fail validation. + +Disabled configuration still receives structural validation: unknown fields, +wrong types, duplicate IDs, and invalid values that are present fail. Missing +active-only required values and inactive secret references do not fail until +the integration is enabled. This permits operators to turn off an integration +without deleting prepared configuration while preventing disabled behavior from +leaking into the runtime plan. The four adapters and CLI share this path. Runtime and deploy validation cannot use different catalogs or integration schemas. @@ -422,6 +507,8 @@ use different catalogs or integration schemas. `[integrations]` is the single concrete integration inventory. No `type` field is added; the table key resolves the statically compiled definition. +The `enabled` field is mandatory on every parent integration table, including +the built-in `openrtb` integration. ```toml [integrations.prebid] @@ -494,6 +581,25 @@ The qualified value is the provider identity used by the compiled plan, backend correlation, diagnostics, and telemetry. The local provider name may repeat under different integrations without collision. +Provider identity uses three strong types rather than broadening the existing +local identifier: + +- `IntegrationId` matches `^[a-z][a-z0-9_]{0,62}$`. The underscore permits the + existing Rust module IDs such as `adserver_mock`; dots are forbidden. +- `LocalProviderId` retains the current + `^[a-z][a-z0-9-]{0,62}$` grammar; dots are forbidden. +- `QualifiedProviderId` stores an `IntegrationId` and `LocalProviderId`, parses + exactly one dot separator, and has a maximum serialized length of 127 ASCII + bytes. + +`QualifiedProviderId` is the type used by bidder routes, provider plans, +backend discriminators, auction responses, diagnostics, and telemetry. Its +canonical `Display` and serde representation is `.`. No +consumer reconstructs it with string concatenation, truncates it, or treats a +local provider ID as globally unique. Adapter target validation continues to +predict and reject backend-name collisions using the complete qualified +identity. + The mediator selects an enabled integration that registered a mediator capability. Its settings remain under that integration: @@ -511,27 +617,48 @@ priority controls runtime order. The contract is: -1. Integration order is the first explicit declaration order of parent +1. Integration order is the explicit declaration order of parent `[integrations.]` tables. -2. Each integration must have an explicit parent table; a nested auction table - cannot implicitly create or position it. -3. Provider order is declaration order under that integration's - `auction.providers` map. -4. Disabled integrations contribute nothing; remaining integrations keep their +2. Each parent integration table must appear before any descendant table. A + nested auction or provider table cannot implicitly create or position an + integration. +3. Provider order is the explicit declaration order of + `[integrations..auction.providers.]` tables. Each provider + parent must appear before descendant tables such as `notifications`. +4. Every parent integration table contains an explicit `enabled` value. +5. Disabled integrations contribute nothing; remaining integrations keep their relative order. -5. The flattened auction plan orders providers first by owning integration and +6. The flattened auction plan orders providers first by owning integration and then by local provider declaration. -6. Hook and immediate/deferred JavaScript lists retain integration order. -7. Browser output is neutral browser core first, the existing fixed +7. Hook and immediate/deferred JavaScript lists retain integration order. +8. Browser output is neutral browser core first, the existing fixed JavaScript-only `creative` prelude second, and configured integration modules afterward. - -Trusted Server must capture source-level table order before the EdgeZero -configuration path reduces TOML to a semantic value. This source-aware stage -also enforces the explicit-parent rule. Every entry point that accepts TOML, -including local settings loading and CLI validation or push, uses that stage. -`IntegrationSettings` may not use `HashMap` or another unordered -representation. +9. Auction provider launch, response, and mediator-input order follows the + flattened plan. With the existing strict-greater-than price comparison, the + first configured provider retains an equal-price tie. Providers later in the + sequence receive the remaining shared auction budget after earlier launches. + +Inline-table and dotted-key shorthand may not define an integration parent or +provider parent. Requiring ordinary table headers makes activation, ownership, +and order visible in one form and lets the pre-pass produce targeted errors. + +The workspace enables the `preserve_order` feature on its single resolved +`toml` package, so Cargo feature unification makes EdgeZero's `toml::Value` +maps order-preserving too. `IntegrationSettings` and provider collections use +ordered sequence-backed types, never `HashMap` or `BTreeMap`. + +Before typed deserialization, `trusted-server-integrations` parses the source +with `toml_edit`. This source-aware pre-pass rejects descendant-before-parent +declarations, missing explicit parent tables, and missing `enabled` fields. It +then permits the existing EdgeZero scalar environment overlay; overlays may +replace values but may not create, remove, or reorder integration or provider +tables. + +Every entry point that accepts TOML uses this pre-pass, including local loading +and the Trusted Server wrappers around CLI validate, diff, and push. EdgeZero's +typed mechanics remain responsible for overlay, validation invocation, diff, +envelope construction, consent, and store writes after the pre-pass succeeds. ### Config-store representation @@ -567,7 +694,14 @@ Conceptually, the stored representation carries: } ``` -The exact private Rust types may differ, but the serialized order must be +`TrustedServerAppConfig` uses custom serde at this boundary: deserialization +accepts the operator TOML table shape after the source pre-pass, while +serialization emits the explicit integration and provider sequences above for +the blob envelope. Runtime loading accepts only the new stored sequence shape; +an old blob containing an integration object map fails with migration guidance +rather than relying on JSON member order. + +The private Rust type names may differ, but the serialized order must be explicit and covered by compatibility tests across: ```text @@ -585,20 +719,26 @@ Core retains global settings and auction-orchestration validation. Each integration owns the typed schema and validation for its full configuration, including its provider instances. -The generated definition catalog supplies integration parsing, validation, -secret metadata, pre-resolution handling for conditionally active secrets, and -capability construction to both runtime startup and the CLI. For example, -DataDome's inactive secret references are filtered by its definition before -the shared secret resolver runs; core's config-payload code does not retain a -DataDome-specific JSON path. `config validate`, `config diff`, and -`config push` must use the same catalog as the adapters. +`TrustedServerAppConfig` and the generated definition catalog live in +`trusted-server-integrations`. The catalog supplies integration parsing, +validation, secret metadata, pre-resolution handling for conditionally active +secrets, and capability construction to both runtime startup and the CLI. Core +exposes its non-integration secret metadata through a neutral helper; the +composition root combines it with catalog metadata. + +For example, DataDome's inactive secret references are filtered by its +definition before the shared secret resolver runs; core's config-payload code +does not retain a DataDome-specific JSON path. Config-store loading, `config +validate`, `config diff`, and `config push` use the same catalog and composition +functions as the adapters. Validation fails for: - An unknown integration ID. +- A parent integration table with a missing or non-boolean `enabled` field. - Integration configuration not accepted by its owner. -- A nested provider table without an explicit parent integration table. -- Auction providers on an explicitly disabled integration. +- A descendant integration or provider table declared before its explicit + parent. - Duplicate local provider IDs or duplicate qualified provider identities. - A bidder route to an unknown, disabled, or incompatible provider. - A selected mediator whose integration is absent, disabled, or lacks the @@ -607,9 +747,12 @@ Validation fails for: - A referenced browser module absent from the generated browser catalog. - An unsupported capability combination. -A globally disabled auction may retain otherwise valid enabled integration and -provider configuration so operators can prepare configuration before enabling -the auction. +A disabled integration may retain structurally valid provider configuration; +those providers are not added to the plan. A globally disabled auction may +likewise retain otherwise valid enabled integration and provider configuration +so operators can prepare configuration before enabling the auction. References +from bidder routing or mediator selection to a disabled integration still fail, +even when the global auction is disabled. ## Configuration Migration @@ -721,15 +864,19 @@ contract requires it. ## Compatibility Contract The change intentionally does not preserve operator configuration or config -blob shape. It does preserve: +blob shape. It also intentionally changes provider priority from lexical local +provider-ID order to qualified configuration order. It preserves: - Integration IDs. - Existing routes and endpoint behavior. - Existing integration hook behavior, now ordered by configuration. -- Auction plan compilation semantics after configuration normalization. +- Auction plan compilation semantics after configuration normalization, except + for the documented provider-priority source. - OpenRTB request, response, routing, timeout, notification, and response admission behavior. -- Auction ranking, mediation, renderer descriptors, and telemetry semantics. +- Auction price comparison, mediation protocol, renderer descriptors, and + telemetry schema. Provider response order and an equal-price winner may + change when configuration order differs from the old lexical order. - Existing provider identity fields, with values migrated from local IDs such as `pbs-main` to qualified IDs such as `prebid.pbs-main`. - Cache privacy and full-buffer decisions. @@ -739,6 +886,23 @@ Bundle hashes and cache-busting URLs may change because browser sources are rebuilt in different crates. The server must emit URLs matching the new embedded hashes; bundle hashes are not a stable public contract. +## Delivery Scope + +This is one architecture design but not one undifferentiated refactor. It has +four reviewable workstreams: + +1. Neutral Rust capability and lifecycle contracts. +2. Browser-core separation and `trusted-server-integrations-js`. +3. Concrete Rust extraction and application composition ownership. +4. Ordered configuration, provider identity, and auction-profile migration. + +The implementation plan must give each workstream its own verification +checkpoint and keep behavior-preserving moves separate from intentional config +and ordering changes. Intermediate commits may add unused neutral contracts or +new crates, but no merged state may have two active catalogs, two provider +inventories, or adapter-specific composition paths. This scope does not include +the external plugin ecosystem proposed by PR #1084. + ## Migration Sequence Implementation may use small commits, but the merged workspace must never have @@ -751,12 +915,16 @@ two active integration or provider inventories. 3. Create `trusted-server-integrations`, add directory discovery, and move all fifteen current Rust implementation units. 4. Replace closed APS and Prebid profile variants with registered OpenRTB - profile behavior and add the built-in `openrtb` integration. -5. Move all integration-specific configuration, validation, and secret metadata - into integration definitions. -6. Change operator and stored configuration to the ordered integration-owned - provider model. -7. Rewire the CLI and all adapters to the single composition entry point. + profile and prepared-exchange behavior, and add the built-in `openrtb` + integration. +5. Move `TrustedServerAppConfig`, all integration-specific configuration, + validation, inactive-secret preprocessing, and secret metadata into the + integrations crate. +6. Add the TOML source pre-pass, order-preserving maps, explicit stored + sequences, strong qualified provider IDs, and the breaking + integration-owned provider schema. +7. Rewire the CLI and all adapters to the single composition entry point that + returns settings, plan, and registry together. 8. Remove the old concrete directories, fixed builder/profile tables, validation lists, and `[auction.providers]` schema. 9. Update examples, fixtures, operator documentation, and migration errors. @@ -778,17 +946,26 @@ two active integration or provider inventories. - TOML parent-table order becomes `IntegrationSettings` order. - Nested provider declaration order is retained. +- A parent integration or provider table declared after one of its descendants + fails before typed deserialization. +- Missing `enabled` fails; omitted integration tables remain inactive. - TOML-to-envelope-to-runtime round trips preserve both orders byte-for-byte at the sequence level. - Config-store loading produces the same registry, JavaScript, and provider order that the CLI validated. -- Disabled integrations are skipped without reordering enabled neighbors. -- Nested-only, unknown, disabled-with-provider, and mixed old/new configurations - fail with actionable messages. +- Disabled integrations may retain valid provider settings, contribute no + providers or capabilities, and do not reorder enabled neighbors. +- Bidder and mediator references to disabled integrations fail, including while + the global auction is disabled. +- Nested-only, unknown, missing-enabled, descendant-before-parent, and mixed + old/new configurations fail with actionable messages. - Qualified provider references resolve correctly and reject missing or incompatible targets. +- Local and qualified provider IDs enforce their separate grammars and bounds. - Multiple provider instances under APS, Prebid, and standard OpenRTB compile with stable qualified identities. +- Provider launch, response, mediator-input, and equal-price tie order follows + integration then local-provider declaration order. ### Capability and behavior parity tests @@ -801,9 +978,12 @@ two active integration or provider inventories. - GPT diagnostics preparation, bootstrap injection, finalization, and caching remain unchanged on every adapter path. - APS and Prebid request construction, transport, parsing, response admission, - and auction results remain equivalent to PR #1016 behavior. + and auction results remain equivalent to PR #1016 behavior except for the + documented provider-priority change. +- Prepared response parsers consume profile-owned request state without `Any`, + downcasts, vendor enums, or cross-provider state reuse. - Bidder routing, backend naming, notification suppression, telemetry identity, - and mediator behavior remain equivalent. + and mediator behavior remain equivalent apart from documented ordering. ### Browser tests @@ -847,22 +1027,33 @@ and parity tests around each boundary. TOML order can be lost through unordered Rust maps or JSON objects. Mitigation: use ordered in-memory types and explicit sequences in the -hash-verified blob. Test the complete push/store/load path rather than only the -TOML parser. +hash-verified blob, enable `toml/preserve_order`, and reject +descendant-before-parent source declarations with the `toml_edit` pre-pass. +Test the complete push/store/load path rather than only the TOML parser. + +### Configuration order silently changes auction priority + +Provider order affects launch budget, mediator input, response order, and equal +price ties. Treating it as cosmetic would make operator edits surprising. + +Mitigation: define configuration order as operational priority, document the +change from PR #1016's lexical order, and test each observable consequence. ### Hidden reverse dependencies Concrete integrations use core-private helpers and vendor-specific enum arms. Mitigation: move owned helpers outward, replace vendor matches with the narrow -typed capability contract, and review every new core public item. +typed capability and prepared-exchange contracts, and review every new core +public item. ### Divergent validation paths CLI validation and adapter startup could use different catalogs or schemas. Mitigation: both call the same generated composition API. No secondary -validation inventory is allowed. +validation inventory is allowed. Adapters receive the already composed +settings, plan, and registry rather than reconstructing any of them. ### Stale or incorrectly ordered browser artifacts @@ -891,15 +1082,24 @@ The change is complete when: 8. `[integrations]` is the only concrete integration and auction-provider inventory. 9. Configuration and provider ordering survive config push and runtime loading - exactly. + exactly and define the documented auction priority. 10. The old `[auction.providers]` schema is rejected with targeted migration guidance. -11. The compiled auction plan retains PR #1016 behavior after normalization. +11. The compiled auction plan retains PR #1016 behavior after normalization, + except for the explicit change from lexical to configuration-order provider + priority. 12. Browser core imports no concrete integration, and APS rendering works through registration. -13. The CLI and all adapters use the same generated composition and validation - path. -14. The full repository verification gates pass. +13. `TrustedServerAppConfig`, integration secret handling, and final runtime + composition are owned by `trusted-server-integrations`; core has no concrete + config or loader dependency. +14. The CLI and all adapters use the same generated composition and validation + path and receive one settings/plan/registry composition. +15. OpenRTB request-local state crosses the transport boundary through a + prepared response parser without `Any` or vendor enum variants in core. +16. Explicit `enabled`, parent-before-descendant, disabled-retention, local-ID, + and qualified-ID rules have end-to-end tests. +17. The full repository verification gates pass. ## Deferred Work @@ -911,5 +1111,5 @@ The following require separate designs and real consumers: - Identity, EC, geo, device, and permission-signal providers. - Permission and jurisdiction policy changes. - Non-OpenRTB auction provider factories. -- EdgeZero composition and host-service changes. +- Upstream EdgeZero composition and host-service changes. - Moving CLI audit detection metadata into integration directories. From 6b450ca58d111b1053e8879b9d218b04b436618c Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 17 Sep 2026 12:36:58 -0700 Subject: [PATCH 06/13] Clarify integration activation boundaries --- ...-09-17-split-integrations-crates-design.md | 32 +++++++++++++------ 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index d8f480e6c..c6a089994 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -41,7 +41,7 @@ Runtime order will come exclusively from TOML declaration order. The config push and config-store representation will preserve that order explicitly; filesystem discovery order will never affect execution. For auction providers, that order is also operational priority: it controls launch and response order, -mediator input order, and equal-price tie-breaking. +mediator input order, and local equal-price tie-breaking. This design intentionally changes the auction configuration introduced by PR #1016 while preserving that work's compiled-plan and runtime guarantees. It @@ -289,6 +289,11 @@ concrete integration. JavaScript-only `creative` remains valid without a Rust directory. Rust-only integrations remain valid without a browser directory. +`creative` is the sole fixed, non-configurable browser prelude in this design; +it is runtime support rather than an operator integration. Directory discovery +may build other JavaScript-only assets, but `[integrations]` cannot activate one +unless a Rust definition with that ID registers its browser-module capability. + ## Dependency Direction The Cargo dependency graph is one-way: @@ -365,8 +370,8 @@ No EdgeZero source change or new host service is required. `trusted-server-integrations/build.rs` scans immediate directories under `src/`. A directory containing `mod.rs` is a concrete integration whose stable -ID is the directory name. IDs must use the existing integration ID grammar and -must be unique. +ID is the directory name. IDs must parse as the `IntegrationId` defined by this +design and must be unique. The build script generates module declarations and a definition catalog. Its lexical sorting makes generated source reproducible but has no runtime ordering @@ -429,6 +434,13 @@ APS └── OpenRTB profile capability ``` +Capability multiplicity is explicit. Collection capabilities such as routes +and rewriters may register multiple entries. Single-valued capabilities such as +an OpenRTB profile or mediator may appear at most once per integration +definition; duplicate registration fails composition. Because provider tables +have no profile discriminator, an integration that owns +`auction.providers` must register exactly one OpenRTB profile capability. + The OpenRTB profile boundary has three stages: 1. `OpenRtbProfileDefinition` is the catalog-level capability. It supplies the @@ -636,8 +648,9 @@ The contract is: afterward. 9. Auction provider launch, response, and mediator-input order follows the flattened plan. With the existing strict-greater-than price comparison, the - first configured provider retains an equal-price tie. Providers later in the - sequence receive the remaining shared auction budget after earlier launches. + first configured provider retains an equal-price tie during local winner + selection. Providers later in the sequence receive the remaining shared + auction budget after earlier launches. Inline-table and dotted-key shorthand may not define an integration parent or provider parent. Requiring ordinary table headers makes activation, ownership, @@ -875,8 +888,9 @@ provider-ID order to qualified configuration order. It preserves: - OpenRTB request, response, routing, timeout, notification, and response admission behavior. - Auction price comparison, mediation protocol, renderer descriptors, and - telemetry schema. Provider response order and an equal-price winner may - change when configuration order differs from the old lexical order. + telemetry schema. Provider response order and a locally selected equal-price + winner may change when configuration order differs from the old lexical + order. - Existing provider identity fields, with values migrated from local IDs such as `pbs-main` to qualified IDs such as `prebid.pbs-main`. - Cache privacy and full-buffer decisions. @@ -964,8 +978,8 @@ two active integration or provider inventories. - Local and qualified provider IDs enforce their separate grammars and bounds. - Multiple provider instances under APS, Prebid, and standard OpenRTB compile with stable qualified identities. -- Provider launch, response, mediator-input, and equal-price tie order follows - integration then local-provider declaration order. +- Provider launch, response, mediator-input, and local equal-price tie order + follows integration then local-provider declaration order. ### Capability and behavior parity tests From 374335cfbd7a5dff3f3bd7cb2e1f14e4de671a44 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 17 Sep 2026 23:57:33 -0700 Subject: [PATCH 07/13] Tighten integrations split design contracts --- ...-09-17-split-integrations-crates-design.md | 708 ++++++++++++++---- 1 file changed, 544 insertions(+), 164 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index c6a089994..ab00ea334 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -37,17 +37,23 @@ auction provider instances. Global `[auction]` settings will continue to own cross-integration orchestration such as the auction timeout, bidder routing, creative policy, and mediator selection. -Runtime order will come exclusively from TOML declaration order. The config -push and config-store representation will preserve that order explicitly; -filesystem discovery order will never affect execution. For auction providers, -that order is also operational priority: it controls launch and response order, -mediator input order, and local equal-price tie-breaking. +Cross-integration runtime order will come exclusively from TOML declaration +order. The config push and config-store representation will preserve that order +explicitly; filesystem discovery order will never affect execution. For +auction providers, that order is also operational priority: it controls launch +and response order, mediator input order, and local equal-price tie-breaking. This design intentionally changes the auction configuration introduced by PR #1016 while preserving that work's compiled-plan and runtime guarantees. It adopts only the typed-registration portion of PR #1084 and excludes that PR's external provider ecosystem and unrelated provider systems. +This remains one design, but it has two merge milestones. The crate and runtime +boundary moves first without changing operator configuration. The ordered, +integration-owned configuration cuts over only after the behavior-preserving +boundary is running. The milestones share one target architecture without +forcing the packaging move and configuration migration into one deployment. + ## Context On `main` at `6cae7f5da`, neutral registry machinery and concrete integrations @@ -145,6 +151,16 @@ multiple typed capabilities. It does not create the external ecosystem. The new contracts serve the integrations compiled in this workspace; they are not a stable third-party SDK or independent release boundary. +The current PR #1084 configuration convention is mutually exclusive with this +design. PR #1084 selects implementations through top-level `[integration]`, +`[demand]`, and `[adserver]` provider selectors, removes `enabled` from +integration blocks, and does not treat APS as an integration. This design +deliberately chooses one ordered `[integrations]` inventory, explicit +`enabled`, and APS as a multi-capability integration. The two configurations +must not merge as parallel conventions. If this design is accepted, the +conflicting configuration and auction sections of PR #1084 must be superseded +or revised before that broader provider work proceeds. + ## Goals 1. Move every concrete Rust integration implementation out of core and into one @@ -153,8 +169,8 @@ a stable third-party SDK or independent release boundary. `trusted-server-integrations-js` crate. 3. Give each concrete integration one Rust directory and, when applicable, one same-named browser directory. -4. Discover Rust and browser integration inventories from directories at build - time. +4. Keep one explicit compile-checked Rust catalog and discover browser modules + from integration directories at build time. 5. Let one integration register multiple typed capabilities without a global integration-kind enum. 6. Remove concrete integration construction, auction-profile, validation, and @@ -195,7 +211,7 @@ separate design with a real consumer. The following terms are distinct: -- **Integration definition:** the statically discovered code definition for a +- **Integration definition:** the statically cataloged code definition for a stable integration ID such as `aps`. - **Capability registration:** one typed contribution made by an integration, such as a proxy, HTML rewriter, JavaScript module, OpenRTB profile, or @@ -229,11 +245,14 @@ crates/ trusted-server-js/ lib/ + package.json + package-lock.json + build-browser.mjs src/core/ + test/core/ src/ trusted-server-integrations/ - build.rs Cargo.toml src/ lib.rs @@ -263,7 +282,6 @@ crates/ build.rs Cargo.toml lib/ - package.json src/integrations/ aps/ index.ts @@ -282,9 +300,11 @@ crates/ Every current flat Rust integration file becomes `/mod.rs`. Existing nested modules and fixtures stay with their owner. `openrtb` is a built-in -Rust-only integration that exposes configuration for the current standard -OpenRTB profile without turning the neutral OpenRTB execution engine into a -concrete integration. +Rust-only, directory-backed integration adapter that exposes configuration for +the current standard OpenRTB profile without turning the neutral OpenRTB +execution engine into concrete code. The target Rust catalog therefore has +sixteen definitions: fifteen moved implementations plus the new `openrtb` +adapter. JavaScript-only `creative` remains valid without a Rust directory. Rust-only integrations remain valid without a browser directory. @@ -317,8 +337,9 @@ The rules are: 4. Integration TypeScript may use the explicit browser-core API, but browser core never imports a concrete integration and integration bundles never embed a private copy of stateful browser-core modules. -5. Every adapter and the CLI uses the same composition and validation entry - points from `trusted-server-integrations`. +5. Every adapter and the CLI uses the same source-validation entry points from + `trusted-server-integrations`; every adapter uses its single runtime + composition entry point. 6. No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or another integration module directly. @@ -334,13 +355,32 @@ directory of implementations. It owns: - Aggregation of core and integration secret metadata. - Integration-owned preprocessing for conditionally active secrets. - Catalog-aware validation and capability construction. +- Public source-validation wrappers used by the CLI before EdgeZero's typed + validate, diff, and push mechanics. - The public runtime entry points that load a config-store blob and return one composed runtime value. +`TrustedServerAppConfig` contains neutral core configuration plus the ordered +integration-owned source configuration. Concrete integration configuration is +not added to core's `Settings`. Composition consumes the integration portion +into capabilities and returns a neutral runtime `Settings` value containing +only state that core execution engines understand. + +The public API has two explicit phases. The source phase performs the pre-pass, +catalog resolution, integration-owned structural and deploy validation, secret +metadata aggregation, and serialization into the storage DTO. The CLI stops at +that phase and never constructs runtime capability objects from unresolved +secret key names. The runtime phase begins only after envelope verification, +inactive-secret preprocessing, and secret resolution; it validates the resolved +values and constructs the plan, registry, and browser assets. Both phases use +the same static catalog and integration schemas, but only adapters receive the +final `TrustedServerComposition`. + That runtime value, conceptually `TrustedServerComposition`, contains the validated neutral `Settings`, one `Arc`, and one -`IntegrationRegistry`. Adapters consume this value; they do not separately -compile the auction plan or rebuild the integration registry. +`IntegrationRegistry`, plus the composed `BrowserDocumentAssets`. Adapters +consume this value; they do not separately compile the auction plan, rebuild +the integration registry, or enumerate browser bundles. Core retains neutral config-store access, Fastly chunk reconstruction, blob envelope verification, secret-resolution primitives, global settings types, @@ -356,50 +396,62 @@ config-store bytes → catalog-aware config validation → core AuctionPlan compilation → core IntegrationRegistry construction from typed registrations + → browser asset composition and document fingerprint → TrustedServerComposition ``` The CLI imports `TrustedServerAppConfig` and its config command wrappers from `trusted-server-integrations`. Each wrapper performs the source-aware pre-pass -before delegating storage and diff mechanics to EdgeZero's typed CLI functions. -No EdgeZero source change or new host service is required. +and source-phase catalog validation before delegating storage and diff mechanics +to EdgeZero's typed CLI functions. No EdgeZero source change or new host service +is required. -## Directory Discovery +## Static Rust Catalog and Browser Discovery ### Rust -`trusted-server-integrations/build.rs` scans immediate directories under -`src/`. A directory containing `mod.rs` is a concrete integration whose stable -ID is the directory name. IDs must parse as the `IntegrationId` defined by this -design and must be unique. - -The build script generates module declarations and a definition catalog. Its -lexical sorting makes generated source reproducible but has no runtime ordering -meaning. Each module must expose the expected crate-private `definition` -function; failure to do so is a compile error. +`trusted-server-integrations/src/lib.rs` declares a small, explicit static +catalog. Each entry names an `IntegrationId` and a crate-private `definition` +function from a same-named directory. Rust compilation checks every listed +module and definition signature. A host-target completeness test enumerates +immediate `src//mod.rs` directories and fails if a valid directory is +missing from the catalog or a catalog ID has no directory. IDs must parse as +the `IntegrationId` defined by this design and must be unique. There is no directory-local numeric order. Runtime order belongs to configuration. -The build fails for malformed IDs, duplicate normalized IDs, unreadable -directories, or an empty catalog. It emits `cargo:rerun-if-changed` directives -for the discovered directories. +This intentionally avoids a Rust build script and generated module +declarations for a sixteen-entry table. The catalog-completeness test provides +the missing-registration guard without making filesystem discovery part of +compilation. ### JavaScript -`trusted-server-integrations-js` owns its Node project, integration tests, -build pipeline, generated Rust module catalog, and browser assets. Its build -discovers immediate directories containing `index.ts` and emits one -self-contained IIFE per entry point. Its Cargo build embeds each output and its -SHA-256 hash. +`trusted-server-js` and `trusted-server-integrations-js` use one Node toolchain +project, one lockfile, and one set of Vitest, ESLint, Prettier, Vite, and Prebid +aliases. The canonical project root remains +`crates/trusted-server-js/lib`; `trusted-server-integrations-js` does not add a +second `package.json` or lockfile. Neutral browser sources remain under +`trusted-server-js`; integration sources, fixtures, and tests live under +`trusted-server-integrations-js`. Separate build targets emit neutral and +integration artifacts into their owning Rust crates. The build helpers +coordinate dependency installation and output generation so parallel Cargo +build scripts cannot race or consume stale artifacts. + +The integration build discovers immediate directories containing `index.ts` +and emits one self-contained IIFE per entry point. Its Cargo build embeds each +output and its SHA-256 hash. CI, Dependabot, browser integration scripts, and +the CLI Prebid builder use the single workspace root rather than maintaining a +second dependency graph. The generated Rust API exposes typed module identifiers rather than accepting unchecked strings. A Rust registration referencing a missing browser module therefore fails compilation. JavaScript-only modules are valid and need no Rust definition. -Rust and JavaScript discovery are independent; neither filesystem inventory is -treated as the canonical list for the other. +The explicit Rust catalog and generated browser catalog are independent; +neither inventory is treated as the canonical list for the other. ## Neutral Core Contracts @@ -411,8 +463,9 @@ The neutral contents of `integrations/registry.rs` move to singular - Proxy, request-filter, attribute-rewriter, script-rewriter, HTML-post-processor, and head-injector traits and contexts. - Neutral request-preparation and response-finalization hooks. -- Neutral response-sharing/private-cache annotations. -- Neutral JavaScript module metadata and load modes. +- Neutral request-processing and response-sharing annotations. +- Neutral browser-asset metadata, byte/hash access, load modes, and composed + document fingerprints. - OpenRTB profile registration contracts consumed by the generic plan and transport engines. - Mediator registration contracts consumed by auction orchestration. @@ -474,22 +527,40 @@ The standard OpenRTB implementation is registered by the built-in `openrtb` integration. APS and Prebid register their implementations from their own directories. `adserver_mock` registers the existing mediator capability. +The mediator does not reuse or expose the legacy `AuctionProvider` trait. Its +neutral boundary has three parts: + +1. `MediatorDefinition` compiles one integration's mediator configuration. +2. `CompiledMediator` receives the original auction request, the already + ordered provider responses, and the remaining bounded runtime context. It + returns a `PreparedMediatorExchange`. +3. `PreparedMediatorExchange` contains the finalized outbound request and a + bound response parser that consumes the upstream response into the existing + normalized mediation result. + +Core retains mediator transport, deadline enforcement, telemetry, and the +existing warning-and-local-ranking fallback policy. `adserver_mock` owns only +request construction and response interpretation. The old mediator use of +`Arc` is deleted when its last caller migrates. + No unused generic `AuctionProviderFactory` extension is added. A provider that cannot use the current OpenRTB engine will define that additional seam in a future design. ## Static Catalog and Activation -Directory discovery establishes what the binary supports. Configuration -establishes what runs. +The explicit static catalog establishes what the binary supports. +Configuration establishes what runs. -At startup or deploy validation: +Source validation and runtime startup both resolve IDs through the same catalog. +Source validation stops after validating the unresolved, storage-safe model. At +runtime, after secrets are resolved: 1. Parse `[integrations]` into an ordered sequence. -2. Resolve each ID against the generated definition catalog. +2. Resolve each ID against the static definition catalog. 3. Ask the owning definition to parse and validate its complete configuration. 4. For each enabled integration, construct its typed capability registrations - in TOML order. + in the validated integration order restored from the storage sidecar. 5. Collect integration-owned auction provider instances and profiles. 6. Ask core to compile the single canonical auction plan. 7. Resolve typed browser modules and construct the neutral integration @@ -503,6 +574,12 @@ cannot activate APS through an auction plan while omitting `[integrations.aps]`, and bidder or mediator references to a disabled integration fail validation. +The APS rule is an intentional activation break from PR #1016. Today an APS +auction profile can activate server-side rendering support without an enabled +`[integrations.aps]` browser block. After cutover, every APS provider requires +an explicit `[integrations.aps]` parent with `enabled = true`; that one parent +activates APS's server, page, and browser capabilities together. + Disabled configuration still receives structural validation: unknown fields, wrong types, duplicate IDs, and invalid values that are present fail. Missing active-only required values and inactive secret references do not fail until @@ -510,8 +587,9 @@ the integration is enabled. This permits operators to turn off an integration without deleting prepared configuration while preventing disabled behavior from leaking into the runtime plan. -The four adapters and CLI share this path. Runtime and deploy validation cannot -use different catalogs or integration schemas. +The four adapters share the runtime composition path. The CLI and adapters +share the source/catalog validation rules; runtime and deploy validation cannot +use different catalogs, integration schemas, or plan-input validation. ## One Integration Configuration @@ -597,9 +675,11 @@ Provider identity uses three strong types rather than broadening the existing local identifier: - `IntegrationId` matches `^[a-z][a-z0-9_]{0,62}$`. The underscore permits the - existing Rust module IDs such as `adserver_mock`; dots are forbidden. + existing Rust module IDs such as `adserver_mock`; dots are forbidden. Its + maximum length is 63 ASCII bytes. - `LocalProviderId` retains the current - `^[a-z][a-z0-9-]{0,62}$` grammar; dots are forbidden. + `^[a-z][a-z0-9-]{0,62}$` grammar; dots are forbidden. Its maximum length is + 63 ASCII bytes. - `QualifiedProviderId` stores an `IntegrationId` and `LocalProviderId`, parses exactly one dot separator, and has a maximum serialized length of 127 ASCII bytes. @@ -610,7 +690,11 @@ canonical `Display` and serde representation is `.`. No consumer reconstructs it with string concatenation, truncates it, or treats a local provider ID as globally unique. Adapter target validation continues to predict and reject backend-name collisions using the complete qualified -identity. +identity. A platform backend name is not itself the provider identity. Where an +adapter's normalization is lossy, as with Axum mapping dots, hyphens, and +underscores to the same character, its correlation name includes a stable +digest of the full qualified ID. Target validation rejects any remaining final +name collision. Tests cover aliases such as `a_b.c` and `a.b-c`. The mediator selects an enabled integration that registered a mediator capability. Its settings remain under that integration: @@ -642,15 +726,32 @@ The contract is: relative order. 6. The flattened auction plan orders providers first by owning integration and then by local provider declaration. -7. Hook and immediate/deferred JavaScript lists retain integration order. +7. Hook and immediate/deferred JavaScript lists retain integration order. When + one integration registers multiple hooks of the same capability, their + relative order is the explicit order returned by that integration's + definition; the integration remains one operator-visible position and does + not expose a second priority mechanism. 8. Browser output is neutral browser core first, the existing fixed JavaScript-only `creative` prelude second, and configured integration modules afterward. 9. Auction provider launch, response, and mediator-input order follows the flattened plan. With the existing strict-greater-than price comparison, the first configured provider retains an equal-price tie during local winner - selection. Providers later in the sequence receive the remaining shared - auction budget after earlier launches. + selection. Dispatch checks the remaining shared deadline immediately before + each back-to-back launch; configuration order becomes budget-observable only + if the deadline expires or adapter timeout canonicalization reaches zero + during that launch loop. + +The current hard-coded requirement that `js_asset_proxy` remain the first +rewriter is retired. Rewriter chaining follows the same operator-visible +integration order as every other hook. Migration guidance places +`js_asset_proxy` first in migrated examples and procedures so the old behavior +is preserved by default, while an operator may deliberately choose a different +order. No engine-only priority is hidden from the configuration. + +All recovery paths obey the same provider ordinals. In particular, the two +current transport-failure branches that sort provider IDs lexically are +replaced with plan-order recovery before the new ordering contract is enabled. Inline-table and dotted-key shorthand may not define an integration parent or provider parent. Requiring ordinary table headers makes activation, ownership, @@ -658,8 +759,9 @@ and order visible in one form and lets the pre-pass produce targeted errors. The workspace enables the `preserve_order` feature on its single resolved `toml` package, so Cargo feature unification makes EdgeZero's `toml::Value` -maps order-preserving too. `IntegrationSettings` and provider collections use -ordered sequence-backed types, never `HashMap` or `BTreeMap`. +maps order-preserving too. The integration-owned source model's integration and +provider collections use ordered map types with explicit iteration semantics, +never `HashMap` or `BTreeMap`. Before typed deserialization, `trusted-server-integrations` parses the source with `toml_edit`. This source-aware pre-pass rejects descendant-before-parent @@ -673,46 +775,75 @@ and the Trusted Server wrappers around CLI validate, diff, and push. EdgeZero's typed mechanics remain responsible for overlay, validation invocation, diff, envelope construction, consent, and store writes after the pre-pass succeeds. +EdgeZero does not currently expose a pre-parse hook. The Trusted Server wrappers +therefore deliberately duplicate its app-config path rule: an explicit +`--app-config` wins; otherwise the path is `/.toml`. +The wrapper reads that source for structural validation and EdgeZero reads it +again for typed processing. Parity tests cover explicit and default paths, +manifest paths with and without parent directories, and `--no-env`. The +environment overlay can replace only scalar leaves already present in TOML; it +cannot create an omitted `enabled` field, integration, provider, table, or +array. Operator templates must contain every leaf intended for overlay. + ### Config-store representation JSON object member order is not an ordering contract. Config push therefore -converts the operator tables into an explicit ordered sequence in the -hash-verified blob envelope. Nested provider maps are likewise encoded with -explicit sequence order. Runtime loading reconstructs ordered settings from -those sequences and never infers order from JSON object iteration. +keeps integration and provider configuration in object maps but adds explicit +order sidecars to the hash-verified blob envelope. This preserves the existing +object paths used by secret metadata while making runtime order independent of +JSON map iteration. `serde_json/preserve_order` is not required and does not +become a workspace-wide behavior change. Conceptually, the stored representation carries: ```json { - "integrations": [ - { - "id": "prebid", - "config": { - "enabled": true, - "auction": { - "providers": [ - { - "id": "pbs-main", - "config": { - "endpoint": "https://prebid.example.com/openrtb2/auction" - } - } - ] + "trusted_server_schema": 2, + "integration_order": ["prebid", "aps"], + "integrations": { + "prebid": { + "enabled": true, + "auction": { + "provider_order": ["pbs-main"], + "providers": { + "pbs-main": { + "endpoint": "https://prebid.example.com/openrtb2/auction" + } } } }, - { "id": "aps", "config": { "enabled": true } } - ] + "aps": { "enabled": true } + } } ``` -`TrustedServerAppConfig` uses custom serde at this boundary: deserialization -accepts the operator TOML table shape after the source pre-pass, while -serialization emits the explicit integration and provider sequences above for -the blob envelope. Runtime loading accepts only the new stored sequence shape; -an old blob containing an integration object map fails with migration guidance -rather than relying on JSON member order. +`TrustedServerAppConfig` uses custom serialization at this boundary. +Deserialization accepts the operator TOML table shape after the source pre-pass; +serialization emits object-shaped configuration plus the schema and order +sidecars above. Serialization fails if a sidecar omits, duplicates, or names a +different key than its corresponding object map. Runtime loading validates the +same bijection before constructing ordered settings. + +This is an explicit asymmetric serde boundary. EdgeZero's typed CLI +deserializes and validates `TrustedServerAppConfig`, then its manual `Serialize` +implementation emits the schema-2 storage DTO. Runtime reads that DTO rather +than deserializing it back into the operator type. `integration_order` is always +present. `provider_order` is required whenever the corresponding `providers` +object is present, including when both are empty; both are absent when an +integration has no provider collection. + +`trusted_server_schema`, `integration_order`, and `provider_order` are reserved +storage fields. They are generated by serialization and rejected if supplied in +operator TOML. + +Object-shaped storage keeps secret paths such as +`integrations.datadome.server_side_key_secret_name` valid. Core and integration +secret metadata traverse that resolution view before ordered settings are +constructed. Integration-owned inactive-secret preprocessing also receives the +object-shaped integration entry by ID and writes any resolved value back to the +same entry; it never searches an `{ id, config }` sequence. End-to-end tests +prove both active DataDome secret fields are presence-checked, resolved, and +removed when inactive. The private Rust type names may differ, but the serialized order must be explicit and covered by compatibility tests across: @@ -732,7 +863,7 @@ Core retains global settings and auction-orchestration validation. Each integration owns the typed schema and validation for its full configuration, including its provider instances. -`TrustedServerAppConfig` and the generated definition catalog live in +`TrustedServerAppConfig` and the static definition catalog live in `trusted-server-integrations`. The catalog supplies integration parsing, validation, secret metadata, pre-resolution handling for conditionally active secrets, and capability construction to both runtime startup and the CLI. Core @@ -742,8 +873,18 @@ composition root combines it with catalog metadata. For example, DataDome's inactive secret references are filtered by its definition before the shared secret resolver runs; core's config-payload code does not retain a DataDome-specific JSON path. Config-store loading, `config -validate`, `config diff`, and `config push` use the same catalog and composition -functions as the adapters. +validate`, `config diff`, and `config push` use the same catalog resolution, +schemas, and pure validation functions as the adapters. Only config-store +loading proceeds through resolved-value validation and runtime capability +composition. + +The DataDome move accounts for every current production coupling rather than +only its response marker: direct HTML-processor state, publisher +template/body/privacy decisions, startup and deploy validation, legacy settings +cleanup, secret metadata and resolved-value validation, and inactive-secret +preprocessing. Integration-owned code moves outward; the neutral processing +requirements and object-shaped secret-resolution view replace the two places +where core genuinely coordinates behavior. Validation fails for: @@ -769,8 +910,11 @@ even when the global auction is disabled. ## Configuration Migration -This is a deliberate breaking migration. The runtime and CLI do not support -both inventories or define precedence between them. +This is a deliberate operator-source migration. The new CLI accepts only the +new inventory and never defines precedence between old and new source fields. +The runtime temporarily supports two stored application schema versions only +to make deployment safe; it normalizes exactly one complete stored shape and +never merges inventories. Representative mappings are: @@ -778,34 +922,81 @@ Representative mappings are: | --------------------------------------------------------------- | --------------------------------------------------------- | | `[auction.providers.pbs-main]` with `profile = "prebid-server"` | `[integrations.prebid.auction.providers.pbs-main]` | | `[auction.providers.aps-main]` with `profile = "aps"` | `[integrations.aps.auction.providers.aps-main]` | +| APS provider with no `[integrations.aps]` parent | Add `[integrations.aps]` with `enabled = true` | | A standard profile provider named `example-direct` | `[integrations.openrtb.auction.providers.example-direct]` | | `[auction.providers..profile_config]` | Flattened into the owning integration's provider table | | Bidder route `provider = "pbs-main"` | `provider = "prebid.pbs-main"` | Old `[auction.providers]`, `profile`, and `profile_config` fields fail with an actionable message naming the new integration-owned location. A mixed old/new -configuration also fails. There is no silent translation at runtime and no -deprecation interval. +operator configuration also fails. There is no deprecation interval for +operator TOML. The temporary old-blob reader is not an accepted source format +and does not make old fields valid in the new CLI. Examples, integration fixtures, environment-overlay tests, CLI documentation, and operator guides migrate in the same change. +### Stored schema and rollout + +The EdgeZero `BlobEnvelope` version describes EdgeZero's envelope and canonical +hash rules; it is not a Trusted Server application-schema version. Both old and +new data therefore remain in envelope version 1. New data carries +`trusted_server_schema = 2` inside `data`; absence of that field identifies the +existing stored shape during the transition. Unknown application schema values +fail before secret resolution. + +One release of the new binary contains a read-only compatibility decoder for +the existing stored object shape and `[auction.providers]`. It converts that +complete legacy value into the same neutral ordered composition used by schema +2, retaining the current fixed integration-builder and special-registration +order, the old lexical provider order, and the existing implicit APS activation +behavior for that legacy blob only. It emits an operator warning to push the +migrated configuration. New CLI writes schema 2 only. + +Rollout is ordered: + +1. Archive the current envelope bytes and record the adapter, store, and key. + EdgeZero has no config-pull command, so this uses the adapter's native + config-store read/export facility and is an explicit release artifact. +2. Deploy the dual-reader binary while the schema-1 blob remains active. +3. Complete health checks on every deployed instance. +4. Push the migrated schema-2 configuration with the new CLI. +5. Verify registry order, provider order, browser asset hashes, and auction + health before declaring the cutover complete. + +An old binary must never serve a schema-2 blob. Rolling back after step 4 first +restores the archived schema-1 envelope, verifies that restoration, and only +then rolls the binary back. If the platform cannot coordinate those operations, +the release is paused rather than accepting an outage window. The compatibility +decoder is removed only in a later release after every supported deployment has +completed the schema-2 cutover. + ## Required Neutral Lifecycle Boundaries -### Response sharing annotation +### Request processing and sharing annotations -DataDome currently communicates a concrete marker to core so core buffers a -full response and applies private caching. Replace that marker with a neutral -response-sharing annotation owned by core. DataDome sets it through a -registered hook; core preserves the current buffering and cache behavior. +DataDome currently communicates a concrete request marker to core before +template lookup. Core uses it to bypass a shared template, require the origin +and a full HTML body, and stamp the final response private. Replacing it with a +late cache-only flag would change behavior. -The annotation represents only the existing shared-versus-request-private -decision. It is not a general policy or permissions system. +Core therefore owns a small monotonic `RequestProcessingRequirements` value +with three independent axes: -DataDome's tag-suppression and other integration-private request state stays -owned by DataDome and moves with the implementation. It may use neutral opaque -request/document state, but it is not folded into the response-sharing -annotation or exposed as a core vendor-specific field. +- shared-template eligibility: eligible or bypass; +- body processing: streaming allowed or full body required; +- response sharing: shared allowed or request-private. + +Every hook may only make a requirement more restrictive. The registry combines +requirements before template-cache and origin-selection decisions, carries the +result through HTML processing, and enforces final response privacy. This is a +processing contract, not a general policy, permission, or vendor-state system. + +DataDome sets these requirements inside the request-filter hook at the point +where it already decides client-tag suppression. Its tag-suppression detail +remains opaque integration-owned request/document state. Core sees only the +neutral requirements and an opaque token returned to later hooks; it has no +DataDome type, field, or JSON path. ### Request preparation and response finalization @@ -814,14 +1005,21 @@ and a direct finalization call in core. Add neutral typed hooks for those two existing lifecycle points. The registry owns opaque request-scoped state between them. -Adapters invoke registry preparation at the existing boundaries. Core invokes -finalization on the existing response path. No additional lifecycle stages are -introduced. +Preparation returns the opaque per-integration decision plus its declared +`RequestProcessingRequirements`. The neutral requirements are available before +the existing template-cache/private decision; under ESI, request-private opaque +state is never copied into a shared template. Adapters invoke registry +preparation at the existing boundaries and core invokes finalization on the +existing response path. No additional lifecycle call site is introduced. ## Browser Composition and APS Renderer -`trusted-server-js` builds only the neutral core IIFE and exposes a narrow API -for integration registration and module combination. +`trusted-server-js` builds only the neutral core IIFE. Its browser API exposes +the existing shared facilities integrations actually use: logging, slot lookup +and render helpers, normalized auction-response access, first-impression state, +and renderer registration and dispatch. Integration bundles import only +type-only declarations and call the installed browser API; Rollup/Vite treats +the runtime shim as external. The core IIFE initializes exactly one stateful registration object on the Trusted Server browser namespace before any integration IIFE runs. Integration @@ -835,40 +1033,90 @@ are concatenated in the ordering contract above. Deferred and standalone modules remain separate assets but retain their configuration-relative order and typed identities. +The Rust registration does not carry only a string module ID. Core owns a +neutral immutable `BrowserAsset` contract containing the typed module ID, +embedded bytes, SHA-256 hash, load mode, and trusted script-tag attributes. +Composition produces a `BrowserDocumentAssets` value containing: + +- the exact ordered byte parts and concatenated hash for the unified asset; +- the ordered deferred and permitted standalone assets with their bytes and + hashes; +- the fixed creative prelude; +- a deterministic document fingerprint covering injected asset IDs, hashes, + order, attributes, immutable inline-asset bytes, and stable configuration + fingerprints for generated inline head output. + +Trusted attributes from immediate assets are merged onto the unified script +tag. Duplicate names with different values fail composition; equal duplicates +collapse to one attribute. + +Static serving, cache-busting URLs, immutable-cache validation, and publisher +template keys consume `BrowserDocumentAssets`; core no longer performs a +crate-global `all_module_ids()` lookup. The fingerprint includes only assets +that can affect the composed document and includes GPT bootstrap bytes. It is +recomputed during composition whenever configuration or embedded bytes change. +Each head injector whose output varies with integration configuration supplies +a deterministic fingerprint contribution from the exact fields that affect its +output. Request-dependent head variation is permitted only when its neutral +`RequestProcessingRequirements` bypass shared-template reuse; request data is +never folded into the composition-wide fingerprint. + Browser core currently imports APS renderer logic directly. Replace that reverse dependency with one neutral renderer registration mechanism: - Core parses the existing renderer envelope far enough to identify its type and retain its payload. -- The APS browser module registers the parser and dispatcher for the existing - APS renderer type. +- The APS browser module registers validation and dispatch for the existing APS + renderer type. - Core dispatches through the registered renderer. -- Missing, duplicate, or rejecting renderers fail closed. +- GPT and Prebid call the neutral dispatcher and never import APS source. +- No APS source is inlined into the core, GPT, or Prebid IIFE. An enabled APS integration whose provider can emit APS renderer descriptors includes its immediate APS browser module. The core IIFE and fixed creative prelude load first, so APS registration is complete before a bid can render. +APS renderer registration is an immediate-only capability; composition rejects +a deferred APS renderer. Its current rendering mode continues to be read while +the synchronous unified script tag is executing, and the APS-owned trusted +attribute remains on that tag. A future switch to a standalone or deferred APS +asset requires replacing `document.currentScript` configuration first. + +Renderer failure is scoped to the owning renderer-bearing bid or message. A +missing or rejecting renderer suppresses that bid with no generic-creative or +native-Prebid fallback; unrelated bids and the page continue. A duplicate type +fails Rust composition, while a defensive browser-side duplicate poisons that +type rather than using last-registration-wins. No renderer route, DOM, message, +or beacon side effect occurs before the selected handler accepts the payload. The serialized descriptor, validation, sandbox flags, message authentication, timeouts, and render results do not change. +`gpt_bootstrap.js` moves with GPT into `trusted-server-integrations-js` and is +exported as a hashed integration-owned inline asset. GPT's Rust head injector +consumes that exported asset; core never embeds it directly. Tests resolve the +asset through the owning package instead of a cross-crate relative path. APS +golden renderer fixtures likewise move to an integration-owned shared fixture +location used by both Rust and browser tests. + ## Error Handling -Failures remain fail-closed and occur as early as the available information -allows. +Failures occur as early as the available information allows, with policy +defined per capability rather than one blanket rule. -Build-time failures include invalid discovered directories, generated catalog -errors, missing typed browser modules, JavaScript compilation failures, and -missing generated bundles. +Build-time failures include an invalid static catalog, missing typed browser +modules, JavaScript compilation failures, and missing generated bundles. CLI or startup failures include retired or mixed configuration shapes, unknown definitions, invalid integration settings, unresolved qualified provider references, duplicate routes, incompatible capabilities, invalid auction plans, and unavailable assets. -Runtime hook errors retain the current `Report` context and -HTTP behavior. Moving a concrete call behind a registry must not convert an -error into a warning, ignore it, or panic. +Runtime hooks retain their current `Report` context and HTTP +behavior. Request filters and HTML hooks preserve their existing propagation; +provider failures remain materialized provider outcomes; mediator launch or +parse failure continues to warn and fall back to local ranking; renderer +failure follows the per-bid rule above. Moving a concrete call behind a registry +must not change that hook's policy or introduce a panic. Core visibility changes remain narrow. Helpers move with their integration when possible. Core exposes a new public item only when a neutral cross-crate @@ -896,6 +1144,29 @@ provider-ID order to qualified configuration order. It preserves: - Cache privacy and full-buffer decisions. - The route and behavioral parity of Fastly, Axum, Cloudflare, and Spin. +The intentional compatibility breaks are: + +- Auction providers move from `[auction.providers]` beneath their owning + integration and references become qualified. +- APS providers no longer activate without an enabled `[integrations.aps]` + parent. +- `js_asset_proxy` is no longer implicitly first; migration examples and + procedures place it first to retain existing behavior unless the operator + reorders it. +- Provider failure responses and mediator input stop using the two remaining + lexical recovery sorts and use configuration order everywhere. +- Conflicting trusted attributes for the unified script tag fail composition + instead of warning and keeping the first value; identical duplicates still + collapse. +- Stored data gains the application schema and explicit order sidecars. The + rollout decoder, not the steady-state schema, provides temporary old-blob + compatibility. + +This specification does not preserve or coexist with PR #1084's current +`[integration]`, `[demand]`, and `[adserver]` configuration convention. That +conflict is resolved in favor of this single `[integrations]` design rather +than hidden behind aliases or precedence rules. + Bundle hashes and cache-busting URLs may change because browser sources are rebuilt in different crates. The server must emit URLs matching the new embedded hashes; bundle hashes are not a stable public contract. @@ -912,43 +1183,88 @@ four reviewable workstreams: The implementation plan must give each workstream its own verification checkpoint and keep behavior-preserving moves separate from intentional config -and ordering changes. Intermediate commits may add unused neutral contracts or -new crates, but no merged state may have two active catalogs, two provider -inventories, or adapter-specific composition paths. This scope does not include -the external plugin ecosystem proposed by PR #1084. +and ordering changes. + +The first merge milestone contains workstreams 1 through 3 and retains the +current operator schema and provider activation semantics through a temporary +normalization adapter. It delivers the crate boundary without an operator +cutover. The second milestone contains workstream 4, introduces stored schema +2 and the rollout decoder, and atomically replaces the temporary source +normalizer with the new source parser. +Intermediate commits may add unused neutral contracts or new crates, but no +merged state may have two active catalogs, two simultaneously interpreted +provider inventories, or adapter-specific composition paths. This scope does +not include the external plugin ecosystem proposed by PR #1084. + +The milestone-one normalizer is a compatibility boundary, not a second +catalog. It accepts only the current operator and stored shape, resolves current +profile strings through the new static definitions, preserves the current fixed +integration-builder order, lexical provider priority, and implicit APS +activation, and emits the one neutral model consumed by composition. Milestone +two atomically replaces that source parser with the new `[integrations]` parser; +it does not accept both operator inventories. Only the read-only stored-blob +decoder continues to accept the complete legacy shape during rollout. ## Migration Sequence Implementation may use small commits, but the merged workspace must never have two active integration or provider inventories. -1. Add neutral typed capability, lifecycle, renderer, and script-module - contracts to core without changing behavior. -2. Create `trusted-server-integrations-js`, move integration browser sources - and tests, and remove the browser-core APS import. -3. Create `trusted-server-integrations`, add directory discovery, and move all - fifteen current Rust implementation units. -4. Replace closed APS and Prebid profile variants with registered OpenRTB - profile and prepared-exchange behavior, and add the built-in `openrtb` - integration. +Steps 3 through 5 form the first atomic merge milestone. Preparatory commits may +compile and parity-test copied code in an unused new crate while the old catalog +remains authoritative, but the final milestone switch rewires every consumer +and deletes the old concrete sources together. No deployable revision selects +some integrations or browser assets from each catalog. + +1. Add neutral capability, processing-requirement, browser-asset, OpenRTB + profile/exchange, and mediator contracts to core. Extend the `test-utils` + feature with only neutral stubs required by extracted integration tests. +2. Replace the closed APS and Prebid profile variants and the mediator's legacy + `AuctionProvider` use while implementations are still in core. Convert GPT + diagnostics and DataDome call sites to the neutral lifecycle contracts. +3. Create `trusted-server-integrations-js`, move all integration browser + sources, tests, GPT bootstrap, and shared fixtures, remove every browser-core + APS import, and make the composed asset set authoritative for bytes and + hashes. +4. Create `trusted-server-integrations` with the explicit static catalog. Move + ordinary integrations first, then move DataDome and GPT diagnostics after + their lifecycle seams, APS and Prebid after the profile seam, and + `adserver_mock` after the mediator seam. Add the new `openrtb` adapter. 5. Move `TrustedServerAppConfig`, all integration-specific configuration, validation, inactive-secret preprocessing, and secret metadata into the - integrations crate. -6. Add the TOML source pre-pass, order-preserving maps, explicit stored - sequences, strong qualified provider IDs, and the breaking - integration-owned provider schema. -7. Rewire the CLI and all adapters to the single composition entry point that - returns settings, plan, and registry together. -8. Remove the old concrete directories, fixed builder/profile tables, - validation lists, and `[auction.providers]` schema. -9. Update examples, fixtures, operator documentation, and migration errors. + integrations crate. Rewire the CLI to the source-validation entry point and + all adapters to the single runtime composition entry point while retaining + the existing operator schema through the temporary normalizer. This is the + behavior-preserving crate-split milestone. +6. Add the TOML source pre-pass, ordered in-memory maps, application schema 2, + object-shaped storage with explicit order sidecars, strong qualified + provider IDs, the dual stored-schema reader, and the breaking + integration-owned provider schema. Atomically replace and remove the + milestone-one old-source normalizer so this CLI accepts only the new source + inventory. +7. Replace every lexical provider recovery sort with compiled-plan ordinals, + update adapter backend correlation naming, and activate configuration-order + semantics only after their parity and failure-path tests pass. +8. Update examples, fixtures, operator documentation, migration diagnostics, + CI aliases, browser scripts, and the rollout runbook. Deploy the dual reader, + then push schema 2 according to the rollout section. +9. Remove the read-only schema-1 blob decoder only in the later release defined + by the rollout contract. + +When files leave core, the Fastly-SDK migration guard is not weakened. Its +integration `include_str!` entries move to an equivalent guard owned by +`trusted-server-integrations`; neutral core entries remain in core. Integration +tests move integration-owned fixtures and helpers outward. Only platform-neutral +stubs become public under `test-utils`; production APIs are not widened merely +to preserve `cfg(test)` imports. ## Testing and Verification ### Discovery and dependency tests -- Every valid Rust directory appears exactly once in generated output. -- Invalid or duplicate directory IDs fail generation. +- The static Rust catalog contains exactly the sixteen expected definitions, + and every valid `src//mod.rs` directory appears exactly once. +- Invalid, missing, or duplicate catalog IDs fail tests or compilation. - JavaScript-only and Rust-only directories are accepted. - A typed Rust reference to an absent browser module fails compilation. - Embedded bundle hashes match built bytes. @@ -958,15 +1274,20 @@ two active integration or provider inventories. ### Configuration and ordering tests -- TOML parent-table order becomes `IntegrationSettings` order. +- TOML parent-table order becomes the integration-owned source-model order. - Nested provider declaration order is retained. - A parent integration or provider table declared after one of its descendants fails before typed deserialization. - Missing `enabled` fails; omitted integration tables remain inactive. -- TOML-to-envelope-to-runtime round trips preserve both orders byte-for-byte at - the sequence level. +- TOML-to-envelope-to-runtime round trips preserve both order sidecars exactly, + independent of JSON object-member order. +- Order sidecars reject missing, duplicate, extra, or mismatched map keys. +- Schema-1 stored blobs normalize through the transition reader; unknown schema + values fail before secrets are resolved. - Config-store loading produces the same registry, JavaScript, and provider order that the CLI validated. +- CLI validation never constructs runtime capabilities from unresolved secret + key names; the post-resolution runtime phase rejects unresolved values. - Disabled integrations may retain valid provider settings, contribute no providers or capabilities, and do not reorder enabled neighbors. - Bidder and mediator references to disabled integrations fail, including while @@ -980,10 +1301,16 @@ two active integration or provider inventories. with stable qualified identities. - Provider launch, response, mediator-input, and local equal-price tie order follows integration then local-provider declaration order. +- Transport-failure recovery paths retain plan order and never fall back to + lexical provider sorting. +- APS, Prebid, and `openrtb` fixtures cover enabled parents, disabled-parent + retention, missing-enabled rejection, and the nested provider migration. +- Qualified IDs that alias under Axum's legacy normalization receive distinct + correlation names or fail target validation before deployment. ### Capability and behavior parity tests -- The generated catalog contains all current integration IDs plus `openrtb`. +- The static catalog contains all current integration IDs plus `openrtb`. - APS and Prebid register page/browser and auction capabilities without core importing their types. - `adserver_mock` registers and is selected through the mediator capability. @@ -998,23 +1325,49 @@ two active integration or provider inventories. downcasts, vendor enums, or cross-provider state reuse. - Bidder routing, backend naming, notification suppression, telemetry identity, and mediator behavior remain equivalent apart from documented ordering. +- Active DataDome secrets are presence-checked and resolved through unchanged + object paths; inactive protection and bypass secrets are removed before the + shared resolver. +- Request-processing requirements preserve DataDome origin bypass, full-body + buffering, and final private caching, and prevent request-private GPT + diagnostics state from entering ESI templates. +- The dedicated mediator capability preserves request construction, ordered + response input, bounded transport, parsing, and local-ranking fallback without + exposing the legacy `AuctionProvider` trait. ### Browser tests - Output order is core, creative prelude, and configured integrations. - Immediate and deferred lists preserve configuration-relative order. - APS is absent from browser core and registers its renderer from its own IIFE. +- Core, GPT, and Prebid artifacts contain no private copy of APS renderer state. - Existing APS validation, sandbox, messaging, timeout, and rendering tests pass through neutral dispatch. -- Missing or duplicate renderer registrations fail closed. +- APS remains immediate and reads its rendering mode from the synchronous + unified tag; a deferred APS renderer is rejected during composition. +- Equal duplicate trusted script attributes collapse, while conflicting values + fail composition before HTML is served. +- Missing or rejecting renderers drop only the renderer-bearing bid with no + fallback or pre-acceptance side effect; duplicate registration poisons the + type or fails composition. +- Unified, deferred, standalone, and inline assets expose bytes and hashes that + match the emitted document fingerprint and static responses. +- Changing any integration setting that affects generated head output changes + the document fingerprint; request-dependent head variation bypasses shared + template reuse through processing requirements. +- GPT bootstrap and APS shared fixtures resolve from their integration-owned + package locations. ### Repository gates Before handoff, run every gate required by `AGENTS.md`, including Rust format, all target-matched clippy aliases, Fastly/Axum/Cloudflare/Spin tests, CLI and cross-adapter parity tests, required native and WASM builds, JavaScript builds -and Vitest suites for both browser crates, JavaScript formatting, and -documentation formatting. +and Vitest suites for both browser source roots, JavaScript formatting, and +documentation formatting. The explicit package lists in every Fastly +build/check/clippy/test alias include both new Rust crates where applicable; +host-target tests still run catalog-completeness and integration test support. +The migrated Fastly-SDK guard continues to scan the moved integration sources. ## Risks and Mitigations @@ -1040,10 +1393,20 @@ and parity tests around each boundary. TOML order can be lost through unordered Rust maps or JSON objects. -Mitigation: use ordered in-memory types and explicit sequences in the -hash-verified blob, enable `toml/preserve_order`, and reject -descendant-before-parent source declarations with the `toml_edit` pre-pass. -Test the complete push/store/load path rather than only the TOML parser. +Mitigation: use ordered in-memory types, object-shaped stored configuration, +and explicit order sidecars in the hash-verified blob. Enable +`toml/preserve_order`, reject descendant-before-parent source declarations with +the `toml_edit` pre-pass, and test the complete push/store/load path rather than +only the TOML parser. + +### Binary and stored-config cutover drift + +An old binary cannot consume application schema 2, and a binary rollback after +config cutover would otherwise fail at startup. + +Mitigation: deploy the dual reader before pushing schema 2, archive the prior +envelope, require restoration before binary rollback, and remove the legacy +reader only in a later release. ### Configuration order silently changes auction priority @@ -1065,16 +1428,21 @@ public item. CLI validation and adapter startup could use different catalogs or schemas. -Mitigation: both call the same generated composition API. No secondary -validation inventory is allowed. Adapters receive the already composed -settings, plan, and registry rather than reconstructing any of them. +Mitigation: both call the same catalog-backed source-validation API, and only +adapter startup continues through the post-secret runtime composition API. No +secondary validation inventory is allowed. Adapters receive the already +composed settings, plan, registry, and browser assets rather than reconstructing +any of them. ### Stale or incorrectly ordered browser artifacts -Splitting the Node build can embed previous output or load APS too late. +Splitting Rust ownership while sharing one Node workspace can embed previous +output, race build scripts, or load APS too late. -Mitigation: retain stale-output refusal, hash built bytes, load core and the -creative prelude first, and run end-to-end renderer and ordering tests. +Mitigation: coordinate the one workspace's build/install lock, retain +stale-output refusal, hash built bytes, load core and the creative prelude +first, reject deferred APS composition, and run artifact-level renderer, +fingerprint, and ordering tests. ## Acceptance Criteria @@ -1085,10 +1453,11 @@ The change is complete when: 2. All fifteen current concrete Rust implementation units live under `trusted-server-integrations/src//`. 3. The standard provider configuration is supplied by the built-in Rust-only - `openrtb` integration. + `openrtb` integration, making sixteen static definitions in total. 4. All integration browser sources, assets, fixtures, and tests live under `trusted-server-integrations-js`. -5. Rust and JavaScript inventories are independently directory-discovered. +5. Rust definitions use one explicit compile-checked catalog with a directory + completeness test; browser modules remain directory-discovered. 6. Core owns only neutral contracts and execution engines and imports no concrete integration. 7. One integration can register multiple typed capabilities; APS, Prebid, and @@ -1096,24 +1465,35 @@ The change is complete when: 8. `[integrations]` is the only concrete integration and auction-provider inventory. 9. Configuration and provider ordering survive config push and runtime loading - exactly and define the documented auction priority. + through validated order sidecars and define the documented auction priority. 10. The old `[auction.providers]` schema is rejected with targeted migration guidance. 11. The compiled auction plan retains PR #1016 behavior after normalization, except for the explicit change from lexical to configuration-order provider priority. 12. Browser core imports no concrete integration, and APS rendering works - through registration. + through one immediate registration without private copies in core, GPT, or + Prebid bundles. 13. `TrustedServerAppConfig`, integration secret handling, and final runtime composition are owned by `trusted-server-integrations`; core has no concrete config or loader dependency. -14. The CLI and all adapters use the same generated composition and validation - path and receive one settings/plan/registry composition. +14. The CLI and adapters use the same catalog-backed source validation, and all + adapters receive one post-secret-resolution + settings/plan/registry/browser-assets composition. 15. OpenRTB request-local state crosses the transport boundary through a prepared response parser without `Any` or vendor enum variants in core. 16. Explicit `enabled`, parent-before-descendant, disabled-retention, local-ID, and qualified-ID rules have end-to-end tests. -17. The full repository verification gates pass. +17. Schema-1 blobs remain readable for the documented rollout release, schema-2 + blobs preserve existing secret paths, and binary rollback requires verified + restoration of the archived schema-1 envelope. +18. Browser assets carry bytes and hashes through composition, publisher + template fingerprints vary with every composition-time external or inline + asset change, and request-dependent head variants bypass shared reuse. +19. Core test support, the Fastly-SDK migration guard, Cargo aliases, CI, + Dependabot, browser scripts, and the CLI Prebid builder cover the new crate + boundaries. +20. The full repository verification gates pass. ## Deferred Work From a3ae9fcb8741a856c7d6fd869b83f8c8547b44cb Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Wed, 23 Sep 2026 22:22:29 -0700 Subject: [PATCH 08/13] Finish deep review of integrations split design --- ...-09-17-split-integrations-crates-design.md | 606 +++++++++++++----- 1 file changed, 443 insertions(+), 163 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index ab00ea334..4e7899686 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -43,10 +43,11 @@ explicitly; filesystem discovery order will never affect execution. For auction providers, that order is also operational priority: it controls launch and response order, mediator input order, and local equal-price tie-breaking. -This design intentionally changes the auction configuration introduced by PR -#1016 while preserving that work's compiled-plan and runtime guarantees. It -adopts only the typed-registration portion of PR #1084 and excludes that PR's -external provider ecosystem and unrelated provider systems. +The compatibility baseline is the behavior shipped on `origin/main` at +`4c6d26a16`, not either prior pull request discussed below. This design changes +only the configuration, activation, and ordering behavior called out explicitly +in this document; all other current runtime, browser, CLI, and cache behavior is +preserved. This remains one design, but it has two merge milestones. The crate and runtime boundary moves first without changing operator configuration. The ordered, @@ -56,7 +57,7 @@ forcing the packaging move and configuration migration into one deployment. ## Context -On `main` at `6cae7f5da`, neutral registry machinery and concrete integrations +On `origin/main` at `4c6d26a16`, neutral registry machinery and concrete integrations share `crates/trusted-server-core/src/integrations`. The concrete Rust units are: @@ -89,6 +90,21 @@ embeds bundles and hashes into the `trusted-server-js` Rust crate. APS renderer code is imported directly by browser core even though APS does not currently have its own `index.ts`. +The same baseline includes managed Prebid User IDs and their OpenRTB EID/EC +flow, LiveRamp configuration through that existing managed-ID facility, +analytics-adapter selection in external Prebid bundles, cookie-keyed publisher +template caching, additional CLI ad-template and audit config consumers, and +documentation-snippet verification. These are current behavior and remain in +scope for parity even though they landed after this design was first drafted. + +`core/src/ec/prebid_eids.rs` is historically named after the first browser +producer, but its `ts-eids` ingestion, consent checks, EC finalization, +partner-graph ingestion, and admin diagnostics are shared identity machinery. +They remain in core and may be renamed to an integration-neutral module during +the move. Prebid-owned configuration and browser-module management move out; +the shared EID/EC machinery does not become a new capability family, and +LiveRamp does not become another integration definition. + Configuration is also split by implementation detail. Browser/page settings use `[integrations.]`, while server auction providers use `[auction.providers.]` plus `profile = "aps"` or @@ -108,16 +124,18 @@ These conditions produce five related problems: 5. Runtime integration order is not a stable property of the operator configuration. -## Relationship to Existing Work +## Historical Pull Requests (Non-Normative) -### PR #1016 +The following pull requests explain how some current code arrived in the +repository. They are not design authorities for this specification. The +normative inputs are the decisions in this document and behavior present on the +current baseline above. -PR #1016 made auction providers configuration-driven and introduced a single -validated, immutable auction plan shared by adapter backend construction, -runtime dispatch, browser demand, routing, and telemetry. It also separated a -configured provider instance from the OpenRTB profile implementation it uses. +### PR #1016 -This design preserves those runtime guarantees: +PR #1016 introduced the ancestor of the current compiled auction plan. The +following properties are now baseline repository behavior and are preserved +because current consumers depend on them, not because the PR is authoritative: - Multiple configured instances may use one integration implementation. - One validated provider identity remains shared by bidder routing, backend @@ -140,26 +158,12 @@ remaining shared auction budget after earlier providers launch. ### PR #1084 -PR #1084 proposes a much broader compile-time provider ecosystem: public -registration for external vendor crates, provider systems for identity, geo, -device, permission signals, demand and ad servers, a permission/jurisdiction -model, client-cycle EC resolution, provider-code governance, adapter and -EdgeZero composition work, and independent vendor ownership expectations. - -This design shares one idea with that proposal: one integration may register -multiple typed capabilities. It does not create the external ecosystem. The -new contracts serve the integrations compiled in this workspace; they are not -a stable third-party SDK or independent release boundary. - -The current PR #1084 configuration convention is mutually exclusive with this -design. PR #1084 selects implementations through top-level `[integration]`, -`[demand]`, and `[adserver]` provider selectors, removes `enabled` from -integration blocks, and does not treat APS as an integration. This design -deliberately chooses one ordered `[integrations]` inventory, explicit -`enabled`, and APS as a multi-capability integration. The two configurations -must not merge as parallel conventions. If this design is accepted, the -conflicting configuration and auction sections of PR #1084 must be superseded -or revised before that broader provider work proceeds. +PR #1084 explores a broader external-provider and plugin ecosystem. That scope +and its alternate configuration convention are not inputs to this design. This +specification independently chooses static workspace crates, typed capabilities +needed by current implementations, one ordered `[integrations]` inventory, +explicit `enabled`, and APS as an integration. No `[integration]`, `[demand]`, +or `[adserver]` selector convention is carried forward. ## Goals @@ -177,9 +181,8 @@ or revised before that broader provider work proceeds. lifecycle imports from core. 7. Make `[integrations]` the single ordered inventory for concrete integration configuration, including auction providers. -8. Preserve the runtime behavior and compiled-plan guarantees of PR #1016, - except that deterministic provider priority moves from lexical ID order to - configuration order. +8. Preserve behavior on the current `origin/main` baseline except for the + explicitly documented configuration, activation, and ordering changes. 9. Make core compile without depending on either integrations crate. 10. Keep all integrations statically linked; no runtime loading is introduced. @@ -193,7 +196,9 @@ This design does not introduce: - A stable public plugin or integration SDK. - Independent vendor release, compatibility, security-response, or governance policies. -- Identity, EC, geo, device, or permission-signal provider systems. +- New identity, EC, geo, device, or permission-signal provider systems. The + existing neutral `ts-eids`/EC flow and managed Prebid User ID behavior remain + supported. - A jurisdiction or permission-policy redesign. - Client-cycle EC resolution or provider-code allocation. - Upstream EdgeZero lifecycle, host-evidence, store, or adapter changes. @@ -247,7 +252,8 @@ crates/ lib/ package.json package-lock.json - build-browser.mjs + build-all.mjs + build-prebid-external.mjs src/core/ test/core/ src/ @@ -286,10 +292,14 @@ crates/ aps/ index.ts render.ts + renderer-document.html creative/ index.ts datadome/ index.ts + prebid/ + index.ts + user_id_modules.json ... test/ integrations/ @@ -307,7 +317,10 @@ sixteen definitions: fifteen moved implementations plus the new `openrtb` adapter. JavaScript-only `creative` remains valid without a Rust directory. Rust-only -integrations remain valid without a browser directory. +integrations remain valid without a browser directory. Cross-adapter Playwright +and parity tests remain in `trusted-server-integration-tests`; their paths and +load-order assertions change, but system tests do not become source owned by +one integration crate. `creative` is the sole fixed, non-configurable browser prelude in this design; it is runtime support rather than an operator integration. Directory discovery @@ -359,6 +372,7 @@ directory of implementations. It owns: validate, diff, and push mechanics. - The public runtime entry points that load a config-store blob and return one composed runtime value. +- A validated source-config view used by non-runtime CLI commands. `TrustedServerAppConfig` contains neutral core configuration plus the ordered integration-owned source configuration. Concrete integration configuration is @@ -376,11 +390,19 @@ values and constructs the plan, registry, and browser assets. Both phases use the same static catalog and integration schemas, but only adapters receive the final `TrustedServerComposition`. +The source phase returns a conceptual `ValidatedSourceConfig`. It retains the +typed operator configuration and exposes only the views CLI consumers need: +neutral global settings, ordered integration and qualified-provider metadata, +and integration-owned read models such as Prebid external-bundle inputs. This +is not a runtime registry and contains no resolved secret values or executable +capability objects. + That runtime value, conceptually `TrustedServerComposition`, contains the -validated neutral `Settings`, one `Arc`, and one -`IntegrationRegistry`, plus the composed `BrowserDocumentAssets`. Adapters -consume this value; they do not separately compile the auction plan, rebuild -the integration registry, or enumerate browser bundles. +validated neutral `Settings`, one `Arc`, one +`IntegrationRegistry`, the composed `BrowserDocumentAssets`, and a canonical +digest of the complete normalized resolved configuration. Adapters consume +this value; they do not separately compile the auction plan, rebuild the +integration registry, enumerate browser bundles, or reconstruct the digest. Core retains neutral config-store access, Fastly chunk reconstruction, blob envelope verification, secret-resolution primitives, global settings types, @@ -400,11 +422,27 @@ config-store bytes → TrustedServerComposition ``` -The CLI imports `TrustedServerAppConfig` and its config command wrappers from -`trusted-server-integrations`. Each wrapper performs the source-aware pre-pass -and source-phase catalog validation before delegating storage and diff mechanics -to EdgeZero's typed CLI functions. No EdgeZero source change or new host service -is required. +The CLI imports `TrustedServerAppConfig`, `ValidatedSourceConfig`, and its +config command wrappers from `trusted-server-integrations`. Each wrapper +performs the source-aware pre-pass and source-phase catalog validation before +delegating storage and diff mechanics to EdgeZero's typed CLI functions. The +validated bytes are passed to EdgeZero through an immutable temporary snapshot, +so the bytes checked by the pre-pass are exactly the bytes diffed or pushed; +the original operator path remains the path shown in diagnostics. No EdgeZero +source change or new host service is required. + +Read-only `config ad-templates` and `audit ad-templates` commands load the +effective source view with the existing optional environment overlay. Mutating +or generator commands load file bytes without the overlay, so environment-only +values are never persisted. Recovery-oriented ad-template generation may run a +structural-only pre-pass against an otherwise invalid baseline, preserving its +current warning and non-disclosure behavior, but the final candidate must pass +the complete source validation before atomic write. `ts prebid bundle` obtains +typed bidder, User ID, analytics, and managed-module requirements through the +Prebid source-view facade; it retains process invocation and atomic metadata +patching but no longer maintains a partial duplicate Prebid schema. Provider +diagnostics display qualified providers in configuration order rather than +alphabetizing a detached map. ## Static Rust Catalog and Browser Discovery @@ -433,17 +471,31 @@ project, one lockfile, and one set of Vitest, ESLint, Prettier, Vite, and Prebid aliases. The canonical project root remains `crates/trusted-server-js/lib`; `trusted-server-integrations-js` does not add a second `package.json` or lockfile. Neutral browser sources remain under -`trusted-server-js`; integration sources, fixtures, and tests live under -`trusted-server-integrations-js`. Separate build targets emit neutral and -integration artifacts into their owning Rust crates. The build helpers -coordinate dependency installation and output generation so parallel Cargo -build scripts cannot race or consume stale artifacts. +`trusted-server-js`; integration sources, owned unit/artifact fixtures, and +owned unit/artifact tests live under `trusted-server-integrations-js`. The +shared TypeScript, lint, format, and test configurations include that sibling +source root explicitly. Separate build targets emit neutral and integration +artifacts into owner-specific output directories and validate per-target +manifests before embedding them. One cross-process lock covers dependency +installation, output cleanup, build execution, discovery, manifest validation, +and artifact copy, so parallel Cargo build scripts cannot race or consume stale +or partially replaced output. The integration build discovers immediate directories containing `index.ts` and emits one self-contained IIFE per entry point. Its Cargo build embeds each -output and its SHA-256 hash. CI, Dependabot, browser integration scripts, and -the CLI Prebid builder use the single workspace root rather than maintaining a -second dependency graph. +output and its SHA-256 hash. CI, browser integration scripts, and the CLI +Prebid builder use the single workspace root rather than maintaining a second +dependency graph; Dependabot continues to watch only its one lockfile. + +`build-prebid-external.mjs` and its npm command remain at the canonical Node +root as build orchestration, not browser runtime. The Prebid registry, aliases, +shims, and other integration-owned source inputs move with Prebid into +`trusted-server-integrations-js`. The launcher receives their resolved sibling +paths explicitly and has no hard-coded `src/integrations/prebid` assumption. +The CLI resolves the canonical package root for dependencies and obtains the +integration-owned input paths and typed module requirements from the +integrations facade; it does not locate a registry through its own relative +path constant. The generated Rust API exposes typed module identifiers rather than accepting unchecked strings. A Rust registration referencing a missing browser module @@ -559,8 +611,9 @@ runtime, after secrets are resolved: 1. Parse `[integrations]` into an ordered sequence. 2. Resolve each ID against the static definition catalog. 3. Ask the owning definition to parse and validate its complete configuration. -4. For each enabled integration, construct its typed capability registrations - in the validated integration order restored from the storage sidecar. +4. For each enabled integration, construct the typed capability registrations + activated by its validated settings, in integration order restored from the + storage sidecar. 5. Collect integration-owned auction provider instances and profiles. 6. Ask core to compile the single canonical auction plan. 7. Resolve typed browser modules and construct the neutral integration @@ -568,24 +621,50 @@ runtime, after secrets are resolved: An absent integration is inactive. Every explicit parent integration table must contain `enabled = true` or `enabled = false`; there is no integration-specific -default. An explicitly disabled integration may retain its settings and provider -instances but contributes no runtime capabilities or providers. Configuration -cannot activate APS through an auction plan while omitting -`[integrations.aps]`, and bidder or mediator references to a disabled -integration fail validation. - -The APS rule is an intentional activation break from PR #1016. Today an APS -auction profile can activate server-side rendering support without an enabled -`[integrations.aps]` browser block. After cutover, every APS provider requires -an explicit `[integrations.aps]` parent with `enabled = true`; that one parent -activates APS's server, page, and browser capabilities together. - -Disabled configuration still receives structural validation: unknown fields, -wrong types, duplicate IDs, and invalid values that are present fail. Missing -active-only required values and inactive secret references do not fail until -the integration is enabled. This permits operators to turn off an integration -without deleting prepared configuration while preventing disabled behavior from -leaking into the runtime plan. +default. `enabled` is the integration's master gate, not an assertion that every +optional capability is configured. An explicitly disabled integration may +retain its settings and provider instances but contributes no runtime +capabilities or providers. Configuration cannot activate APS or Prebid through +an auction plan while omitting its parent, and bidder or mediator references to +a disabled integration fail validation. + +Provider-owning integrations use the following explicit activation rules. They +do not add capability names or implementation discriminators to operator +configuration. + +| Integration | Enabled configuration | Runtime contribution | +| --------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| `aps` | No provider instances | No runtime contribution; retained settings remain structurally validated. | +| `aps` | One or more provider instances | One OpenRTB provider plan per instance plus coupled APS renderer/head/browser support and the mode-dependent proxy required by those providers. | +| `prebid` | No providers and no `external_bundle_url` | No runtime contribution; retained settings remain structurally validated. | +| `prebid` | Provider instances, but no `external_bundle_url` | Server-side OpenRTB provider plans only; no proxy, rewriter, head injector, managed browser User IDs, or deferred browser module. | +| `prebid` | Valid `external_bundle_url` and browser settings | Existing Prebid proxy, rewriter, head injector, managed User IDs, and deferred browser module, with zero or more server providers. | +| `openrtb` | Zero or more provider instances | One standard OpenRTB provider plan per instance; zero instances is a valid staged no-op. | +| `adserver_mock` | Enabled parent | The existing mediator capability, independently of provider count. | + +Thus a current server-only Prebid provider migrates beneath an enabled Prebid +parent without activating Prebid's page/browser behavior. Browser-only Prebid +continues to be selected by the same required external-bundle URL that current +startup validation already uses. Managed User ID or other browser-only settings +without that URL fail validation rather than activating a partial browser path. +For APS, a configured provider necessarily activates its renderer support; an +enabled APS parent with no provider is a staged no-op. + +Requiring the enabled parent is an intentional activation change from the +current split inventory. Today an APS auction profile can activate rendering +support without an enabled `[integrations.aps]` block. After cutover, every APS +provider requires an explicit enabled parent, and its coupled server and +renderer capabilities activate together. + +Disabled configuration receives schema-safety validation only: unknown fields, +wrong types, integration/provider ID grammar, duplicates, and secret-reference +name/store-reference/collision/adapter rules still fail. Active-only required +fields and value validators—including ranges, endpoint policy, format patterns, +and cross-field rules—are deferred until the integration is enabled. This preserves current +disabled placeholders such as the example Google Tag Manager container while +preventing malformed structure or secret references from being stored. At +runtime, integration-owned preprocessing removes inactive secret paths before +value resolution, so disabled behavior does not leak into the runtime plan. The four adapters share the runtime composition path. The CLI and adapters share the source/catalog validation rules; runtime and deploy validation cannot @@ -603,7 +682,6 @@ the built-in `openrtb` integration. ```toml [integrations.prebid] enabled = true -client_side_bidders = ["example-browser"] [integrations.prebid.auction.providers.pbs-main] endpoint = "https://prebid.example.com/openrtb2/auction" @@ -636,6 +714,12 @@ providers therefore do not accept `profile`, `implementation`, or fields form one typed provider schema owned by that integration. Common notification settings may remain in the nested `notifications` table. +The legacy `protocol = "openrtb-2.6"` field is also retired and rejected with +migration guidance. Every provider capability in this design uses the existing +OpenRTB 2.6 engine, so repeating its one accepted protocol value adds no choice. +A future non-OpenRTB engine requires a separate capability design rather than a +string switch in this schema. + Multiple named provider instances are supported beneath one integration. The standard generic path uses the same inventory: @@ -742,6 +826,12 @@ The contract is: if the deadline expires or adapter timeout canonicalization reaches zero during that launch loop. +Ordered runtime introspection carries the configured ordinal with each +integration and provider. Lookup indexes may use maps, but iterating a +`BTreeMap`, `HashMap`, or alphabetized metadata view never defines or displays +execution priority. CLI diagnostics and registry metadata that show order use +the ordered plan/registration view. + The current hard-coded requirement that `js_asset_proxy` remain the first rewriter is retired. Rewriter chaining follows the same operator-visible integration order as every other hook. Migration guidance places @@ -770,20 +860,25 @@ then permits the existing EdgeZero scalar environment overlay; overlays may replace values but may not create, remove, or reorder integration or provider tables. -Every entry point that accepts TOML uses this pre-pass, including local loading -and the Trusted Server wrappers around CLI validate, diff, and push. EdgeZero's -typed mechanics remain responsible for overlay, validation invocation, diff, -envelope construction, consent, and store writes after the pre-pass succeeds. +Every entry point that accepts TOML uses this pre-pass, including local loading, +CLI validate/diff/push, ad-template diagnostics and candidate validation, and +the Prebid bundle command. Recovery mutators may request the structural-only +mode described above, but final candidate validation always uses the complete +mode. EdgeZero's typed mechanics remain responsible for overlay, validation +invocation, diff, envelope construction, consent, and store writes after the +pre-pass succeeds. EdgeZero does not currently expose a pre-parse hook. The Trusted Server wrappers -therefore deliberately duplicate its app-config path rule: an explicit -`--app-config` wins; otherwise the path is `/.toml`. -The wrapper reads that source for structural validation and EdgeZero reads it -again for typed processing. Parity tests cover explicit and default paths, -manifest paths with and without parent directories, and `--no-env`. The -environment overlay can replace only scalar leaves already present in TOML; it -cannot create an omitted `enabled` field, integration, provider, table, or -array. Operator templates must contain every leaf intended for overlay. +therefore duplicate its app-config path rule: an explicit `--app-config` wins; +otherwise the path is `/.toml`. A wrapper reads the +source once, performs the pre-pass, and delegates typed processing against a +private immutable snapshot of those exact bytes; it does not validate one read +and allow EdgeZero to reopen a concurrently changed operator file. Parity tests +cover exact-byte delegation, explicit and default paths, manifest paths with and +without parent directories, and `--no-env`. The environment overlay can replace +only scalar leaves already present in TOML; it cannot create an omitted +`enabled` field, integration, provider, table, or array. Operator templates must +contain every leaf intended for overlay. ### Config-store representation @@ -843,7 +938,10 @@ constructed. Integration-owned inactive-secret preprocessing also receives the object-shaped integration entry by ID and writes any resolved value back to the same entry; it never searches an `{ id, config }` sequence. End-to-end tests prove both active DataDome secret fields are presence-checked, resolved, and -removed when inactive. +removed when inactive. Before storage, EdgeZero's static secret metadata still +validates every reference actually present in source, including a reference +retained under a disabled integration; runtime filtering prevents inactive +value resolution, not source syntax or adapter validation. The private Rust type names may differ, but the serialized order must be explicit and covered by compatibility tests across: @@ -853,8 +951,8 @@ trusted-server.toml → typed CLI configuration → hash-verified blob envelope → config store - → runtime Settings - → registry, JavaScript lists, and AuctionPlan + → runtime composition + → Settings, registry, JavaScript lists, configuration digest, and AuctionPlan ``` ## Configuration Ownership and Validation @@ -890,7 +988,8 @@ Validation fails for: - An unknown integration ID. - A parent integration table with a missing or non-boolean `enabled` field. -- Integration configuration not accepted by its owner. +- Integration configuration not accepted by its owner under the enabled or + disabled validation phase defined above. - A descendant integration or provider table declared before its explicit parent. - Duplicate local provider IDs or duplicate qualified provider identities. @@ -918,20 +1017,25 @@ never merges inventories. Representative mappings are: -| Previous configuration | New configuration | -| --------------------------------------------------------------- | --------------------------------------------------------- | -| `[auction.providers.pbs-main]` with `profile = "prebid-server"` | `[integrations.prebid.auction.providers.pbs-main]` | -| `[auction.providers.aps-main]` with `profile = "aps"` | `[integrations.aps.auction.providers.aps-main]` | -| APS provider with no `[integrations.aps]` parent | Add `[integrations.aps]` with `enabled = true` | -| A standard profile provider named `example-direct` | `[integrations.openrtb.auction.providers.example-direct]` | -| `[auction.providers..profile_config]` | Flattened into the owning integration's provider table | -| Bidder route `provider = "pbs-main"` | `provider = "prebid.pbs-main"` | - -Old `[auction.providers]`, `profile`, and `profile_config` fields fail with an -actionable message naming the new integration-owned location. A mixed old/new -operator configuration also fails. There is no deprecation interval for -operator TOML. The temporary old-blob reader is not an accepted source format -and does not make old fields valid in the new CLI. +| Previous configuration | New configuration | +| --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| `[auction.providers.pbs-main]` with `profile = "prebid-server"` | `[integrations.prebid.auction.providers.pbs-main]` | +| `[auction.providers.aps-main]` with `profile = "aps"` | `[integrations.aps.auction.providers.aps-main]` | +| APS provider with no `[integrations.aps]` parent | Add `[integrations.aps]` with `enabled = true` | +| Server-only Prebid provider with no Prebid browser block | Add `[integrations.prebid]` with `enabled = true`; omit `external_bundle_url` and browser-only fields | +| Existing browser Prebid block with `enabled = true` | Retain its browser fields and nest any server providers beneath the same enabled parent | +| Any retained integration parent that relied on a default | Add an explicit `enabled = true` or `enabled = false` | +| A standard profile provider named `example-direct` | `[integrations.openrtb.auction.providers.example-direct]` | +| `[auction.providers..profile_config]` | Flattened into the owning integration's provider table | +| `protocol = "openrtb-2.6"` | Remove it; the registered provider capability fixes the protocol | +| Bidder route `provider = "pbs-main"` | `provider = "prebid.pbs-main"` | + +Old `[auction.providers]`, `profile`, `profile_config`, and `protocol` fields +fail with an actionable message naming the new integration-owned location or +instructing the operator to remove the fixed protocol. A mixed old/new operator +configuration also fails. There is no deprecation interval for operator TOML. +The temporary old-blob reader is not an accepted source format and does not +make old fields valid in the new CLI. Examples, integration fixtures, environment-overlay tests, CLI documentation, and operator guides migrate in the same change. @@ -964,6 +1068,13 @@ Rollout is ordered: 5. Verify registry order, provider order, browser asset hashes, and auction health before declaring the cutover complete. +The rollout runbook must name and drill a concrete export and restore mechanism +for every deployed adapter before milestone 2. This is an external release +precondition, not an assumed CLI feature. In particular, the current default +remote Spin deployment path cannot read deployed config through the locked +EdgeZero CLI; it must use a verified platform/control-plane export and restore +facility or the schema-2 rollout for that target is blocked. + An old binary must never serve a schema-2 blob. Rolling back after step 4 first restores the archived schema-1 envelope, verifies that restoration, and only then rolls the binary back. If the platform cannot coordinate those operations, @@ -992,6 +1103,15 @@ requirements before template-cache and origin-selection decisions, carries the result through HTML processing, and enforces final response privacy. This is a processing contract, not a general policy, permission, or vendor-state system. +Registry requirements are an additional monotonic veto, never an alternate +cache authorization path. Shared-template eligibility remains the conjunction +of all current method, request-cache, diagnostics, key-cookie, bypass-cookie, +unlisted-cookie, cookie-independent-origin, assembly-mode, and origin-response +checks plus the registry requirement. The same combined request decision still +governs both warm lookup and cold-store authorization. A registry hook can make +an otherwise shareable request private or origin-bound; it cannot make a +request shareable when any existing cookie or cache gate rejected it. + DataDome sets these requirements inside the request-filter hook at the point where it already decides client-tag suppression. Its tag-suppression detail remains opaque integration-owned request/document state. Core sees only the @@ -1008,9 +1128,13 @@ between them. Preparation returns the opaque per-integration decision plus its declared `RequestProcessingRequirements`. The neutral requirements are available before the existing template-cache/private decision; under ESI, request-private opaque -state is never copied into a shared template. Adapters invoke registry -preparation at the existing boundaries and core invokes finalization on the -existing response path. No additional lifecycle call site is introduced. +state is never copied into a shared template. Registry preparation remains at +both existing locations: adapter boundaries and the idempotent core publisher +safety-net boundary used by direct core callers. Request extensions make a +second preparation a no-op. Core invokes finalization on the existing response +path. The current GPT-enabled auction-correlation check becomes a neutral +registry/request-state query, so core retains neither a concrete GPT import nor +a new lifecycle call site. ## Browser Composition and APS Renderer @@ -1018,8 +1142,10 @@ existing response path. No additional lifecycle call site is introduced. the existing shared facilities integrations actually use: logging, slot lookup and render helpers, normalized auction-response access, first-impression state, and renderer registration and dispatch. Integration bundles import only -type-only declarations and call the installed browser API; Rollup/Vite treats -the runtime shim as external. +type-only declarations. Runtime calls go through a stateless accessor mapped by +the integration build to the already-installed Trusted Server browser +namespace; an integration IIFE does not rely on an unresolved ESM import or +bundle a second state owner. The core IIFE initializes exactly one stateful registration object on the Trusted Server browser namespace before any integration IIFE runs. Integration @@ -1050,16 +1176,35 @@ Trusted attributes from immediate assets are merged onto the unified script tag. Duplicate names with different values fail composition; equal duplicates collapse to one attribute. -Static serving, cache-busting URLs, immutable-cache validation, and publisher -template keys consume `BrowserDocumentAssets`; core no longer performs a -crate-global `all_module_ids()` lookup. The fingerprint includes only assets -that can affect the composed document and includes GPT bootstrap bytes. It is -recomputed during composition whenever configuration or embedded bytes change. -Each head injector whose output varies with integration configuration supplies -a deterministic fingerprint contribution from the exact fields that affect its +Static serving, cache-busting URLs, and immutable-cache validation consume +`BrowserDocumentAssets`; core no longer performs a crate-global +`all_module_ids()` lookup. Its document fingerprint includes only assets that +can affect the composed document and includes GPT bootstrap bytes. Each head +injector whose generated inline output varies with integration configuration +supplies the exact immutable bytes or a deterministic contribution for that output. Request-dependent head variation is permitted only when its neutral `RequestProcessingRequirements` bypass shared-template reuse; request data is -never folded into the composition-wide fingerprint. +never folded into a composition-wide fingerprint. + +Publisher template invalidation is broader than the browser document. During +composition, `trusted-server-integrations` hashes a canonical serialization of +the complete normalized resolved configuration: all neutral `Settings` and the +full configuration of every integration in configuration order, including +disabled retained entries. Hashing the complete model deliberately +over-invalidates so a future rewriter, postprocessor, proxy mapping, cookie +policy, or other HTML-shaping field cannot be omitted from a hand-maintained +allowlist. The serialized bytes and resolved secret values are fed directly to +the digest and are never logged, returned, or used as cache-key plaintext. + +Core computes the existing template fingerprint from that configuration digest +and `BrowserDocumentAssets.document_fingerprint`. This composite replaces only +the old complete-`Settings` plus global-bundle digest; it does not replace any +other `TemplateCacheKey` dimension. Full URL, request host and scheme, origin +identity, assembly mode, ordered `Vary` values, selected cookie values, and +`TEMPLATE_SCHEMA_VERSION` remain independent key inputs. Transform-shape +changes still bump `TEMPLATE_SCHEMA_VERSION`. Tests prove neutral settings, +non-head integration rewriter settings, external and inline assets, and cookie +policy changes invalidate templates without exposing raw configuration. Browser core currently imports APS renderer logic directly. Replace that reverse dependency with one neutral renderer registration mechanism: @@ -1081,6 +1226,12 @@ the synchronous unified script tag is executing, and the APS-owned trusted attribute remains on that tag. A future switch to a standalone or deferred APS asset requires replacing `document.currentScript` configuration first. +GPT is also immediate. Its current bootstrap reads `document.currentScript` +during module evaluation, so `data-ts-gam-attribution` remains on the unified +synchronous tag and an artifact-level test proves the value is available at +evaluation time. Moving source ownership must not silently make GPT deferred or +move that attribute to a later tag. + Renderer failure is scoped to the owning renderer-bearing bid or message. A missing or rejecting renderer suppresses that bid with no generic-creative or native-Prebid fallback; unrelated bids and the page continue. A duplicate type @@ -1091,6 +1242,15 @@ or beacon side effect occurs before the selected handler accepts the payload. The serialized descriptor, validation, sandbox flags, message authentication, timeouts, and render results do not change. +The production `APS_RENDERER_DOCUMENT`, including its inline browser +JavaScript, moves from the APS Rust source into +`trusted-server-integrations-js/lib/src/integrations/aps/renderer-document.html`. +The browser crate exports its immutable bytes and hash; APS Rust serves those +exact bytes with the existing content type, CSP, and other response headers. +The document's nonce binding, sandbox, message authentication, runner load, and +failure tests move with the asset. No production browser program remains as a +Rust string literal. + `gpt_bootstrap.js` moves with GPT into `trusted-server-integrations-js` and is exported as a hashed integration-owned inline asset. GPT's Rust head injector consumes that exported asset; core never embeds it directly. Tests resolve the @@ -1141,13 +1301,28 @@ provider-ID order to qualified configuration order. It preserves: order. - Existing provider identity fields, with values migrated from local IDs such as `pbs-main` to qualified IDs such as `prebid.pbs-main`. -- Cache privacy and full-buffer decisions. +- Managed Prebid User ID aliases, collision checks, consent gating, opaque + LiveRamp envelopes, OpenRTB EID production, EC partner ingestion, and admin + diagnostics. +- External Prebid bidder, User ID, and analytics-module selection, manifests, + hashes, SRI values, and runtime codes. +- Cache privacy, full-buffer decisions, cookie-key and bypass policy, and every + existing publisher template-key dimension. +- Current CLI ad-template diagnostics, audit/generator recovery behavior, and + Prebid bundle mutation behavior. - The route and behavioral parity of Fastly, Axum, Cloudflare, and Spin. The intentional compatibility breaks are: - Auction providers move from `[auction.providers]` beneath their owning integration and references become qualified. +- The fixed legacy `protocol = "openrtb-2.6"` field is removed from operator + source rather than copied into each integration-owned provider. +- Every retained integration parent requires an explicit `enabled` value, + including parents whose current schema supplies a default. +- Integration and provider parents must use ordinary table headers in + parent-before-descendant order; dotted-key or inline-table parent shorthand + is rejected. - APS providers no longer activate without an enabled `[integrations.aps]` parent. - `js_asset_proxy` is no longer implicitly first; migration examples and @@ -1162,10 +1337,9 @@ The intentional compatibility breaks are: rollout decoder, not the steady-state schema, provides temporary old-blob compatibility. -This specification does not preserve or coexist with PR #1084's current -`[integration]`, `[demand]`, and `[adserver]` configuration convention. That -conflict is resolved in favor of this single `[integrations]` design rather -than hidden behind aliases or precedence rules. +No alternate `[integration]`, `[demand]`, or `[adserver]` selector convention +is supported alongside the single `[integrations]` design. There are no aliases +or precedence rules between competing inventories. Bundle hashes and cache-busting URLs may change because browser sources are rebuilt in different crates. The server must emit URLs matching the new @@ -1194,7 +1368,7 @@ normalizer with the new source parser. Intermediate commits may add unused neutral contracts or new crates, but no merged state may have two active catalogs, two simultaneously interpreted provider inventories, or adapter-specific composition paths. This scope does -not include the external plugin ecosystem proposed by PR #1084. +not include an external plugin ecosystem. The milestone-one normalizer is a compatibility boundary, not a second catalog. It accepts only the current operator and stored shape, resolves current @@ -1205,6 +1379,23 @@ two atomically replaces that source parser with the new `[integrations]` parser; it does not accept both operator inventories. Only the read-only stored-blob decoder continues to accept the complete legacy shape during rollout. +Milestone exit criteria are independent: + +- **Milestone 1 — crate boundary:** only the current operator and stored schema + are accepted; fixed integration-builder order, lexical provider priority, + implicit APS activation, and all current browser, CLI, cache, and adapter + behavior remain unchanged. Every live consumer uses the new composition root, + old concrete sources and the old catalog are deleted together, and the full + repository gates pass. +- **Milestone 2 — ordered configuration cutover:** the new operator source is + the only writable shape; the dual stored-schema reader is deployed; qualified + identities and order sidecars are used end to end; every normal and recovery + path uses plan ordinals; activation rules, CLI output, examples, diagnostics, + scripts, documentation, and the rollout runbook are updated together; and the + full repository and rollout tests pass before schema 2 is pushed. +- **Later release — cleanup:** the schema-1 reader is removed only after the + rollback and support conditions in the rollout contract are satisfied. + ## Migration Sequence Implementation may use small commits, but the merged workspace must never have @@ -1222,10 +1413,12 @@ some integrations or browser assets from each catalog. 2. Replace the closed APS and Prebid profile variants and the mediator's legacy `AuctionProvider` use while implementations are still in core. Convert GPT diagnostics and DataDome call sites to the neutral lifecycle contracts. -3. Create `trusted-server-integrations-js`, move all integration browser - sources, tests, GPT bootstrap, and shared fixtures, remove every browser-core - APS import, and make the composed asset set authoritative for bytes and - hashes. +3. Create `trusted-server-integrations-js`, move all integration-owned browser + sources, unit/artifact tests, GPT bootstrap, the APS renderer document, + registry inputs, and shared fixtures, remove every browser-core APS import, + and make the composed asset set authoritative for bytes and hashes. Retain + cross-adapter system tests in `trusted-server-integration-tests` and update + their paths and load order. 4. Create `trusted-server-integrations` with the explicit static catalog. Move ordinary integrations first, then move DataDome and GPT diagnostics after their lifecycle seams, APS and Prebid after the profile seam, and @@ -1251,6 +1444,11 @@ some integrations or browser assets from each catalog. 9. Remove the read-only schema-1 blob decoder only in the later release defined by the rollout contract. +Steps 6 through 8 are one deployable milestone-two cutover. They may be +implemented as separately reviewed commits, but schema 2 must not merge or +deploy while lexical recovery ordering, CLI consumers, or operator guidance +still implement the old contract. + When files leave core, the Fastly-SDK migration guard is not weakened. Its integration `include_str!` entries move to an equivalent guard owned by `trusted-server-integrations`; neutral core entries remain in core. Integration @@ -1271,6 +1469,9 @@ to preserve `cfg(test)` imports. - Core has no dependency on either integrations crate. - Browser core imports no integration source. - Adapters and CLI import no concrete integration module. +- A dedicated native test/clippy gate executes the integrations catalog and + host-only completeness tests; relying on adapter dependency builds is not + sufficient to run them. ### Configuration and ordering tests @@ -1286,10 +1487,28 @@ to preserve `cfg(test)` imports. values fail before secrets are resolved. - Config-store loading produces the same registry, JavaScript, and provider order that the CLI validated. +- Registry metadata and CLI provider diagnostics report configured ordinals; + lookup-map or alphabetic iteration cannot masquerade as execution order. - CLI validation never constructs runtime capabilities from unresolved secret key names; the post-resolution runtime phase rejects unresolved values. -- Disabled integrations may retain valid provider settings, contribute no +- Read-only CLI consumers see the effective overlay through + `ValidatedSourceConfig`; mutators and generators operate on file-only bytes, + retain invalid-baseline recovery where currently supported, and fully + validate the final candidate without persisting overlay values. +- Validate/diff/push delegate the exact immutable source snapshot that passed + the pre-pass; a concurrent edit cannot substitute different pushed bytes. +- Prebid bundle selection and managed-module requirements come from the + integration-owned source view, not a CLI-local partial schema or hard-coded + registry path. +- Disabled integrations may retain structurally valid provider settings, contribute no providers or capabilities, and do not reorder enabled neighbors. +- Disabled placeholder values that current examples rely on, including the + Google Tag Manager placeholder container, deserialize safely and defer their + active-only format validation until enabled. +- Retained secret references in disabled source still pass EdgeZero name, + store-reference, collision, and adapter validation; omitted active-only + references are accepted, and inactive paths are not value-resolved at + runtime. - Bidder and mediator references to disabled integrations fail, including while the global auction is disabled. - Nested-only, unknown, missing-enabled, descendant-before-parent, and mixed @@ -1305,6 +1524,9 @@ to preserve `cfg(test)` imports. lexical provider sorting. - APS, Prebid, and `openrtb` fixtures cover enabled parents, disabled-parent retention, missing-enabled rejection, and the nested provider migration. +- Activation-matrix tests cover APS and Prebid with zero and multiple + providers, server-only Prebid without browser activation, browser-only Prebid, + standard OpenRTB, and the `adserver_mock` mediator. - Qualified IDs that alias under Axum's legacy normalization receive distinct correlation names or fail target validation before deployment. @@ -1318,9 +1540,12 @@ to preserve `cfg(test)` imports. - DataDome privacy and buffering behavior remains unchanged. - GPT diagnostics preparation, bootstrap injection, finalization, and caching remain unchanged on every adapter path. +- GPT diagnostics preparation is idempotent across adapter preparation and the + core publisher safety net, and auction correlation uses neutral registry + request state rather than a concrete GPT import. - APS and Prebid request construction, transport, parsing, response admission, - and auction results remain equivalent to PR #1016 behavior except for the - documented provider-priority change. + and auction results remain equivalent to current `origin/main` behavior + except for the documented provider-priority change. - Prepared response parsers consume profile-owned request state without `Any`, downcasts, vendor enums, or cross-provider state reuse. - Bidder routing, backend naming, notification suppression, telemetry identity, @@ -1328,9 +1553,16 @@ to preserve `cfg(test)` imports. - Active DataDome secrets are presence-checked and resolved through unchanged object paths; inactive protection and bypass secrets are removed before the shared resolver. +- Neutral `ts-eids` ingestion retains managed User ID aliases and collision + checks, opaque LiveRamp envelopes, consent gating, OpenRTB EID production, EC + partner ingestion, and admin diagnostics without adding a LiveRamp catalog + definition or an identity-provider capability system. - Request-processing requirements preserve DataDome origin bypass, full-body buffering, and final private caching, and prevent request-private GPT diagnostics state from entering ESI templates. +- Warm-hit and cold-store matrices combine DataDome/GPT requirements with key, + bypass, malformed, unlisted, absent, and empty cookies; integration privacy + can only restrict the existing cookie/cache decision. - The dedicated mediator capability preserves request construction, ordered response input, bounded transport, parsing, and local-ranking fallback without exposing the legacy `AuctionProvider` trait. @@ -1343,8 +1575,13 @@ to preserve `cfg(test)` imports. - Core, GPT, and Prebid artifacts contain no private copy of APS renderer state. - Existing APS validation, sandbox, messaging, timeout, and rendering tests pass through neutral dispatch. +- APS Rust serves the exported integration-owned renderer-document bytes with + unchanged CSP and response headers; no production APS browser program remains + embedded as a Rust literal. - APS remains immediate and reads its rendering mode from the synchronous unified tag; a deferred APS renderer is rejected during composition. +- GPT remains immediate and reads `data-ts-gam-attribution` from the unified + tag through `document.currentScript` at evaluation time. - Equal duplicate trusted script attributes collapse, while conflicting values fail composition before HTML is served. - Missing or rejecting renderers drop only the renderer-bearing bid with no @@ -1357,6 +1594,16 @@ to preserve `cfg(test)` imports. template reuse through processing requirements. - GPT bootstrap and APS shared fixtures resolve from their integration-owned package locations. +- External Prebid artifacts preserve bidder, User ID, and analytics category + selection, manifest/hash/SRI generation, managed-name alias and collision + checks, `identityLinkIdSystem` requirements, consent behavior, and runtime + codes after registry and shim paths move. +- Owner-specific output directories and manifests reject stale or partial + output, and concurrent neutral/integration Cargo builds exercise the one + cross-process toolchain lock. +- Cross-adapter Playwright tests remain in + `trusted-server-integration-tests` and verify core, creative, APS, GPT, and + Prebid load order using the moved assets. ### Repository gates @@ -1366,8 +1613,24 @@ cross-adapter parity tests, required native and WASM builds, JavaScript builds and Vitest suites for both browser source roots, JavaScript formatting, and documentation formatting. The explicit package lists in every Fastly build/check/clippy/test alias include both new Rust crates where applicable; -host-target tests still run catalog-completeness and integration test support. -The migrated Fastly-SDK guard continues to scan the moved integration sources. +the CLI and codegen host lint gates remain intact; and a dedicated host gate runs +catalog completeness and integration test support. The migrated Fastly-SDK +guard continues to scan the moved integration sources. + +The path migration covers repository automation as well as compiled code: +GitHub workflows and PR templates, Dependabot, `AGENTS.md`, `.claude` agents and +commands, the CLI Prebid builder, browser and template-cache smoke scripts, +TypeScript/Vitest/format/lint configuration, crate READMEs, reader-facing docs, +`trusted-server.example.toml`, and documentation-snippet tests. In particular, +the GPT bootstrap fixture path, APS Rust fixture includes, browser integration +build script, and local template-cache harness must resolve the new owners. +VitePress lint/build and `documentation_snippets` remain gates. Historical +archived specs are not rewritten as though they described the new layout. + +A repository path guard rejects active code or tooling that still points to +`trusted-server-js/lib/src/integrations` or concrete +`trusted-server-core/src/integrations/` paths, except for an explicitly +allowlisted historical reference. Dependabot remains rooted at the one lockfile. ## Risks and Mitigations @@ -1414,7 +1677,7 @@ Provider order affects launch budget, mediator input, response order, and equal price ties. Treating it as cosmetic would make operator edits surprising. Mitigation: define configuration order as operational priority, document the -change from PR #1016's lexical order, and test each observable consequence. +change from the baseline's lexical order, and test each observable consequence. ### Hidden reverse dependencies @@ -1431,18 +1694,20 @@ CLI validation and adapter startup could use different catalogs or schemas. Mitigation: both call the same catalog-backed source-validation API, and only adapter startup continues through the post-secret runtime composition API. No secondary validation inventory is allowed. Adapters receive the already -composed settings, plan, registry, and browser assets rather than reconstructing -any of them. +composed settings, plan, registry, browser assets, and configuration digest +rather than reconstructing any of them. Non-runtime CLI commands consume the +validated source view rather than deserializing integration fragments locally. ### Stale or incorrectly ordered browser artifacts Splitting Rust ownership while sharing one Node workspace can embed previous output, race build scripts, or load APS too late. -Mitigation: coordinate the one workspace's build/install lock, retain -stale-output refusal, hash built bytes, load core and the creative prelude -first, reject deferred APS composition, and run artifact-level renderer, -fingerprint, and ordering tests. +Mitigation: hold one cross-process lock across install, cleanup, build, +discovery, manifest verification, and copy; use owner-specific output +directories; retain stale-output refusal; hash built bytes; load core and the +creative prelude first; reject deferred APS composition; and run artifact-level +renderer, fingerprint, and ordering tests. ## Acceptance Criteria @@ -1454,8 +1719,9 @@ The change is complete when: `trusted-server-integrations/src//`. 3. The standard provider configuration is supplied by the built-in Rust-only `openrtb` integration, making sixteen static definitions in total. -4. All integration browser sources, assets, fixtures, and tests live under - `trusted-server-integrations-js`. +4. All integration-owned browser sources, assets, unit/artifact fixtures, and + unit/artifact tests live under `trusted-server-integrations-js`; cross-adapter + system tests remain in the integration-test crate. 5. Rust definitions use one explicit compile-checked catalog with a directory completeness test; browser modules remain directory-discovered. 6. Core owns only neutral contracts and execution engines and imports no @@ -1466,34 +1732,48 @@ The change is complete when: inventory. 9. Configuration and provider ordering survive config push and runtime loading through validated order sidecars and define the documented auction priority. -10. The old `[auction.providers]` schema is rejected with targeted migration - guidance. -11. The compiled auction plan retains PR #1016 behavior after normalization, - except for the explicit change from lexical to configuration-order provider - priority. +10. The old `[auction.providers]`, `profile`, `profile_config`, and fixed + `protocol` fields are rejected with targeted migration guidance. +11. The compiled auction plan retains current-baseline behavior after + normalization, except for the explicit change from lexical to + configuration-order provider priority. 12. Browser core imports no concrete integration, and APS rendering works through one immediate registration without private copies in core, GPT, or - Prebid bundles. -13. `TrustedServerAppConfig`, integration secret handling, and final runtime - composition are owned by `trusted-server-integrations`; core has no concrete - config or loader dependency. + Prebid bundles; GPT also retains its synchronous-tag bootstrap contract. +13. `TrustedServerAppConfig`, `ValidatedSourceConfig`, integration secret + handling, and final runtime composition are owned by + `trusted-server-integrations`; core has no concrete config or loader + dependency, and the CLI has no duplicate integration schema. 14. The CLI and adapters use the same catalog-backed source validation, and all adapters receive one post-secret-resolution - settings/plan/registry/browser-assets composition. + settings/plan/registry/browser-assets/configuration-digest composition. 15. OpenRTB request-local state crosses the transport boundary through a prepared response parser without `Any` or vendor enum variants in core. -16. Explicit `enabled`, parent-before-descendant, disabled-retention, local-ID, - and qualified-ID rules have end-to-end tests. +16. Explicit `enabled`, parent-before-descendant, disabled-retention, activation + matrix, local-ID, and qualified-ID rules have end-to-end tests, including a + server-only Prebid migration that does not activate browser behavior. 17. Schema-1 blobs remain readable for the documented rollout release, schema-2 blobs preserve existing secret paths, and binary rollback requires verified - restoration of the archived schema-1 envelope. -18. Browser assets carry bytes and hashes through composition, publisher - template fingerprints vary with every composition-time external or inline - asset change, and request-dependent head variants bypass shared reuse. + restoration of the archived schema-1 envelope through a drill-tested + adapter-specific mechanism. +18. Browser assets carry bytes and hashes through composition; publisher + template fingerprints combine the full normalized configuration digest with + the exact document-assets fingerprint; all existing URL, host, scheme, + origin, assembly, Vary, cookie, and schema-version key dimensions remain; + and request-dependent variants bypass shared reuse. 19. Core test support, the Fastly-SDK migration guard, Cargo aliases, CI, - Dependabot, browser scripts, and the CLI Prebid builder cover the new crate - boundaries. -20. The full repository verification gates pass. + Dependabot, repository automation, browser and cache smoke scripts, + documentation and snippet tests, and the CLI Prebid builder cover the new + crate boundaries, with a stale-path guard and a dedicated native + integrations-crate gate. +20. Managed Prebid User IDs and external bundle bidder/User-ID/analytics + selection retain their current alias, collision, consent, manifest, hash, + SRI, and runtime-code behavior. +21. Neutral OpenRTB-EID/EC ingestion, including opaque LiveRamp envelopes, + partner ingestion, and admin diagnostics, remains in core without creating + another integration definition or provider framework. +22. Both milestone exit criteria and the full repository verification gates + pass. ## Deferred Work @@ -1502,7 +1782,7 @@ The following require separate designs and real consumers: - External vendor-owned crates or adapter-supplied registrations. - Runtime integration loading or a stable integration SDK. - Independent integration release and compatibility policies. -- Identity, EC, geo, device, and permission-signal providers. +- New identity, EC, geo, device, and permission-signal provider systems. - Permission and jurisdiction policy changes. - Non-OpenRTB auction provider factories. - Upstream EdgeZero composition and host-service changes. From b376ef18a4c531aa74550c874e45e102d0679259 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Wed, 23 Sep 2026 22:46:09 -0700 Subject: [PATCH 09/13] Resolve red-team integrations design findings --- ...-09-17-split-integrations-crates-design.md | 423 +++++++++++++----- 1 file changed, 309 insertions(+), 114 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index 4e7899686..43725b243 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -100,10 +100,11 @@ scope for parity even though they landed after this design was first drafted. `core/src/ec/prebid_eids.rs` is historically named after the first browser producer, but its `ts-eids` ingestion, consent checks, EC finalization, partner-graph ingestion, and admin diagnostics are shared identity machinery. -They remain in core and may be renamed to an integration-neutral module during -the move. Prebid-owned configuration and browser-module management move out; -the shared EID/EC machinery does not become a new capability family, and -LiveRamp does not become another integration definition. +They remain in core under their current name for this split. Renaming the +neutral module is unrelated cleanup and is deferred. Prebid-owned configuration +and browser-module management move out; the shared EID/EC machinery does not +become a new capability family, and LiveRamp does not become another +integration definition. Configuration is also split by implementation detail. Browser/page settings use `[integrations.]`, while server auction providers use @@ -350,9 +351,10 @@ The rules are: 4. Integration TypeScript may use the explicit browser-core API, but browser core never imports a concrete integration and integration bundles never embed a private copy of stateful browser-core modules. -5. Every adapter and the CLI uses the same source-validation entry points from +5. The CLI uses the source-validation APIs from `trusted-server-integrations`; every adapter uses its single runtime - composition entry point. + composition entry point from that crate. Both phases resolve the same static + catalog and integration schemas. 6. No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or another integration module directly. @@ -368,8 +370,8 @@ directory of implementations. It owns: - Aggregation of core and integration secret metadata. - Integration-owned preprocessing for conditionally active secrets. - Catalog-aware validation and capability construction. -- Public source-validation wrappers used by the CLI before EdgeZero's typed - validate, diff, and push mechanics. +- Pure source parsing, structural validation, and catalog-validation APIs used + by the CLI before EdgeZero's typed validate, diff, and push mechanics. - The public runtime entry points that load a config-store blob and return one composed runtime value. - A validated source-config view used by non-runtime CLI commands. @@ -400,20 +402,23 @@ capability objects. That runtime value, conceptually `TrustedServerComposition`, contains the validated neutral `Settings`, one `Arc`, one `IntegrationRegistry`, the composed `BrowserDocumentAssets`, and a canonical -digest of the complete normalized resolved configuration. Adapters consume +digest of the complete composition snapshot defined below. Adapters consume this value; they do not separately compile the auction plan, rebuild the integration registry, enumerate browser bundles, or reconstruct the digest. Core retains neutral config-store access, Fastly chunk reconstruction, blob -envelope verification, secret-resolution primitives, global settings types, -and auction-plan compilation. Those helpers accept or return neutral data and -never call the concrete catalog. The integration crate calls them in this -order: +envelope verification, preprocessing for inactive neutral/global secret +references, secret-resolution primitives, global settings types, and +auction-plan compilation. Today the neutral preprocessor removes inactive +Tinybird token references and disabled EC-partner pull-token references; that +behavior remains in core. These helpers accept or return neutral data and never +call the concrete catalog. The integration crate calls them in this order: ```text config-store bytes → core chunk reconstruction and envelope verification - → integration-owned inactive-secret preprocessing + → core-owned neutral/global inactive-secret preprocessing + → catalog-owned integration inactive-secret preprocessing → aggregated core + integration secret resolution → catalog-aware config validation → core AuctionPlan compilation @@ -422,27 +427,48 @@ config-store bytes → TrustedServerComposition ``` -The CLI imports `TrustedServerAppConfig`, `ValidatedSourceConfig`, and its -config command wrappers from `trusted-server-integrations`. Each wrapper -performs the source-aware pre-pass and source-phase catalog validation before -delegating storage and diff mechanics to EdgeZero's typed CLI functions. The -validated bytes are passed to EdgeZero through an immutable temporary snapshot, -so the bytes checked by the pre-pass are exactly the bytes diffed or pushed; -the original operator path remains the path shown in diagnostics. No EdgeZero -source change or new host service is required. +The CLI imports `TrustedServerAppConfig`, `ValidatedSourceConfig`, and pure +source-validation APIs from `trusted-server-integrations`. The host-only CLI +continues to own the config command wrappers and the direct `edgezero-cli` +dependency; `trusted-server-integrations`, which is linked into every WASM +adapter, never depends on `edgezero-cli`. Each CLI wrapper performs the +source-aware pre-pass and source-phase catalog validation before delegating +storage and diff mechanics to EdgeZero's typed CLI functions. + +Because locked EdgeZero accepts paths rather than validated bytes, a wrapper +copies the operator config and manifest inputs to private mode-`0600`, immutable +temporary snapshots, rewrites the delegated arguments to those snapshots, and +removes them afterward. The manifest snapshot is created beside the original +manifest so all manifest-relative adapter paths retain the same base directory; +failure to create the secure snapshot aborts before remote I/O. Snapshot paths +in errors and the validate-success line are rewritten or replaced with the +original operator paths. The config and manifest bytes used to choose the +adapter, store, and app-config path are therefore the same bytes EdgeZero +processes; a concurrent edit cannot redirect the delegated operation. No +EdgeZero source change or new host service is required. Read-only `config ad-templates` and `audit ad-templates` commands load the effective source view with the existing optional environment overlay. Mutating or generator commands load file bytes without the overlay, so environment-only values are never persisted. Recovery-oriented ad-template generation may run a structural-only pre-pass against an otherwise invalid baseline, preserving its -current warning and non-disclosure behavior, but the final candidate must pass -the complete source validation before atomic write. `ts prebid bundle` obtains -typed bidder, User ID, analytics, and managed-module requirements through the -Prebid source-view facade; it retains process invocation and atomic metadata -patching but no longer maintains a partial duplicate Prebid schema. Provider -diagnostics display qualified providers in configuration order rather than -alphabetizing a detached map. +current warning and non-disclosure behavior. Candidate and baseline then use +the same complete integration-owned validation path: a valid candidate is +written; an invalid candidate is refused when the baseline was valid; and an +invalid candidate over an already-invalid baseline retains the current warning +and atomic-write escape hatch without disclosing source values. + +`ts prebid bundle` obtains typed bidder, User ID, analytics, and managed-module +requirements through an integration-owned partial source view rather than a +duplicate CLI schema. It intentionally does not require unrelated app +configuration or an `external_bundle_url` to be deploy-valid: `bundle.modules`, +`external_bundle_sha256`, and `external_bundle_sri` are inert staging metadata, +while `external_bundle_url` activates the browser capability. The command +retains its current ability to build first, atomically patch hash/SRI metadata, +and tell the operator to upload and set the URL. It validates the structural +pre-pass and affected Prebid subtree before writing; full app validation remains +the contract of config validate/push. Provider diagnostics display qualified +providers in configuration order rather than alphabetizing a detached map. ## Static Rust Catalog and Browser Discovery @@ -487,6 +513,14 @@ output and its SHA-256 hash. CI, browser integration scripts, and the CLI Prebid builder use the single workspace root rather than maintaining a second dependency graph; Dependabot continues to watch only its one lockfile. +Canonical npm build, typecheck, lint, format, and test commands include both +source roots explicitly, and CI invokes those commands rather than core-only +paths. Both Rust build scripts emit complete `rerun-if-changed` coverage for +their own manifest, configuration, and source inputs, including the sibling +integration root. Clean and incremental build tests change one integration +source and prove the integration manifest is regenerated without spuriously +changing the neutral manifest. + `build-prebid-external.mjs` and its npm command remain at the canonical Node root as build orchestration, not browser runtime. The Prebid registry, aliases, shims, and other integration-owned source inputs move with Prebid into @@ -645,10 +679,12 @@ configuration. Thus a current server-only Prebid provider migrates beneath an enabled Prebid parent without activating Prebid's page/browser behavior. Browser-only Prebid continues to be selected by the same required external-bundle URL that current -startup validation already uses. Managed User ID or other browser-only settings -without that URL fail validation rather than activating a partial browser path. -For APS, a configured provider necessarily activates its renderer support; an -enabled APS parent with no provider is a staged no-op. +startup validation already uses. Runtime browser settings such as managed User +IDs or client-side bidders without that URL fail validation rather than +activating a partial browser path. CLI-only `bundle.modules` and generated +hash/SRI metadata may be staged without a URL and never activate runtime +browser behavior. For APS, a configured provider necessarily activates its +renderer support; an enabled APS parent with no provider is a staged no-op. Requiring the enabled parent is an intentional activation change from the current split inventory. Today an APS auction profile can activate rendering @@ -810,11 +846,11 @@ The contract is: relative order. 6. The flattened auction plan orders providers first by owning integration and then by local provider declaration. -7. Hook and immediate/deferred JavaScript lists retain integration order. When - one integration registers multiple hooks of the same capability, their - relative order is the explicit order returned by that integration's - definition; the integration remains one operator-visible position and does - not expose a second priority mechanism. +7. Every hook list and each immediate/deferred JavaScript list retains + integration order. When one integration registers multiple hooks of the same + capability, their relative order is the explicit order returned by that + integration's definition; the integration remains one operator-visible + position and does not expose a second priority mechanism. 8. Browser output is neutral browser core first, the existing fixed JavaScript-only `creative` prelude second, and configured integration modules afterward. @@ -825,6 +861,10 @@ The contract is: each back-to-back launch; configuration order becomes budget-observable only if the deadline expires or adapter timeout canonicalization reaches zero during that launch loop. +10. Shared browser dispatchers execute handlers by the owning integration's + configured ordinal and then definition-local registration order. Wall-clock + registration timing, numeric priority, and lexical handler ID are not + alternate ordering mechanisms. Ordered runtime introspection carries the configured ordinal with each integration and provider. Lookup indexes may use maps, but iterating a @@ -839,6 +879,15 @@ integration order as every other hook. Migration guidance places is preserved by default, while an operator may deliberately choose a different order. No engine-only priority is hidden from the configuration. +Browser load mode remains a lifecycle phase, not a second operator priority: +immediate code necessarily evaluates before deferred code. Config order is +preserved within each phase and remains the stored ordinal used by any shared +dispatcher after a module registers. A hook that must arbitrate during initial +document mutation, including a DOM-insertion guard, is immediate-only; +composition rejects it on a deferred asset. Diagnostics display each module's +fixed load mode so this phase boundary is visible rather than inferred from +timing. + All recovery paths obey the same provider ordinals. In particular, the two current transport-failure branches that sort provider IDs lexically are replaced with plan-order recovery before the new ordering contract is enabled. @@ -860,25 +909,34 @@ then permits the existing EdgeZero scalar environment overlay; overlays may replace values but may not create, remove, or reorder integration or provider tables. +The direct `toml_edit` dependency is pinned to the same TOML 1.1 parser +generation as the workspace `toml` package and EdgeZero's typed parser. Parser +parity fixtures cover otherwise-unrelated valid and invalid TOML syntax so the +pre-pass cannot accept a document the typed path rejects, or reject one merely +because it used an older TOML grammar. + Every entry point that accepts TOML uses this pre-pass, including local loading, CLI validate/diff/push, ad-template diagnostics and candidate validation, and the Prebid bundle command. Recovery mutators may request the structural-only -mode described above, but final candidate validation always uses the complete -mode. EdgeZero's typed mechanics remain responsible for overlay, validation -invocation, diff, envelope construction, consent, and store writes after the -pre-pass succeeds. - -EdgeZero does not currently expose a pre-parse hook. The Trusted Server wrappers +mode described above. Ad-template generation applies the comparative +candidate/baseline rule, and the Prebid builder validates only its owned partial +view; neither recovery path is silently tightened into unconditional full-app +validation. EdgeZero's typed mechanics remain responsible for overlay, +validation invocation, diff, envelope construction, consent, and store writes +after the pre-pass succeeds. + +EdgeZero does not currently expose a pre-parse hook. The host-only CLI wrappers therefore duplicate its app-config path rule: an explicit `--app-config` wins; otherwise the path is `/.toml`. A wrapper reads the -source once, performs the pre-pass, and delegates typed processing against a -private immutable snapshot of those exact bytes; it does not validate one read -and allow EdgeZero to reopen a concurrently changed operator file. Parity tests -cover exact-byte delegation, explicit and default paths, manifest paths with and -without parent directories, and `--no-env`. The environment overlay can replace -only scalar leaves already present in TOML; it cannot create an omitted -`enabled` field, integration, provider, table, or array. Operator templates must -contain every leaf intended for overlay. +manifest and source once, performs the pre-pass, and delegates typed processing +against private immutable snapshots of those exact bytes; it does not validate +one read and allow EdgeZero to reopen a concurrently changed operator or +manifest file. Parity tests cover exact-byte delegation, diagnostic path +rewriting, explicit and default paths, manifest paths with and without parent +directories, and `--no-env`. The environment overlay can replace only scalar +leaves already present in TOML; it cannot create an omitted `enabled` field, +integration, provider, table, or array. Operator templates must contain every +leaf intended for overlay. ### Config-store representation @@ -966,7 +1024,10 @@ including its provider instances. validation, secret metadata, pre-resolution handling for conditionally active secrets, and capability construction to both runtime startup and the CLI. Core exposes its non-integration secret metadata through a neutral helper; the -composition root combines it with catalog metadata. +composition root combines it with catalog metadata. Core also retains its +neutral/global inactive-secret preprocessor for Tinybird and EC partners; the +composition root runs both core and catalog preprocessors before one shared +resolution pass. For example, DataDome's inactive secret references are filtered by its definition before the shared secret resolver runs; core's config-payload code @@ -1064,8 +1125,12 @@ Rollout is ordered: config-store read/export facility and is an explicit release artifact. 2. Deploy the dual-reader binary while the schema-1 blob remains active. 3. Complete health checks on every deployed instance. -4. Push the migrated schema-2 configuration with the new CLI. -5. Verify registry order, provider order, browser asset hashes, and auction +4. Freeze configuration writes and fence old CLI artifacts from the deployment + credentials or release path; only the schema-2 CLI may write after this + point. +5. Push the migrated schema-2 configuration with the new CLI and read back or + otherwise assert `trusted_server_schema = 2` from the stored envelope. +6. Verify registry order, provider order, browser asset hashes, and auction health before declaring the cutover complete. The rollout runbook must name and drill a concrete export and restore mechanism @@ -1075,13 +1140,19 @@ remote Spin deployment path cannot read deployed config through the locked EdgeZero CLI; it must use a verified platform/control-plane export and restore facility or the schema-2 rollout for that target is blocked. -An old binary must never serve a schema-2 blob. Rolling back after step 4 first +An old binary must never serve a schema-2 blob. Rolling back after step 5 first restores the archived schema-1 envelope, verifies that restoration, and only then rolls the binary back. If the platform cannot coordinate those operations, the release is paused rather than accepting an outage window. The compatibility decoder is removed only in a later release after every supported deployment has completed the schema-2 cutover. +The write fence remains until old CLI credentials/artifacts can no longer push. +Because the dual reader intentionally accepts schema 1, schema-1 reappearance +after cutover is an explicit rollback event, never a tolerated ordinary write; +release monitoring alerts on it and the runbook requires either immediate +schema-2 restoration with the new CLI or the complete binary-rollback sequence. + ## Required Neutral Lifecycle Boundaries ### Request processing and sharing annotations @@ -1138,27 +1209,55 @@ a new lifecycle call site. ## Browser Composition and APS Renderer -`trusted-server-js` builds only the neutral core IIFE. Its browser API exposes -the existing shared facilities integrations actually use: logging, slot lookup -and render helpers, normalized auction-response access, first-impression state, -and renderer registration and dispatch. Integration bundles import only -type-only declarations. Runtime calls go through a stateless accessor mapped by -the integration build to the already-installed Trusted Server browser -namespace; an integration IIFE does not rely on an unresolved ESM import or -bundle a second state owner. +`trusted-server-js` builds only the neutral core IIFE. Before extraction, the +implementation audits every production value import from an integration or the +fixed `creative` prelude into today's `core/` and `shared/` trees. Each import +is classified rather than copied blindly: + +- Stateful facilities remain owned once by browser core and are exposed through + one versioned `TrustedServerBrowserRuntime` namespace. The initial surface + includes logging, context-provider registration and context collection, + auction request construction and normalized response parsing, queue + installation, slot lookup and rendering helpers, first-impression state, DOM + insertion-handler registration, and renderer registration/dispatch/lifecycle + operations. +- Pure stateless helpers may move to an integration-owned shared source module + and be bundled into the consuming IIFE. They must not close over or initialize + browser-core state. +- Types are imported from declaration-only entry points. There are no runtime + relative imports across the two crate source roots. + +Runtime calls use a stateless accessor mapped by the integration build to the +already-installed namespace. The integration build fails on any undeclared +cross-root value import. This covers current imports such as Permutive context +registration, Prebid auction helpers, Testlight queue installation, GPT slot +resolution, and the shared script/beacon guards; it is not limited to the APS +renderer examples. The core IIFE initializes exactly one stateful registration object on the Trusted Server browser namespace before any integration IIFE runs. Integration bundles consume that object through an external runtime shim and type-only browser-core declarations; their bundler must not inline the stateful registry -implementation. Artifact tests prove that a renderer registered by an -integration IIFE is visible to the already-loaded core IIFE. +implementation. Artifact tests prove that state registered by an integration +IIFE is visible to the already-loaded core IIFE, including a Permutive context +provider observed by core collection and a renderer observed by GPT and Prebid. `trusted-server-integrations-js` builds integration IIFEs. Immediate modules are concatenated in the ordering contract above. Deferred and standalone modules remain separate assets but retain their configuration-relative order and typed identities. +The browser DOM-insertion dispatcher follows the same simple ordering rule as +Rust hooks. Its current numeric priority and ID-lexical sort are removed. +Composition installs the immutable integration ID-to-ordinal mapping before any +IIFE executes. A handler registers under its owning integration ID and executes +by configured ordinal, then by that integration's local registration sequence. +Handler IDs remain diagnostic identities only, and DOM-insertion handlers are +immediate-only so an absent deferred handler cannot observe mutations too late. +Reversing two configured integrations therefore reverses the winner when their +script guards both claim the same candidate; no hidden browser priority or +bundle timing can override TOML order. + The Rust registration does not carry only a string module ID. Core owns a neutral immutable `BrowserAsset` contract containing the typed module ID, embedded bytes, SHA-256 hash, load mode, and trusted script-tag attributes. @@ -1188,13 +1287,24 @@ never folded into a composition-wide fingerprint. Publisher template invalidation is broader than the browser document. During composition, `trusted-server-integrations` hashes a canonical serialization of -the complete normalized resolved configuration: all neutral `Settings` and the -full configuration of every integration in configuration order, including -disabled retained entries. Hashing the complete model deliberately -over-invalidates so a future rewriter, postprocessor, proxy mapping, cookie -policy, or other HTML-shaping field cannot be omitted from a hand-maintained -allowlist. The serialized bytes and resolved secret values are fed directly to -the digest and are never logged, returned, or used as cache-key plaintext. +the complete composition snapshot: resolved neutral `Settings`, resolved +enabled-integration configuration, and structurally validated retained source +configuration for disabled integrations, all in configured integration and +provider order. Active secret values enter the hash after resolution; inactive +secret references remain unresolved retained source values. Hashing the +complete model deliberately over-invalidates so a future rewriter, +postprocessor, proxy mapping, cookie policy, or other HTML-shaping field cannot +be omitted from a hand-maintained allowlist. Input bytes and resolved secret +values are fed directly to the digest and are never logged, returned, or used +as cache-key plaintext. + +“Canonical” is a versioned encoding contract, not ordinary `Serialize` output. +It preserves the explicitly ordered integration/provider sequences, sorts keys +of semantically unordered maps and elements of semantically unordered sets, +uses length-delimited domain-separated fields, and has golden vectors. Tests +build equivalent `HashMap`/`HashSet` values in different insertion orders and +require the same digest, while reversing an operator-ordered sequence must +change it. Core computes the existing template fingerprint from that configuration digest and `BrowserDocumentAssets.document_fingerprint`. This composite replaces only @@ -1207,15 +1317,32 @@ non-head integration rewriter settings, external and inline assets, and cookie policy changes invalidate templates without exposing raw configuration. Browser core currently imports APS renderer logic directly. Replace that -reverse dependency with one neutral renderer registration mechanism: +reverse dependency with one neutral renderer registration mechanism. A +renderer-type handler owns descriptor validation, render dispatch, any +renderer-specific bounded capability state, and the renderer metadata needed +to construct a Prebid Universal Creative response. Browser core owns only the +type-keyed handler registry and neutral calls into it: - Core parses the existing renderer envelope far enough to identify its type - and retain its payload. -- The APS browser module registers validation and dispatch for the existing APS - renderer type. -- Core dispatches through the registered renderer. -- GPT and Prebid call the neutral dispatcher and never import APS source. -- No APS source is inlined into the core, GPT, or Prebid IIFE. + and retain its opaque payload. +- The APS IIFE registers the one APS handler. GPT and Prebid call the neutral + registry and never import APS source; no APS source or private APS state is + inlined into core, GPT, or Prebid. +- Prebid validates a descriptor before bid admission, carries it only until + Prebid assigns an `adId`, and scrubs the carrier from both the normalized bid + and metadata. On `bidAccepted`, with the current `bidResponse` fallback, it + registers a bounded-TTL capability containing the `adId`, ad-unit binding, + validated descriptor, and `markWinningBidAsUsed` callback. Registration + failure demotes the bid to the current negative-CPM failure state. +- GPT authenticates the requesting iframe against the expected slot or ad unit + before it consumes a capability. Consumption is compare-and-consume atomic, + source-bound, TTL-bounded, and replay-safe. Only the selected renderer handler + supplies its renderer source, version, URL, payload, and dimensions for the + Universal Creative response; GPT does not know APS constants. +- Server-originated renderer descriptors use the same registered validation and + dispatch handler while retaining the current slot-scoped replay protection. + The Prebid `adId` path and server-bid path remain distinct neutral entry + points so one cannot consume the other's authority accidentally. An enabled APS integration whose provider can emit APS renderer descriptors includes its immediate APS browser module. The core IIFE and fixed creative @@ -1238,6 +1365,10 @@ native-Prebid fallback; unrelated bids and the page continue. A duplicate type fails Rust composition, while a defensive browser-side duplicate poisons that type rather than using last-registration-wins. No renderer route, DOM, message, or beacon side effect occurs before the selected handler accepts the payload. +Carrier scrubbing, failed admission, bounded capacity and TTL, source binding, +atomic consumption, replay rejection, and `markWinningBidAsUsed` are part of +the compatibility contract rather than APS-private implementation details that +may disappear during extraction. The serialized descriptor, validation, sandbox flags, message authentication, timeouts, and render results do not change. @@ -1248,8 +1379,8 @@ JavaScript, moves from the APS Rust source into The browser crate exports its immutable bytes and hash; APS Rust serves those exact bytes with the existing content type, CSP, and other response headers. The document's nonce binding, sandbox, message authentication, runner load, and -failure tests move with the asset. No production browser program remains as a -Rust string literal. +failure tests move with the asset. No production APS renderer program remains +as a Rust string literal. `gpt_bootstrap.js` moves with GPT into `trusted-server-integrations-js` and is exported as a hashed integration-owned inline asset. GPT's Rust head injector @@ -1258,6 +1389,19 @@ asset through the owning package instead of a cross-crate relative path. APS golden renderer fixtures likewise move to an integration-owned shared fixture location used by both Rust and browser tests. +The move includes a repository-wide inventory of production executable browser +programs assembled by integration Rust, not only files that already end in +`.js`. The current DataDome, Didomi, GPT, Prebid, and Sourcepoint config +initializers; Sourcepoint `_sp_` property trap; and GPT-diagnostics +activation/history bootstrap all have integration-owned static program bodies +or typed templates in `trusted-server-integrations-js`. Integration Rust may +serialize safe data, invoke the generated typed template renderer, and assemble +script tags; it does not retain handwritten browser algorithms in Rust string +literals. Exact rendered inline bytes and hashes participate in the document +fingerprint. A source/artifact guard fails when a new production executable +integration script is introduced directly in Rust without an explicitly +reviewed data-only exception. + ## Error Handling Failures occur as early as the available information allows, with policy @@ -1320,6 +1464,9 @@ The intentional compatibility breaks are: source rather than copied into each integration-owned provider. - Every retained integration parent requires an explicit `enabled` value, including parents whose current schema supplies a default. +- Disabled retained blocks must deserialize against the new structural schema: + unknown fields, wrong types, malformed IDs, and invalid retained secret + references now fail even though active-only value checks remain deferred. - Integration and provider parents must use ordinary table headers in parent-before-descendant order; dotted-key or inline-table parent shorthand is rejected. @@ -1364,7 +1511,10 @@ current operator schema and provider activation semantics through a temporary normalization adapter. It delivers the crate boundary without an operator cutover. The second milestone contains workstream 4, introduces stored schema 2 and the rollout decoder, and atomically replaces the temporary source -normalizer with the new source parser. +normalizer with the new source parser. Neutral contracts from workstream 1 may +land as behavior-inert preparatory commits, but milestone 1 is not complete or +deployable as the new architecture until workstreams 1 through 3 all meet its +exit criteria. Intermediate commits may add unused neutral contracts or new crates, but no merged state may have two active catalogs, two simultaneously interpreted provider inventories, or adapter-specific composition paths. This scope does @@ -1401,11 +1551,13 @@ Milestone exit criteria are independent: Implementation may use small commits, but the merged workspace must never have two active integration or provider inventories. -Steps 3 through 5 form the first atomic merge milestone. Preparatory commits may -compile and parity-test copied code in an unused new crate while the old catalog -remains authoritative, but the final milestone switch rewires every consumer -and deletes the old concrete sources together. No deployable revision selects -some integrations or browser assets from each catalog. +Steps 1 through 5 comprise milestone 1. Steps 1 and 2 are independently +mergeable, behavior-inert preparation; within that milestone, steps 3 through 5 +form the atomic ownership cutover. Preparatory commits may compile and +parity-test copied code in an unused new crate while the old catalog remains +authoritative, but the final milestone switch rewires every consumer and +deletes the old concrete sources together. No deployable revision selects some +integrations or browser assets from each catalog. 1. Add neutral capability, processing-requirement, browser-asset, OpenRTB profile/exchange, and mediator contracts to core. Extend the `test-utils` @@ -1415,19 +1567,22 @@ some integrations or browser assets from each catalog. diagnostics and DataDome call sites to the neutral lifecycle contracts. 3. Create `trusted-server-integrations-js`, move all integration-owned browser sources, unit/artifact tests, GPT bootstrap, the APS renderer document, - registry inputs, and shared fixtures, remove every browser-core APS import, - and make the composed asset set authoritative for bytes and hashes. Retain - cross-adapter system tests in `trusted-server-integration-tests` and update - their paths and load order. + every integration-owned executable inline template, registry inputs, and + shared fixtures; complete the cross-root import audit; remove every + browser-core APS import; retire DOM-handler priority sorting; and make the + composed asset set authoritative for bytes and hashes. Retain cross-adapter + system tests in `trusted-server-integration-tests` and update their paths and + load order. 4. Create `trusted-server-integrations` with the explicit static catalog. Move ordinary integrations first, then move DataDome and GPT diagnostics after their lifecycle seams, APS and Prebid after the profile seam, and `adserver_mock` after the mediator seam. Add the new `openrtb` adapter. 5. Move `TrustedServerAppConfig`, all integration-specific configuration, validation, inactive-secret preprocessing, and secret metadata into the - integrations crate. Rewire the CLI to the source-validation entry point and - all adapters to the single runtime composition entry point while retaining - the existing operator schema through the temporary normalizer. This is the + integrations crate. Rewire the CLI to the pure source-validation entry point + while keeping host-only EdgeZero command wrappers in the CLI, and rewire all + adapters to the single runtime composition entry point while retaining the + existing operator schema through the temporary normalizer. This is the behavior-preserving crate-split milestone. 6. Add the TOML source pre-pass, ordered in-memory maps, application schema 2, object-shaped storage with explicit order sidecars, strong qualified @@ -1467,6 +1622,9 @@ to preserve `cfg(test)` imports. - A typed Rust reference to an absent browser module fails compilation. - Embedded bundle hashes match built bytes. - Core has no dependency on either integrations crate. +- `trusted-server-integrations` has no `edgezero-cli` dependency and compiles in + every native and WASM adapter graph; host-only path/snapshot/delegation code + remains in `trusted-server-cli`. - Browser core imports no integration source. - Adapters and CLI import no concrete integration module. - A dedicated native test/clippy gate executes the integrations catalog and @@ -1476,6 +1634,8 @@ to preserve `cfg(test)` imports. ### Configuration and ordering tests - TOML parent-table order becomes the integration-owned source-model order. +- The `toml_edit` pre-pass and typed `toml`/EdgeZero parser use the same TOML + language generation and agree on parity fixtures outside `[integrations]`. - Nested provider declaration order is retained. - A parent integration or provider table declared after one of its descendants fails before typed deserialization. @@ -1493,13 +1653,19 @@ to preserve `cfg(test)` imports. key names; the post-resolution runtime phase rejects unresolved values. - Read-only CLI consumers see the effective overlay through `ValidatedSourceConfig`; mutators and generators operate on file-only bytes, - retain invalid-baseline recovery where currently supported, and fully - validate the final candidate without persisting overlay values. -- Validate/diff/push delegate the exact immutable source snapshot that passed - the pre-pass; a concurrent edit cannot substitute different pushed bytes. + retain invalid-baseline recovery where currently supported, apply the same + complete validation to baseline and candidate, and never persist overlay + values. Invalid candidate plus valid baseline refuses; two invalid values + retain the current non-disclosing warning and write escape hatch. +- Validate/diff/push delegate exact private snapshots of both the config and + manifest bytes used by the pre-pass; concurrent edits cannot substitute + different pushed bytes or change the adapter/store target. Snapshot paths are + absent from user-facing success and error output. - Prebid bundle selection and managed-module requirements come from the - integration-owned source view, not a CLI-local partial schema or hard-coded - registry path. + integration-owned partial source view, not a CLI-local schema or hard-coded + registry path. Bundle modules and generated hash/SRI metadata can be staged + without an external URL, remain runtime-inert, and are patched atomically + over an otherwise-invalid unrelated baseline. - Disabled integrations may retain structurally valid provider settings, contribute no providers or capabilities, and do not reorder enabled neighbors. - Disabled placeholder values that current examples rely on, including the @@ -1553,6 +1719,8 @@ to preserve `cfg(test)` imports. - Active DataDome secrets are presence-checked and resolved through unchanged object paths; inactive protection and bypass secrets are removed before the shared resolver. +- Core-owned preprocessing continues to remove inactive Tinybird token and + disabled EC-partner pull-token references before the shared resolver. - Neutral `ts-eids` ingestion retains managed User ID aliases and collision checks, opaque LiveRamp envelopes, consent gating, OpenRTB EID production, EC partner ingestion, and admin diagnostics without adding a LiveRamp catalog @@ -1571,6 +1739,13 @@ to preserve `cfg(test)` imports. - Output order is core, creative prelude, and configured integrations. - Immediate and deferred lists preserve configuration-relative order. +- Artifact/import-graph checks reject undeclared cross-root value imports and + duplicate state-owner signatures. Permutive registration through its IIFE is + visible to core context collection; Prebid auction helpers and Testlight + queue behavior still use the single installed runtime. +- Colliding DOM-insertion handlers run in configured order, and reversing two + integration tables reverses their winner. Numeric priority and lexical + handler ID cannot affect the result; a deferred DOM handler fails composition. - APS is absent from browser core and registers its renderer from its own IIFE. - Core, GPT, and Prebid artifacts contain no private copy of APS renderer state. - Existing APS validation, sandbox, messaging, timeout, and rendering tests pass @@ -1587,13 +1762,23 @@ to preserve `cfg(test)` imports. - Missing or rejecting renderers drop only the renderer-bearing bid with no fallback or pre-acceptance side effect; duplicate registration poisons the type or fails composition. +- Both server-bid and Prebid-`adId` renderer paths cover carrier scrubbing, + failed admission/registration, bounded TTL and capacity, authenticated source + and slot binding, atomic consume, replay rejection, renderer-owned Universal + Creative response metadata, and `markWinningBidAsUsed` preservation. - Unified, deferred, standalone, and inline assets expose bytes and hashes that match the emitted document fingerprint and static responses. +- Canonical composition-digest golden vectors are stable across construction + order for unordered maps/sets, change when configured integration/provider + order changes, and distinguish active resolved secrets from retained inactive + references without exposing either input. - Changing any integration setting that affects generated head output changes the document fingerprint; request-dependent head variation bypasses shared template reuse through processing requirements. -- GPT bootstrap and APS shared fixtures resolve from their integration-owned - package locations. +- GPT bootstrap, APS renderer document, Sourcepoint trap, GPT-diagnostics + bootstrap, and DataDome/Didomi/GPT/Prebid/Sourcepoint inline templates resolve + from their integration-owned package locations. A guard rejects handwritten + production integration browser algorithms in Rust string literals. - External Prebid artifacts preserve bidder, User ID, and analytics category selection, manifest/hash/SRI generation, managed-name alias and collision checks, `identityLinkIdSystem` requirements, consent behavior, and runtime @@ -1601,6 +1786,9 @@ to preserve `cfg(test)` imports. - Owner-specific output directories and manifests reject stale or partial output, and concurrent neutral/integration Cargo builds exercise the one cross-process toolchain lock. +- Clean and incremental Cargo builds prove that changing a sibling integration + source reruns the integration embed build and changes its manifest/hash while + leaving an unrelated neutral artifact unchanged. - Cross-adapter Playwright tests remain in `trusted-server-integration-tests` and verify core, creative, APS, GPT, and Prebid load order using the moved assets. @@ -1668,8 +1856,9 @@ An old binary cannot consume application schema 2, and a binary rollback after config cutover would otherwise fail at startup. Mitigation: deploy the dual reader before pushing schema 2, archive the prior -envelope, require restoration before binary rollback, and remove the legacy -reader only in a later release. +envelope, fence schema-1 CLI writers during and after cutover, assert the stored +schema after the push, require restoration before binary rollback, and remove +the legacy reader only in a later release. ### Configuration order silently changes auction priority @@ -1720,8 +1909,9 @@ The change is complete when: 3. The standard provider configuration is supplied by the built-in Rust-only `openrtb` integration, making sixteen static definitions in total. 4. All integration-owned browser sources, assets, unit/artifact fixtures, and - unit/artifact tests live under `trusted-server-integrations-js`; cross-adapter - system tests remain in the integration-test crate. + unit/artifact tests, including production executable inline templates, live + under `trusted-server-integrations-js`; cross-adapter system tests remain in + the integration-test crate. 5. Rust definitions use one explicit compile-checked catalog with a directory completeness test; browser modules remain directory-discovered. 6. Core owns only neutral contracts and execution engines and imports no @@ -1739,13 +1929,17 @@ The change is complete when: configuration-order provider priority. 12. Browser core imports no concrete integration, and APS rendering works through one immediate registration without private copies in core, GPT, or - Prebid bundles; GPT also retains its synchronous-tag bootstrap contract. + Prebid bundles; the complete shared-state facade and DOM dispatcher obey + configuration order; and GPT retains its synchronous-tag bootstrap + contract. 13. `TrustedServerAppConfig`, `ValidatedSourceConfig`, integration secret handling, and final runtime composition are owned by `trusted-server-integrations`; core has no concrete config or loader - dependency, and the CLI has no duplicate integration schema. -14. The CLI and adapters use the same catalog-backed source validation, and all - adapters receive one post-secret-resolution + dependency, the CLI has no duplicate integration schema, and host-only + EdgeZero wrappers remain in the CLI rather than the WASM-linked crate. +14. The CLI source phase and adapter runtime phase use the same catalog-backed + schemas and pure validation definitions, and all adapters receive one + post-secret-resolution settings/plan/registry/browser-assets/configuration-digest composition. 15. OpenRTB request-local state crosses the transport boundary through a prepared response parser without `Any` or vendor enum variants in core. @@ -1755,10 +1949,11 @@ The change is complete when: 17. Schema-1 blobs remain readable for the documented rollout release, schema-2 blobs preserve existing secret paths, and binary rollback requires verified restoration of the archived schema-1 envelope through a drill-tested - adapter-specific mechanism. + adapter-specific mechanism; old schema-1 CLI writers are fenced after + cutover and stored-schema regression is monitored as a rollback event. 18. Browser assets carry bytes and hashes through composition; publisher - template fingerprints combine the full normalized configuration digest with - the exact document-assets fingerprint; all existing URL, host, scheme, + template fingerprints combine the versioned canonical composition digest + with the exact document-assets fingerprint; all existing URL, host, scheme, origin, assembly, Vary, cookie, and schema-version key dimensions remain; and request-dependent variants bypass shared reuse. 19. Core test support, the Fastly-SDK migration guard, Cargo aliases, CI, From 1a9253d82e8cf8dfd2efcdcdd307d95b7e206ddc Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 24 Sep 2026 01:01:34 -0700 Subject: [PATCH 10/13] Address review feedback in integrations split design --- ...-09-17-split-integrations-crates-design.md | 1563 ++++++++++++----- 1 file changed, 1089 insertions(+), 474 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index 43725b243..38b95b0d9 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -37,29 +37,33 @@ auction provider instances. Global `[auction]` settings will continue to own cross-integration orchestration such as the auction timeout, bidder routing, creative policy, and mediator selection. -Cross-integration runtime order will come exclusively from TOML declaration -order. The config push and config-store representation will preserve that order -explicitly; filesystem discovery order will never affect execution. For -auction providers, that order is also operational priority: it controls launch -and response order, mediator input order, and local equal-price tie-breaking. +Cross-integration runtime order within the same comparable capability phase +will come exclusively from TOML declaration order. Fixed engine seams such as +immediate versus deferred loading and the diagnostics post-unified position are +named phases, not hidden integration priorities. The config push and +config-store representation will preserve order explicitly; filesystem +discovery order will never affect execution. For auction providers, that order +is also operational priority: it controls launch and response order, mediator +input order, and local equal-price tie-breaking. The compatibility baseline is the behavior shipped on `origin/main` at -`4c6d26a16`, not either prior pull request discussed below. This design changes +`a4e01eb55`, not either prior pull request discussed below. This design changes only the configuration, activation, and ordering behavior called out explicitly in this document; all other current runtime, browser, CLI, and cache behavior is preserved. This remains one design, but it has two merge milestones. The crate and runtime boundary moves first without changing operator configuration. The ordered, -integration-owned configuration cuts over only after the behavior-preserving -boundary is running. The milestones share one target architecture without +integration-owned configuration cuts over only after the compatibility-focused +boundary, including its explicitly listed baseline bug fixes, is running. The +milestones share one target architecture without forcing the packaging move and configuration migration into one deployment. ## Context -On `origin/main` at `4c6d26a16`, neutral registry machinery and concrete integrations -share `crates/trusted-server-core/src/integrations`. The concrete Rust units -are: +On `origin/main` at `a4e01eb55`, neutral registry machinery and concrete +integrations share `crates/trusted-server-core/src/integrations`. The concrete +Rust units are: - `adserver_mock` - `aps` @@ -90,7 +94,8 @@ embeds bundles and hashes into the `trusted-server-js` Rust crate. APS renderer code is imported directly by browser core even though APS does not currently have its own `index.ts`. -The same baseline includes managed Prebid User IDs and their OpenRTB EID/EC +The same baseline includes the parser-aware streaming Next.js processor from PR +#1135, managed Prebid User IDs and their OpenRTB EID/EC flow, LiveRamp configuration through that existing managed-ID facility, analytics-adapter selection in external Prebid bundles, cookie-keyed publisher template caching, additional CLI ad-template and audit config consumers, and @@ -125,7 +130,7 @@ These conditions produce five related problems: 5. Runtime integration order is not a stable property of the operator configuration. -## Historical Pull Requests (Non-Normative) +## Related Pull Requests and Disposition (Non-Normative) The following pull requests explain how some current code arrived in the repository. They are not design authorities for this specification. The @@ -166,6 +171,130 @@ needed by current implementations, one ordered `[integrations]` inventory, explicit `enabled`, and APS as an integration. No `[integration]`, `[demand]`, or `[adserver]` selector convention is carried forward. +PRs #1043 through #1047 and #1094 implement parts of that alternate convention. +They cannot merge concurrently with this configuration contract. Their tests or +neutral wire-format work may be reused after independent verification, but their +inventory, discriminator, provider-naming, and crate-layout decisions are +superseded for in-tree integrations by this specification. That disposition is +coordination, not evidence for the architecture chosen here. + +PR #1135 is part of the current baseline. Its parser-aware streaming Next.js +processor, test support, and cross-adapter parity case move with the integration; +the removed HTML post-processor is not recreated by this work. + +Before either milestone branches for implementation, its baseline commit and +locked EdgeZero revision are recorded again and every baseline-dependent +inventory in this document is rechecked. Open or previously approved pull +requests never override a decision in this specification merely because of +their review status. + +## Rejected Alternatives and Rationale + +The following alternatives were considered and are deliberately not part of +the target design. Each rejection is narrow: where review exposed a valid +failure mode, the concern is accepted even when the proposed remedy is not. +The objection and the replacement decision are stated separately so an +implementation cannot quietly reintroduce the rejected mechanism. + +- **One crate per integration or an external plugin ABI.** Concrete ownership + boundaries and independent testability are required; that concern is + accepted. **Objection:** per-vendor crates, dynamic discovery, or a public SDK + would multiply dependency, compatibility, release, and governance surfaces + without a current external consumer. **Decision:** use one statically linked + Rust integrations crate, one integration-browser crate, per-integration + directories, and one compile-checked catalog. +- **A separate top-level auction-provider inventory, including APS.** APS does + provide auction behavior, but it also owns browser, renderer, route, and page + behavior. **Objection:** classifying APS as only a provider would split one + implementation's activation and ownership across unrelated top-level + sections and would make `[integrations]` incomplete. **Decision:** APS is an + integration that registers several capabilities. Its provider instances live + below `[integrations.aps.auction.providers.]`, as do provider instances + owned by other integrations. +- **A config `type`, `kind`, or `implementation` discriminator.** Multiple + configured instances must be able to reuse one implementation; that concern + is accepted. **Objection:** an extra discriminator would duplicate the static + integration ID already present in the TOML path, admit contradictory ID/type + combinations, and expose internal registration types to operators. + **Decision:** `[integrations.]` selects the statically cataloged + integration, while nested instance names identify reusable provider + configurations. Capability types remain Rust contracts, not configuration. +- **An operator-written `[auction] provider_order` list.** Provider priority + must represent schema-1's globally interleaved lexical order; that concern is + accepted. **Objection:** a second operator list would duplicate every provider + identity, permit the inventory and priority to drift, and separate priority + from the configuration block an operator is reviewing. It would make the + promised single integrations configuration untrue. **Decision:** schema 2 + derives priority from declaration order in the one `[integrations]` + inventory. The neutral compatibility model carries a flat legacy sequence so + schema 1 remains exact; that internal sequence is not a second operator + syntax. +- **A hidden engine override that always runs `js_asset_proxy` first.** Attribute + overlap and terminal-removal behavior must be explicit and tested; that + concern is accepted. **Objection:** a hidden first phase would make the TOML + order contract false and still would not reproduce baseline Prebid removal + behavior, because Prebid precedes `js_asset_proxy` today. **Decision:** legacy + compatibility paths reproduce the complete baseline sequence. Schema 2 + exposes chained replacement and terminal removal in declaration order and + diagnoses known overlaps rather than silently overriding the operator's + order. +- **Treating PR #1016, PR #1084, or earlier review statements as design + authority.** Their code and tests can reveal compatibility constraints, and + conflicting in-flight work needs an explicit disposition. **Objection:** + review or merge status does not make a prior proposal correct for this design, + and importing its architecture would silently expand this spec's scope. + **Decision:** current behavior is evidence and prior proposals are context. + This specification records its own ownership, ordering, activation, and + rollout decisions and records the disposition of incompatible work above. +- **Requiring the Node package and lockfile to move to a common ancestor.** One + Node project must reliably resolve, type-check, lint, format, test, and build + both source roots; that concern is accepted. **Objection:** moving the package + root is not required to meet that contract and would add unrelated + repository-wide path and automation churn. **Decision:** keep one canonical + project with explicit, tested resolver and tool-root configuration for the + sibling sources. Moving the root remains a fallback only if that contract + cannot be made reliable. +- **A new public configuration-status endpoint.** Runtime settings must be + loaded and the expected schema must be observable during rollout; that + concern is accepted. **Objection:** a new endpoint would add an authentication + and public-API surface unrelated to the crate split. + **Decision:** use adapter-native version/binding inspection, startup + schema-and-digest logging, and an existing authenticated or + settings-dependent probe. +- **Filesystem snapshots around EdgeZero config commands.** Config commands must + validate and serialize the same app-config bytes; that concern is accepted. + **Objection:** same-directory manifest copies add write + requirements, can survive process termination, expose operator configuration, + and require log/path rewriting without freezing every adapter manifest and + store target. **Decision:** a narrow two-stage EdgeZero typed-config extension + gives the app the exact source bytes and the effective typed command context, + providing the required consistency without filesystem snapshots. +- **Hashing resolved secret values into template identity.** Current integration + configuration changes that alter document bytes must invalidate templates; + that concern is accepted. **Objection:** current integration secrets authorize + upstream calls and do not form HTML variants, so hashing their values adds + rotation churn and sensitive derived material without improving correctness. + **Decision:** the verified stored-data hash covers secret references. A future + secret that shapes output must declare a non-secret behavior fingerprint or + force private output. +- **A second generic per-capability enablement system.** A simple master kill + switch and precise optional-feature activation are both required; that + concern is accepted. **Objection:** operator-visible capability kinds or a + parallel browser/provider activation inventory would recreate the + discriminator-driven configuration this design is removing and introduce two + answers to whether an integration is active. **Decision:** `enabled` remains + the integration master gate, existing typed fields decide which optional + capabilities are configured, and routes to known-disabled integrations are + pruned so `enabled = false` remains a kill switch. +- **Silently accepting unknown integrations or providers in schema 2.** Exact + baseline acceptance must remain exact while schema 1 can still be loaded; + that concern is accepted. **Objection:** extending that permissiveness to new + source would turn typos into silently inactive configuration and prevent the + static catalog from validating ownership. **Decision:** exact legacy + acceptance belongs only to the schema-1 compatibility reader. New source and + stored schema 2 fail on unknown IDs; only references to an explicitly + disabled, known integration receive the kill-switch treatment defined below. + ## Goals 1. Move every concrete Rust integration implementation out of core and into one @@ -202,7 +331,10 @@ This design does not introduce: supported. - A jurisdiction or permission-policy redesign. - Client-cycle EC resolution or provider-code allocation. -- Upstream EdgeZero lifecycle, host-evidence, store, or adapter changes. +- Upstream EdgeZero lifecycle, host-evidence, store, or adapter changes. One + narrow typed-config validation extension is allowed: a source-bytes check and + a post-parse command-validation callback over the same loaded value. It does + not change target selection, deployment, storage, or adapter behavior. - New auction protocols, pricing algorithms, notification policies, or telemetry schemas. Configuration order intentionally replaces lexical provider-ID order wherever deterministic provider priority is observable. @@ -263,27 +395,28 @@ crates/ Cargo.toml src/ lib.rs - adserver_mock/ - mod.rs - aps/ - mod.rs - datadome/ - mod.rs - protection.rs - protection_scope.rs - didomi/ - mod.rs - ... - nextjs/ - mod.rs - html_post_process.rs - rsc.rs - rsc_placeholders.rs - script_rewriter.rs - shared.rs - fixtures/ - openrtb/ - mod.rs + integrations/ + adserver_mock/ + mod.rs + aps/ + mod.rs + datadome/ + mod.rs + protection.rs + protection_scope.rs + didomi/ + mod.rs + ... + nextjs/ + mod.rs + rsc.rs + rsc_placeholders.rs + rsc_stream.rs + script_rewriter.rs + shared.rs + fixtures/ + openrtb/ + mod.rs trusted-server-integrations-js/ build.rs @@ -309,13 +442,16 @@ crates/ lib.rs ``` -Every current flat Rust integration file becomes `/mod.rs`. Existing -nested modules and fixtures stay with their owner. `openrtb` is a built-in -Rust-only, directory-backed integration adapter that exposes configuration for -the current standard OpenRTB profile without turning the neutral OpenRTB -execution engine into concrete code. The target Rust catalog therefore has -sixteen definitions: fifteen moved implementations plus the new `openrtb` -adapter. +Every current flat Rust integration file becomes +`src/integrations//mod.rs`. Restricting the completeness scan to that +directory prevents helper modules from being mistaken for integrations. +Existing nested modules and fixtures stay with their owner. The current +Next.js `rsc_stream` implementation moves; the removed `html_post_process` +module does not return. `openrtb` is a built-in Rust-only, directory-backed +integration adapter that exposes configuration for the current standard +OpenRTB profile without turning the neutral OpenRTB execution engine into +concrete code. The target Rust catalog therefore has sixteen definitions: +fifteen moved implementations plus the new `openrtb` adapter. JavaScript-only `creative` remains valid without a Rust directory. Rust-only integrations remain valid without a browser directory. Cross-adapter Playwright @@ -334,11 +470,13 @@ The Cargo dependency graph is one-way: ```text trusted-server-integrations ──→ trusted-server-core ──→ trusted-server-js - │ + │ │ + │ └──→ trusted-server-openrtb └───────────────→ trusted-server-integrations-js adapters and CLI ────────────→ trusted-server-integrations adapters and CLI ────────────→ trusted-server-core +trusted-server-integration-tests ──→ trusted-server-integrations ``` The rules are: @@ -370,11 +508,14 @@ directory of implementations. It owns: - Aggregation of core and integration secret metadata. - Integration-owned preprocessing for conditionally active secrets. - Catalog-aware validation and capability construction. -- Pure source parsing, structural validation, and catalog-validation APIs used - by the CLI before EdgeZero's typed validate, diff, and push mechanics. +- Pure source parsing, structural validation, and deploy-validation APIs used by + the CLI at the validation level appropriate to each command. - The public runtime entry points that load a config-store blob and return one composed runtime value. -- A validated source-config view used by non-runtime CLI commands. +- A structural `SourceConfigView` and a deploy-validated + `ValidatedSourceConfig` used by non-runtime CLI commands. +- A generic `PartialSourceConfigView` for recovery-oriented commands that + intentionally type only one owned subtree of an otherwise invalid document. `TrustedServerAppConfig` contains neutral core configuration plus the ordered integration-owned source configuration. Concrete integration configuration is @@ -382,93 +523,166 @@ not added to core's `Settings`. Composition consumes the integration portion into capabilities and returns a neutral runtime `Settings` value containing only state that core execution engines understand. -The public API has two explicit phases. The source phase performs the pre-pass, -catalog resolution, integration-owned structural and deploy validation, secret -metadata aggregation, and serialization into the storage DTO. The CLI stops at -that phase and never constructs runtime capability objects from unresolved -secret key names. The runtime phase begins only after envelope verification, -inactive-secret preprocessing, and secret resolution; it validates the resolved -values and constructs the plan, registry, and browser assets. Both phases use -the same static catalog and integration schemas, but only adapters receive the -final `TrustedServerComposition`. - -The source phase returns a conceptual `ValidatedSourceConfig`. It retains the -typed operator configuration and exposes only the views CLI consumers need: -neutral global settings, ordered integration and qualified-provider metadata, -and integration-owned read models such as Prebid external-bundle inputs. This -is not a runtime registry and contains no resolved secret values or executable -capability objects. - -That runtime value, conceptually `TrustedServerComposition`, contains the -validated neutral `Settings`, one `Arc`, one -`IntegrationRegistry`, the composed `BrowserDocumentAssets`, and a canonical -digest of the complete composition snapshot defined below. Adapters consume -this value; they do not separately compile the auction plan, rebuild the +The public API has four explicit levels: + +1. `SourceConfigView` performs the TOML pre-pass, typed parse, catalog + resolution, and structural validation. It does not run deploy validation. + Read-only diagnostics that already require a complete typed root use this + level so unrelated deployment checks do not become new failures. +2. `PartialSourceConfigView` retains the source document plus one typed, + command-owned subtree. Recovery-oriented mutators and generators use it when + their current contract tolerates an invalid unrelated root. It cannot be + converted into a storage DTO or runtime composition, and each command names + the only source paths it may read or write. +3. `ValidatedSourceConfig` applies the selected environment overlay, aggregates + secret metadata, runs every secret-independent integration and + cross-integration check, serializes the storage DTO, and validates its order + sidecars. It invokes the same pure auction-plan compiler used at runtime, + including provider routing, browser/server bidder ownership, mediator + capability and enablement, and duplicate routes, then discards the + validation-only plan. A separate pure `validate_for_targets` operation runs + the resulting plan against the command's resolved target set. Neither path + creates executable capabilities or attempts to use unresolved secret values. +4. Runtime composition begins after envelope verification, inactive-secret + preprocessing, and secret resolution. It repeats the shared pure validation + kernel against resolved values, compiles the authoritative plan, constructs + plan-dependent capabilities, and returns the final composition. + +This distinction makes "runtime-only construction" precise: it does not move +any currently push-time, secret-independent plan failure to startup. Diff and +push have one selected adapter and fail when its target validation fails. +`config validate` has no adapter selector: it runs target validation for every +supported adapter declared by the manifest and prints a named result per target. +Ordinary validate fails on target-neutral errors and reports target-specific +failures as warnings so a valid Fastly configuration is not rejected merely +because the same multi-provider plan cannot run on Cloudflare or Spin; +`config validate --strict` fails if any declared target fails. A fixture rejected +by runtime composition for a secret-independent reason must produce the same +target-neutral error or the same named target result in the CLI. + +The complete source and validated views retain the typed operator configuration +and expose only the data their CLI consumers need: neutral global settings, +ordered integration and qualified-provider metadata, and integration-owned read +models. A partial view retains only its owned typed subtree and the source +document needed for a bounded edit. None is a runtime registry or contains +resolved secret values or executable capability objects. + +The runtime value, conceptually `TrustedServerComposition`, contains the +validated neutral `Settings`, one `Arc`, the plan-backed auction +orchestrator including the selected mediator, one `IntegrationRegistry`, the +composed `BrowserDocumentAssets`, the validated deployment target, and a lazy +canonical composition digest. Adapters consume this value; they do not +separately compile the auction plan, construct the orchestrator, rebuild the integration registry, enumerate browser bundles, or reconstruct the digest. +The composition is immutable and may live for a request, a Fastly sandbox, or a +bounded adapter cache without changing semantics. Stored capabilities are +`Send + Sync` and request-stateless. Script text buffers, Next.js stream state, +document observations, and other mutable transformation state are created by +per-document factories and live in request/processor state, never in a reused +registry object. Composition work is O(configuration); exact asset hashes are +build-time inputs and template identity is lazy/memoized as described below. + +Composition also exposes a narrow settings-only result before capability +construction. Fastly retains its current JA4 gate and failed-startup +finalization paths through that view; a registry or orchestrator failure must +not erase settings those degraded paths already use. + Core retains neutral config-store access, Fastly chunk reconstruction, blob envelope verification, preprocessing for inactive neutral/global secret references, secret-resolution primitives, global settings types, and auction-plan compilation. Today the neutral preprocessor removes inactive Tinybird token references and disabled EC-partner pull-token references; that behavior remains in core. These helpers accept or return neutral data and never -call the concrete catalog. The integration crate calls them in this order: +call the concrete catalog. Adapter-specific readers produce one verified +envelope/data value; Fastly chunk reconstruction remains a core loader helper, +while Cloudflare and Spin adapt their existing binding/KV inputs to the same +verified-data boundary. The integration crate composes it in this order: ```text config-store bytes → core chunk reconstruction and envelope verification + → settings-only neutral view → core-owned neutral/global inactive-secret preprocessing → catalog-owned integration inactive-secret preprocessing → aggregated core + integration secret resolution → catalog-aware config validation → core AuctionPlan compilation - → core IntegrationRegistry construction from typed registrations + → plan-dependent capability construction + → core IntegrationRegistry and orchestrator construction → browser asset composition and document fingerprint → TrustedServerComposition ``` -The CLI imports `TrustedServerAppConfig`, `ValidatedSourceConfig`, and pure -source-validation APIs from `trusted-server-integrations`. The host-only CLI -continues to own the config command wrappers and the direct `edgezero-cli` -dependency; `trusted-server-integrations`, which is linked into every WASM -adapter, never depends on `edgezero-cli`. Each CLI wrapper performs the -source-aware pre-pass and source-phase catalog validation before delegating -storage and diff mechanics to EdgeZero's typed CLI functions. - -Because locked EdgeZero accepts paths rather than validated bytes, a wrapper -copies the operator config and manifest inputs to private mode-`0600`, immutable -temporary snapshots, rewrites the delegated arguments to those snapshots, and -removes them afterward. The manifest snapshot is created beside the original -manifest so all manifest-relative adapter paths retain the same base directory; -failure to create the secure snapshot aborts before remote I/O. Snapshot paths -in errors and the validate-success line are rewritten or replaced with the -original operator paths. The config and manifest bytes used to choose the -adapter, store, and app-config path are therefore the same bytes EdgeZero -processes; a concurrent edit cannot redirect the delegated operation. No -EdgeZero source change or new host service is required. +The CLI imports `TrustedServerAppConfig`, the complete, partial, and validated +source views, and pure validation APIs from `trusted-server-integrations`. The +host-only CLI continues to own command dispatch and the direct `edgezero-cli` +dependency; +`trusted-server-integrations`, which is linked into every WASM adapter, never +depends on `edgezero-cli`. + +The current locked EdgeZero revision does not expose enough context for this +contract. EdgeZero therefore gains one narrow typed-config extension with two +default no-op stages. The source stage receives the selected app-config path and +exact raw bytes before deserialization. The effective-config stage receives a +borrow of the overlay-applied typed value plus the command kind and target +context EdgeZero already resolved: one adapter for diff/push and the manifest's +declared supported adapter set for validate. It does not select or mutate a +target. EdgeZero reads the file once, runs the source-aware pre-pass, constructs +one typed value, invokes command validation on that value, and serializes that +same value. Validate, diff, and push therefore cannot validate one app-config +read or typed value and serialize another. + +All workspace `edgezero-*` dependencies are then repinned together from v0.0.8 +to one immutable, reviewed tag or commit containing this extension. The +extension does not alter manifest parsing, target selection, +environment-overlay mechanics, logging, storage, or adapter behavior. It +replaces the filesystem snapshot wrapper entirely: config commands do not write +temporary operator or manifest copies, require a writable checkout, rewrite +logged paths, or rely on destructor cleanup around `process::exit` and signals. +If the extension cannot land and the workspace cannot repin to its reviewed +revision, milestone 2 is blocked rather than restoring the snapshot design. Read-only `config ad-templates` and `audit ad-templates` commands load the -effective source view with the existing optional environment overlay. Mutating -or generator commands load file bytes without the overlay, so environment-only -values are never persisted. Recovery-oriented ad-template generation may run a -structural-only pre-pass against an otherwise invalid baseline, preserving its -current warning and non-disclosure behavior. Candidate and baseline then use -the same complete integration-owned validation path: a valid candidate is -written; an invalid candidate is refused when the baseline was valid; and an -invalid candidate over an already-invalid baseline retains the current warning -and atomic-write escape hatch without disclosing source values. +effective `SourceConfigView` with the existing optional environment overlay when +they currently require the complete typed root. Mutating or generator commands +load file bytes without the overlay, so environment-only values are never +persisted. Recovery-oriented ad-template generation uses an explicitly bounded +`PartialSourceConfigView` against an otherwise invalid +baseline, preserving its current warning and non-disclosure behavior. +"Structural-only" means valid TOML, explicit parent/descendant structure, ID +grammar, and typed parsing of the subtree the command reads or writes; it +excludes unrelated required values, plan compilation, target validation, active +secret values, and publisher-domain deploy checks. Candidate and baseline use +the same selected validation level: a valid candidate is written; an invalid +candidate is refused when the baseline was valid; and an invalid candidate over +an already-invalid baseline retains the current warning and atomic-write escape +hatch without disclosing source values. Final validate, diff, and push always +use the complete `ValidatedSourceConfig` path. `ts prebid bundle` obtains typed bidder, User ID, analytics, and managed-module -requirements through an integration-owned partial source view rather than a -duplicate CLI schema. It intentionally does not require unrelated app +requirements through an integration-owned +`PartialSourceConfigView` rather than a duplicate CLI schema. It +intentionally does not require unrelated app configuration or an `external_bundle_url` to be deploy-valid: `bundle.modules`, `external_bundle_sha256`, and `external_bundle_sri` are inert staging metadata, while `external_bundle_url` activates the browser capability. The command retains its current ability to build first, atomically patch hash/SRI metadata, and tell the operator to upload and set the URL. It validates the structural pre-pass and affected Prebid subtree before writing; full app validation remains -the contract of config validate/push. Provider diagnostics display qualified -providers in configuration order rather than alphabetizing a detached map. +the contract of config validate/push. Because parent position is runtime order, +the command no longer invents or appends a missing `[integrations.prebid]` +parent. It requires an existing parent with explicit `enabled`, or exits with a +placement example; it may create only owned descendants beneath that parent. +Provider diagnostics display qualified providers in configuration order rather +than alphabetizing a detached map. + +`ts audit generate` may retain detector and edit metadata for concrete +integrations under this design's audit-reorganization non-goal. That metadata is +not a second runtime schema: audit candidates and final writes are parsed and +validated through `SourceConfigView`/`ValidatedSourceConfig`, and the audit code +does not define activation, defaults, secret metadata, or capability +construction independently. ## Static Rust Catalog and Browser Discovery @@ -478,9 +692,9 @@ providers in configuration order rather than alphabetizing a detached map. catalog. Each entry names an `IntegrationId` and a crate-private `definition` function from a same-named directory. Rust compilation checks every listed module and definition signature. A host-target completeness test enumerates -immediate `src//mod.rs` directories and fails if a valid directory is -missing from the catalog or a catalog ID has no directory. IDs must parse as -the `IntegrationId` defined by this design and must be unique. +immediate `src/integrations//mod.rs` directories and fails if a valid +directory is missing from the catalog or a catalog ID has no directory. IDs +must parse as the `IntegrationId` defined by this design and must be unique. There is no directory-local numeric order. Runtime order belongs to configuration. @@ -492,34 +706,53 @@ compilation. ### JavaScript -`trusted-server-js` and `trusted-server-integrations-js` use one Node toolchain -project, one lockfile, and one set of Vitest, ESLint, Prettier, Vite, and Prebid -aliases. The canonical project root remains +`trusted-server-js` and `trusted-server-integrations-js` use one browser Node +toolchain project, one browser lockfile, and one coordinated set of Vitest, +ESLint, Prettier, Vite, and Prebid resolution rules. The canonical project root remains `crates/trusted-server-js/lib`; `trusted-server-integrations-js` does not add a second `package.json` or lockfile. Neutral browser sources remain under `trusted-server-js`; integration sources, owned unit/artifact fixtures, and owned unit/artifact tests live under `trusted-server-integrations-js`. The -shared TypeScript, lint, format, and test configurations include that sibling -source root explicitly. Separate build targets emit neutral and integration -artifacts into owner-specific output directories and validate per-target -manifests before embedding them. One cross-process lock covers dependency -installation, output cleanup, build execution, discovery, manifest validation, -and artifact copy, so parallel Cargo build scripts cannot race or consume stale -or partially replaced output. +shared configuration does more than include that sibling source root: + +- Vite and Vitest resolve bare packages through the canonical project's + exports-aware resolver and deduplicate stateful browser-core entry points. +- TypeScript includes both source roots and supplies exact package paths where + Node's ancestor walk cannot reach the canonical `node_modules`. +- Vitest names the sibling test directory, includes its runtime and type tests, + and has a deliberately failing typecheck canary so `ignoreSourceErrors` or an + empty glob cannot make the gate pass silently. +- ESLint uses an explicit common base path and source globs. Prettier always + receives the canonical `--config` path for sibling files. +- Generated external-Prebid entries live under the canonical project or use an + explicit resolver that preserves the `prebid.js` package `exports` map; a + directory alias that bypasses package exports is forbidden. + +Separate build targets emit neutral and integration artifacts directly into +private owner-specific directories below their Cargo `$OUT_DIR` and validate +per-target manifests before embedding them. They never clean, discover, or copy +from one shared `dist` directory. A cross-process lock covers dependency +installation or mutation only; completed dependency trees may be read by +parallel owner builds. This removes stale/partial discovery races without +serializing every adapter build. The integration build discovers immediate directories containing `index.ts` -and emits one self-contained IIFE per entry point. Its Cargo build embeds each +and emits one IIFE per entry point. An IIFE may call the external versioned +browser runtime facade and therefore is not described as self-contained. Its Cargo build embeds each output and its SHA-256 hash. CI, browser integration scripts, and the CLI Prebid builder use the single workspace root rather than maintaining a second -dependency graph; Dependabot continues to watch only its one lockfile. +dependency graph; Dependabot continues to watch the one browser lockfile in +addition to the repository's unrelated Node projects. Canonical npm build, typecheck, lint, format, and test commands include both source roots explicitly, and CI invokes those commands rather than core-only -paths. Both Rust build scripts emit complete `rerun-if-changed` coverage for -their own manifest, configuration, and source inputs, including the sibling -integration root. Clean and incremental build tests change one integration -source and prove the integration manifest is regenerated without spuriously -changing the neutral manifest. +paths. A resolution test imports `vitest`, `prebid.js`, and one exported Prebid +module from a sibling integration file. Both Rust build scripts emit complete +`rerun-if-changed` coverage for their own manifest, configuration, and source +inputs, including the sibling integration root. Clean, incremental, and +concurrent build tests change one integration source and prove the integration +manifest is regenerated without spuriously changing the neutral manifest or +observing another build's partial output. `build-prebid-external.mjs` and its npm command remain at the canonical Node root as build orchestration, not browser runtime. The Prebid registry, aliases, @@ -547,7 +780,7 @@ The neutral contents of `integrations/registry.rs` move to singular - `IntegrationDefinition` and `IntegrationRegistration` contracts. - `IntegrationRegistry` and registry execution. - Proxy, request-filter, attribute-rewriter, script-rewriter, - HTML-post-processor, and head-injector traits and contexts. + HTML-stream-processor, and head-injector traits and contexts. - Neutral request-preparation and response-finalization hooks. - Neutral request-processing and response-sharing annotations. - Neutral browser-asset metadata, byte/hash access, load modes, and composed @@ -588,16 +821,34 @@ The OpenRTB profile boundary has three stages: 2. `CompiledOpenRtbProfile` is an object-safe, `Send + Sync` immutable profile stored as an `Arc` in each provider plan. It exposes only neutral routing facts and prepares one provider exchange from neutral auction input. -3. `PreparedOpenRtbExchange` contains the finalized outbound request plus a - boxed, object-safe response parser bound to that exact provider and request. - The parser owns any request-local APS, Prebid, or standard parsing state and - consumes itself when parsing the response. +3. Preparing a provider returns `Exchange(PreparedOpenRtbExchange)` or + `Skip { metadata }`. `Skip` preserves the current no-impressions/no-transport + outcome and any profile-specific diagnostic metadata. An exchange contains + the finalized outbound request plus a boxed, object-safe response parser + bound to that exact provider and request. The parser owns any request-local + APS, Prebid, or standard parsing state and consumes itself when parsing the + response. The prepared exchange lets the core engine retain shared backend registration, transport, deadlines, notification policy, normalized response handling, and telemetry. The profile owns request specialization, profile-specific headers, debug capture, response parsing, diagnostics, and renderer descriptors. +The neutral preparation input explicitly carries the current request facts: +DNT, raw Cookie, User-Agent, Referer, Accept-Language, attested client IP, +zone, consent-filtered identity/EIDs, routed demand, logical budget, transport +budget, and optional signer. Core constructs that snapshot, stamps the +`QualifiedProviderId`, and owns the generic request/response envelope. A +profile owns only its current branch points and bound parser state; moving code +does not duplicate the shared OpenRTB engine inside each integration. + +Renderer-bearing bids use a neutral, versioned descriptor with +`renderer_type`, opaque payload, and the handler-supplied targeting bid ID. Its +serde representation is byte-for-byte compatible with the existing tagged +`BidRenderer::Aps` JSON. Core may carry and serialize the descriptor, but only +the registered renderer handler interprets its payload or supplies APS/GPT +browser metadata. + Neutral routing policy replaces checks such as `is_prebid_server`. It expresses only behaviors the generic router needs, including whether `all_eligible` is allowed, whether trusted stored-request demand is recognized, and how bidder @@ -625,9 +876,16 @@ neutral boundary has three parts: normalized mediation result. Core retains mediator transport, deadline enforcement, telemetry, and the -existing warning-and-local-ranking fallback policy. `adserver_mock` owns only -request construction and response interpretation. The old mediator use of -`Arc` is deleted when its last caller migrates. +existing outcome policy. Dispatch or parse errors warn and fall back to local +ranking. A successfully parsed non-2xx `adserver_mock` error response is a +mediator response with zero winners and does not fall back; that current +distinction is covered by a contract test. The current mediator endpoint policy, +including an allowed HTTP endpoint where supported today, is preserved. Its +response correlation remains the current mediator integration identity rather +than being silently rewritten as an auction-provider identity. +`adserver_mock` owns only request construction and response interpretation. The +old mediator use of `Arc` is deleted when its last caller +migrates. No unused generic `AuctionProviderFactory` extension is added. A provider that cannot use the current OpenRTB engine will define that additional seam in a @@ -639,19 +897,21 @@ The explicit static catalog establishes what the binary supports. Configuration establishes what runs. Source validation and runtime startup both resolve IDs through the same catalog. -Source validation stops after validating the unresolved, storage-safe model. At +Source validation runs the validation-only plan path described above. At runtime, after secrets are resolved: 1. Parse `[integrations]` into an ordered sequence. 2. Resolve each ID against the static definition catalog. -3. Ask the owning definition to parse and validate its complete configuration. -4. For each enabled integration, construct the typed capability registrations - activated by its validated settings, in integration order restored from the - storage sidecar. -5. Collect integration-owned auction provider instances and profiles. -6. Ask core to compile the single canonical auction plan. -7. Resolve typed browser modules and construct the neutral integration - registry. +3. Ask each owning definition to parse and validate its complete configuration + and expose catalog-level profile/mediator definitions. +4. Collect enabled integration-owned provider instances into the independent + flat provider sequence and ask core to compile the single canonical auction + plan. +5. Construct plan-independent capabilities, then construct plan-dependent + capabilities such as Prebid bidder ownership, head injection, and mediator + selection with a read-only `Arc`. +6. Resolve typed browser modules and construct the neutral integration registry + and orchestrator in configured order. An absent integration is inactive. Every explicit parent integration table must contain `enabled = true` or `enabled = false`; there is no integration-specific @@ -659,8 +919,12 @@ default. `enabled` is the integration's master gate, not an assertion that every optional capability is configured. An explicitly disabled integration may retain its settings and provider instances but contributes no runtime capabilities or providers. Configuration cannot activate APS or Prebid through -an auction plan while omitting its parent, and bidder or mediator references to -a disabled integration fail validation. +an auction plan while omitting its parent. References to an absent or unknown +parent fail validation. References to an explicitly disabled, known integration +remain in source as a kill switch: they are omitted from the compiled route or +mediator plan with one aggregated warning and ordered diagnostics, and become +subject to full validation again when the parent is enabled. No unknown ID is +silently treated this way. Provider-owning integrations use the following explicit activation rules. They do not add capability names or implementation discriminators to operator @@ -679,10 +943,14 @@ configuration. Thus a current server-only Prebid provider migrates beneath an enabled Prebid parent without activating Prebid's page/browser behavior. Browser-only Prebid continues to be selected by the same required external-bundle URL that current -startup validation already uses. Runtime browser settings such as managed User -IDs or client-side bidders without that URL fail validation rather than -activating a partial browser path. CLI-only `bundle.modules` and generated -hash/SRI metadata may be staged without a URL and never activate runtime +startup validation already uses. The closed browser-only source-field set is +`account_id`, `debug`, `script_patterns`, `client_side_bidders`, +`excluded_gam_ad_unit_path_suffixes`, and `managed_user_ids`. Any explicitly +configured or effectively non-default value in that set without +`external_bundle_url` fails validation rather than activating a partial browser +path; an implicit default script-pattern list alone does not activate or fail. +CLI-only `bundle.modules`, `external_bundle_sha256`, and +`external_bundle_sri` may be staged without a URL and never activate runtime browser behavior. For APS, a configured provider necessarily activates its renderer support; an enabled APS parent with no provider is a staged no-op. @@ -816,6 +1084,16 @@ underscores to the same character, its correlation name includes a stable digest of the full qualified ID. Target validation rejects any remaining final name collision. Tests cover aliases such as `a_b.c` and `a.b-c`. +Internal qualified identity and externally serialized identity cut over at +different times. The schema-1 compatibility reader constructs a qualified +internal key but retains the legacy local provider ID for backend names, +mediator wire correlation, public `/auction` provider details, ts-debug, and +telemetry. Schema 2 uses the qualified value on those external surfaces. Thus +deploying the dual reader over an unchanged schema-1 blob does not split metrics +or mediator correlation; the documented rename occurs only when schema 2 is +pushed. The rollout updates exact-match dashboards, alerts, Tinybird filters, +mediator expectations, and the local template-cache harness before that push. + The mediator selects an enabled integration that registered a mediator capability. Its settings remain under that integration: @@ -844,8 +1122,10 @@ The contract is: 4. Every parent integration table contains an explicit `enabled` value. 5. Disabled integrations contribute nothing; remaining integrations keep their relative order. -6. The flattened auction plan orders providers first by owning integration and - then by local provider declaration. +6. Schema-2 plan input flattens providers first by owning integration and then + by local provider declaration. The neutral plan input nevertheless carries + one independent flat provider sequence so legacy compatibility can represent + globally interleaved provider IDs without regrouping them. 7. Every hook list and each immediate/deferred JavaScript list retains integration order. When one integration registers multiple hooks of the same capability, their relative order is the explicit order returned by that @@ -858,9 +1138,9 @@ The contract is: flattened plan. With the existing strict-greater-than price comparison, the first configured provider retains an equal-price tie during local winner selection. Dispatch checks the remaining shared deadline immediately before - each back-to-back launch; configuration order becomes budget-observable only - if the deadline expires or adapter timeout canonicalization reaches zero - during that launch loop. + each back-to-back launch, so every later launch observes a smaller logical + budget and may also cross an adapter timeout bucket; order is budget-visible + even when the deadline does not expire. 10. Shared browser dispatchers execute handlers by the owning integration's configured ordinal and then definition-local registration order. Wall-clock registration timing, numeric priority, and lexical handler ID are not @@ -872,12 +1152,32 @@ integration and provider. Lookup indexes may use maps, but iterating a execution priority. CLI diagnostics and registry metadata that show order use the ordered plan/registration view. -The current hard-coded requirement that `js_asset_proxy` remain the first -rewriter is retired. Rewriter chaining follows the same operator-visible -integration order as every other hook. Migration guidance places -`js_asset_proxy` first in migrated examples and procedures so the old behavior -is preserved by default, while an operator may deliberately choose a different -order. No engine-only priority is hidden from the configuration. +The stale comment that `js_asset_proxy` is globally first is retired. The +effective baseline registration sequence is, when each entry is active: + +```text +prebid, aps, js_asset_proxy, testlight, nextjs, permutive, lockr, didomi, +sourcepoint, osano, google_tag_manager, datadome, gpt, gpt_diagnostics +``` + +Milestone-one normalization and schema-one decoding use that complete sequence; +they do not move `js_asset_proxy` ahead of Prebid. Schema 2 instead chains +attribute rewriters in operator-visible integration order: a replacement becomes +the next rewriter's input and a removal is terminal. Operators are told that an +earlier native rewrite can prevent a later exact-original-URL proxy rule from +matching, and that moving `js_asset_proxy` ahead of Prebid can prevent the +publisher-Prebid remover from recognizing a rewritten URL. Overlap tests cover +GPT, Google Tag Manager, DataDome, Sourcepoint, Permutive, Lockr, Testlight, and +Prebid in both legacy-compatible and deliberately reordered configurations. No +engine-only override is hidden from TOML. + +Script-text rewriters also receive the current text produced by the preceding +rewriter rather than having independent `lol_html` handlers repeatedly replace +the original chunk. DOM-insertion guards claim canonical parsed host/path +ownership, not substring coincidences in a query string. Built-in ownership +sets are disjoint and tested with exactly one claimant for every canonical URL; +configured order applies to genuinely composable handlers, not to select the +winner of an ownership bug. Browser load mode remains a lifecycle phase, not a second operator priority: immediate code necessarily evaluates before deferred code. Config order is @@ -888,26 +1188,34 @@ composition rejects it on a deferred asset. Diagnostics display each module's fixed load mode so this phase boundary is visible rather than inferred from timing. -All recovery paths obey the same provider ordinals. In particular, the two -current transport-failure branches that sort provider IDs lexically are -replaced with plan-order recovery before the new ordering contract is enabled. +All recovery paths obey the same provider ordinals. Existing stable plan-index +sorts remain. The abandon/failure path that can emit providers from `HashMap` +iteration is changed to plan-order recovery before the new ordering contract is +enabled; no no-op lexical-sort change is claimed as a compatibility break. Inline-table and dotted-key shorthand may not define an integration parent or provider parent. Requiring ordinary table headers makes activation, ownership, and order visible in one form and lets the pre-pass produce targeted errors. -The workspace enables the `preserve_order` feature on its single resolved -`toml` package, so Cargo feature unification makes EdgeZero's `toml::Value` -maps order-preserving too. The integration-owned source model's integration and -provider collections use ordered map types with explicit iteration semantics, -never `HashMap` or `BTreeMap`. - -Before typed deserialization, `trusted-server-integrations` parses the source -with `toml_edit`. This source-aware pre-pass rejects descendant-before-parent -declarations, missing explicit parent tables, and missing `enabled` fields. It -then permits the existing EdgeZero scalar environment overlay; overlays may +The integrations crate directly enables the `preserve_order` feature on the +workspace's single resolved `toml` package; it does not rely on an unrelated +dependency to activate that feature through Cargo unification. EdgeZero's +`toml::Value` then sees the same ordered map implementation. A test asserts that +typed integration/provider order equals the independent `toml_edit` pre-pass +order. The integration-owned source model uses ordered map types with explicit +iteration semantics, never `HashMap` or `BTreeMap`. + +Before typed deserialization, the host-only `source-config` module of +`trusted-server-integrations` parses the source with `toml_edit`. The dependency +and code are feature-gated out of every WASM adapter graph. This source-aware +pre-pass rejects descendant-before-parent declarations, missing explicit parent +tables, and missing `enabled` fields. It then permits the existing EdgeZero +scalar environment overlay; overlays may replace values but may not create, remove, or reorder integration or provider -tables. +tables. Validate, diff, and push inspect app-prefixed overlay variable names +without reading their values into diagnostics and fail when a name targets a +retired auction-provider or integration path; silently ignoring an old endpoint +override is not permitted. The direct `toml_edit` dependency is pinned to the same TOML 1.1 parser generation as the workspace `toml` package and EdgeZero's typed parser. Parser @@ -917,26 +1225,26 @@ because it used an older TOML grammar. Every entry point that accepts TOML uses this pre-pass, including local loading, CLI validate/diff/push, ad-template diagnostics and candidate validation, and -the Prebid bundle command. Recovery mutators may request the structural-only -mode described above. Ad-template generation applies the comparative -candidate/baseline rule, and the Prebid builder validates only its owned partial -view; neither recovery path is silently tightened into unconditional full-app -validation. EdgeZero's typed mechanics remain responsible for overlay, -validation invocation, diff, envelope construction, consent, and store writes -after the pre-pass succeeds. - -EdgeZero does not currently expose a pre-parse hook. The host-only CLI wrappers -therefore duplicate its app-config path rule: an explicit `--app-config` wins; -otherwise the path is `/.toml`. A wrapper reads the -manifest and source once, performs the pre-pass, and delegates typed processing -against private immutable snapshots of those exact bytes; it does not validate -one read and allow EdgeZero to reopen a concurrently changed operator or -manifest file. Parity tests cover exact-byte delegation, diagnostic path -rewriting, explicit and default paths, manifest paths with and without parent -directories, and `--no-env`. The environment overlay can replace only scalar -leaves already present in TOML; it cannot create an omitted `enabled` field, -integration, provider, table, or array. Operator templates must contain every -leaf intended for overlay. +the Prebid bundle command. The integration-test `generate-viceroy-config` +binary also uses the same source parser and schema-2 storage serializer rather +than retaining a direct `toml::from_str`/legacy-envelope path. Recovery mutators +may request the structural-only mode described above. Ad-template generation +applies the comparative candidate/baseline rule, and the Prebid builder +validates only its owned partial view; neither recovery path is silently +tightened into unconditional full-app validation. EdgeZero's typed mechanics +remain responsible for overlay, validation invocation, diff, envelope +construction, consent, and store writes after the pre-pass succeeds. + +The EdgeZero extension uses EdgeZero's existing path and target resolution: an +explicit `--app-config` wins; otherwise the path is +`/.toml`; diff/push provide their resolved adapter; and +validate provides the supported adapter set declared by the manifest. Parity +tests cover exact-byte validation and serialization, command/target context, +explicit and default paths, manifest paths with and without parent directories, +concurrent app-config replacement, and `--no-env`. The environment overlay can +replace only scalar leaves already present in TOML; it cannot create an omitted +`enabled` field, integration, provider, table, or array. Operator templates must +contain every leaf intended for overlay. ### Config-store representation @@ -973,9 +1281,11 @@ Conceptually, the stored representation carries: `TrustedServerAppConfig` uses custom serialization at this boundary. Deserialization accepts the operator TOML table shape after the source pre-pass; serialization emits object-shaped configuration plus the schema and order -sidecars above. Serialization fails if a sidecar omits, duplicates, or names a -different key than its corresponding object map. Runtime loading validates the -same bijection before constructing ordered settings. +sidecars above. `Validate` checks the sidecar/map bijection even for commands +such as `config validate` that do not serialize. Serialization defensively +checks it again, and runtime loading performs the same check before constructing +ordered settings. Missing, duplicate, extra, or mismatched names fail at all +three boundaries. This is an explicit asymmetric serde boundary. EdgeZero's typed CLI deserializes and validates `TrustedServerAppConfig`, then its manual `Serialize` @@ -989,6 +1299,13 @@ integration has no provider collection. storage fields. They are generated by serialization and rejected if supplied in operator TOML. +The schema-2 sidecars are sufficient to derive the neutral flat provider +sequence: iterate `integration_order`, then each present `provider_order`. +Schema 1 instead supplies the independent sequence directly in global lexical +local-ID order before local IDs are mapped to qualified internal keys. The +neutral compiler never reconstructs legacy order by grouping providers under +their owners. + Object-shaped storage keeps secret paths such as `integrations.datadome.server_side_key_secret_name` valid. Core and integration secret metadata traverse that resolution view before ordered settings are @@ -1001,6 +1318,12 @@ validates every reference actually present in source, including a reference retained under a disabled integration; runtime filtering prevents inactive value resolution, not source syntax or adapter validation. +Provider tables do not contain secret-reference fields in this design because +the locked EdgeZero secret metadata cannot address arbitrary map keys. Adding a +provider secret requires a separate EdgeZero metadata prerequisite or a fixed, +integration-owned non-map path; implementations may not add an unresolvable +map-keyed secret ad hoc. + The private Rust type names may differ, but the serialized order must be explicit and covered by compatibility tests across: @@ -1037,6 +1360,12 @@ schemas, and pure validation functions as the adapters. Only config-store loading proceeds through resolved-value validation and runtime capability composition. +Source validation treats secret leaves as key/store references only. It does +not call constructors such as DataDome `try_new` with an unresolved key name; +format and semantic checks on the resolved secret value run only after runtime +resolution. References retained under disabled integrations still receive the +locked EdgeZero name, store, collision, and adapter checks described below. + The DataDome move accounts for every current production coupling rather than only its response marker: direct HTML-processor state, publisher template/body/privacy decisions, startup and deploy validation, legacy settings @@ -1054,9 +1383,9 @@ Validation fails for: - A descendant integration or provider table declared before its explicit parent. - Duplicate local provider IDs or duplicate qualified provider identities. -- A bidder route to an unknown, disabled, or incompatible provider. -- A selected mediator whose integration is absent, disabled, or lacks the - mediator capability. +- A bidder route to an unknown or incompatible provider. +- A selected mediator whose integration is absent, unknown, or lacks the + mediator capability when enabled. - Duplicate routes or renderer types. - A referenced browser module absent from the generated browser catalog. - An unsupported capability combination. @@ -1065,16 +1394,23 @@ A disabled integration may retain structurally valid provider configuration; those providers are not added to the plan. A globally disabled auction may likewise retain otherwise valid enabled integration and provider configuration so operators can prepare configuration before enabling the auction. References -from bidder routing or mediator selection to a disabled integration still fail, -even when the global auction is disabled. +from bidder routing or mediator selection to an explicitly disabled integration +are retained but omitted from the compiled plan with one aggregated warning. +This makes `enabled = false` a one-line integration kill switch without making +typos or absent definitions valid. The global auction switch remains the +cross-integration emergency stop. The design does not add a second generic +capability-specific enablement system; server-only Prebid remains an enabled +parent without browser-activating fields. ## Configuration Migration -This is a deliberate operator-source migration. The new CLI accepts only the -new inventory and never defines precedence between old and new source fields. -The runtime temporarily supports two stored application schema versions only -to make deployment safe; it normalizes exactly one complete stored shape and -never merges inventories. +This is a deliberate operator-source migration. Ordinary config commands in the +new CLI accept only the new inventory and never define precedence between old +and new source fields. `ts config migrate` is the sole legacy-source entry +point; it transforms one complete old source into a candidate and never treats +that source as deployable input. The runtime temporarily supports two stored +application schema versions only to make deployment safe; it normalizes exactly +one complete stored shape and never merges inventories. Representative mappings are: @@ -1091,6 +1427,41 @@ Representative mappings are: | `protocol = "openrtb-2.6"` | Remove it; the registered provider capability fixes the protocol | | Bidder route `provider = "pbs-main"` | `provider = "prebid.pbs-main"` | +Milestone 2 includes `ts config migrate [--dry-run]`. The command is a +comment-preserving `toml_edit` transformation, never a push. It: + +- rejects mixed old/new inventories and operates on one complete legacy source; +- maps every profile to its owning integration, flattens `profile_config`, drops + the fixed protocol, qualifies cross-integration references, and creates each + required parent before its descendants; +- emits retained/required integration parents in the deterministic migration + sequence `prebid, aps, js_asset_proxy, testlight, nextjs, permutive, lockr, +didomi, sourcepoint, osano, google_tag_manager, datadome, gpt, +gpt_diagnostics, openrtb, adserver_mock`, filtered to configured entries. The + first fourteen reproduce the complete baseline hook-registration sequence; + `openrtb` and the mediator have no competing page-hook position. This does not + preserve the previously cosmetic order of legacy `[integrations.*]` tables or + assume `js_asset_proxy` was globally first; +- writes explicit `enabled` using a frozen table of the baseline defaults rather + than guessing one value for every integration; the baseline-true set is + Prebid, GPT, Didomi, Lockr, and Permutive, and retained blocks for other + integrations remain false unless legacy server activation requires an + enabled parent; +- handles server-only Prebid and implicit APS activation explicitly, and warns + before enabling a retained disabled APS block whose `rendering_mode` would + change renderer-route behavior; +- emits providers in the new integration/provider declaration order, reports + every legacy globally interleaved priority that schema 2 cannot retain, and + shows the resulting qualified sequence before writing; +- prints old-to-new environment-overlay variable paths and fails when an + app-prefixed environment variable targets a retired path; +- preserves file permissions and uses the existing permission-preserving atomic + writer. + +Missing-`enabled` diagnostics name the exact old default even when operators +migrate by hand. The shipped example, `config init`, audit drafts, operator +guides, and environment-overlay examples contain a complete migrated shape. + Old `[auction.providers]`, `profile`, `profile_config`, and `protocol` fields fail with an actionable message naming the new integration-owned location or instructing the operator to remove the fixed protocol. A mixed old/new operator @@ -1110,68 +1481,71 @@ new data therefore remain in envelope version 1. New data carries existing stored shape during the transition. Unknown application schema values fail before secret resolution. -One release of the new binary contains a read-only compatibility decoder for -the existing stored object shape and `[auction.providers]`. It converts that -complete legacy value into the same neutral ordered composition used by schema -2, retaining the current fixed integration-builder and special-registration -order, the old lexical provider order, and the existing implicit APS activation -behavior for that legacy blob only. It emits an operator warning to push the -migrated configuration. New CLI writes schema 2 only. - -Rollout is ordered: - -1. Archive the current envelope bytes and record the adapter, store, and key. - EdgeZero has no config-pull command, so this uses the adapter's native - config-store read/export facility and is an explicit release artifact. -2. Deploy the dual-reader binary while the schema-1 blob remains active. -3. Complete health checks on every deployed instance. -4. Freeze configuration writes and fence old CLI artifacts from the deployment - credentials or release path; only the schema-2 CLI may write after this - point. -5. Push the migrated schema-2 configuration with the new CLI and read back or - otherwise assert `trusted_server_schema = 2` from the stored envelope. -6. Verify registry order, provider order, browser asset hashes, and auction - health before declaring the cutover complete. - -The rollout runbook must name and drill a concrete export and restore mechanism -for every deployed adapter before milestone 2. This is an external release -precondition, not an assumed CLI feature. In particular, the current default -remote Spin deployment path cannot read deployed config through the locked -EdgeZero CLI; it must use a verified platform/control-plane export and restore -facility or the schema-2 rollout for that target is blocked. - -An old binary must never serve a schema-2 blob. Rolling back after step 5 first -restores the archived schema-1 envelope, verifies that restoration, and only -then rolls the binary back. If the platform cannot coordinate those operations, -the release is paused rather than accepting an outage window. The compatibility -decoder is removed only in a later release after every supported deployment has -completed the schema-2 cutover. - -The write fence remains until old CLI credentials/artifacts can no longer push. -Because the dual reader intentionally accepts schema 1, schema-1 reappearance -after cutover is an explicit rollback event, never a tolerated ordinary write; -release monitoring alerts on it and the runbook requires either immediate -schema-2 restoration with the new CLI or the complete binary-rollback sequence. +One shared legacy DTO-to-neutral converter serves both the milestone-one source +normalizer and the schema-one stored-blob reader. It accepts exactly what the +baseline typed loader plus runtime validation accepts: unknown integration IDs +are ignored with a rate-limited warning, integration-specific omitted +`enabled` fields retain their baseline defaults, implicit APS activation +remains, and providers enter the independent flat sequence in global lexical +local-ID order. New schema-2 strictness is never applied retroactively to an +unchanged schema-one blob. The application-schema marker remains at the data +root, and every rollback binary is proven to reject an unknown root marker. +New CLI writes schema 2 only. + +Rollout is adapter-specific: + +| Adapter | Forward cutover | Rollback isolation and evidence | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | Export and reconstruct the complete schema-one envelope, including every referenced chunk. Create a new physical Config Store, seed it with schema one, and link it as `trusted_server_config` only in the service version containing the dual reader. After settings-dependent probes pass at representative POPs, push schema 2 to that new store. | The prior service version remains linked to the untouched schema-one store, so reactivation cannot observe schema two. Config GC is forbidden for either rollback generation until the rollback window closes. Control-plane confirmation alone is insufficient because store visibility is eventual. | +| Cloudflare | Create a new Worker version whose `TRUSTED_SERVER_CONFIG` variable contains schema 2; `ts config push` to KV is not treated as a runtime cutover. | Worker rollback restores the previous code and schema-one binding together. Verify the bound version, startup schema/digest log, and a settings-dependent route. | +| Spin | Use a named versioned KV key/store only when the deployment platform can export, restore, and select it atomically with the component version. | The default remote Spin path is blocked from schema-2 rollout until a concrete control-plane export/restore drill exists; liveness alone is not evidence because the startup-error router stays healthy. | +| Axum | Use the migrated local file/environment as a development-only cutover. | Retain the archived schema-one file and restart the matching binary/config pair. | + +The common release order is: archive and verify schema one; deploy the +dual-reader code against schema one; verify a settings-dependent route and +startup log reporting application schema and non-secret composition digest; +freeze writes; run `ts config migrate --dry-run`; cut over through the +adapter-specific mechanism; then verify registry order, provider order, browser +asset hashes, external provider IDs, and auction behavior. Existing +authenticated diagnostics may expose the same evidence, but no new public +status route is introduced. + +Store/version isolation makes downgrade structurally safe where the platform +supports it. The operational old-CLI fence remains defense in depth, not the +only protection: an old writer has no release credentials or destination +mapping for the new Fastly store/version or Cloudflare binding. Schema-one +reappearance after cutover is an explicit rollback event. The compatibility +reader is removed only in a later release after every supported deployment has +completed and drilled its rollback window. ## Required Neutral Lifecycle Boundaries ### Request processing and sharing annotations DataDome currently communicates a concrete request marker to core before -template lookup. Core uses it to bypass a shared template, require the origin -and a full HTML body, and stamp the final response private. Replacing it with a -late cache-only flag would change behavior. +template lookup. Core uses it to bypass a shared template, require an +unconditional complete origin representation for HTML documents by stripping +conditional and range headers, and make synthesized body-carrying HTML private. +It does not select buffered body processing. Replacing it with either a late +cache-only flag or a blanket buffering requirement would change behavior. Core therefore owns a small monotonic `RequestProcessingRequirements` value with three independent axes: -- shared-template eligibility: eligible or bypass; -- body processing: streaming allowed or full body required; -- response sharing: shared allowed or request-private. +- origin freshness and sharing: ordinary, bypass shared template/readthrough, + or require an unconditional full HTML representation; +- response privacy scope: unchanged, synthesized body-carrying HTML only, or + every response; +- fixed browser placement requirements, including a request-conditional + synchronous asset immediately after the unified tag. + +These requirements do not choose `Stream` versus `Buffered` routing and cannot +turn a DataDome-suppressed document into a size-limited full-body buffer. Every hook may only make a requirement more restrictive. The registry combines -requirements before template-cache and origin-selection decisions, carries the -result through HTML processing, and enforces final response privacy. This is a +requirements before template-cache, origin-readthrough, and origin-selection +decisions, carries the result through HTML processing, and enforces privacy at +the declared response scope. This is a processing contract, not a general policy, permission, or vendor-state system. Registry requirements are an additional monotonic veto, never an alternate @@ -1183,6 +1557,13 @@ governs both warm lookup and cold-store authorization. A registry hook can make an otherwise shareable request private or origin-bound; it cannot make a request shareable when any existing cookie or cache gate rejected it. +The verification matrix covers key-cookie, bypass-cookie, malformed, unlisted, +absent, and empty cookie cases plus cookie-independent origins. Integration +requirements may only restrict those existing decisions. DataDome request +filtering and template-cache effects remain Fastly-only where they are +Fastly-only on the baseline; this split does not invent equivalent adapter +behavior. + DataDome sets these requirements inside the request-filter hook at the point where it already decides client-tag suppression. Its tag-suppression detail remains opaque integration-owned request/document state. Core sees only the @@ -1194,7 +1575,20 @@ DataDome type, field, or JSON path. GPT diagnostics currently has direct preparation calls in adapters and core and a direct finalization call in core. Add neutral typed hooks for those two existing lifecycle points. The registry owns opaque request-scoped state -between them. +between them. Separately, the static catalog exposes reserved-input normalizers +for every compiled definition whether that integration is absent, disabled, or +enabled. + +The GPT-diagnostics reserved-input normalizer preserves the baseline behavior +that is independent of activation: it captures then removes `ts_console`, +removes `__Host-ts-console`, merges retained Cookie fields, and drops Cookie +fields that cannot be represented as visible ASCII. It runs before EC setup, +generic cookie parsing/classification, template-key construction, and origin +forwarding at every existing adapter boundary, with the direct publisher call +remaining an idempotent safety net. This static normalization contract is not +an enabled runtime capability. Therefore an absent GPT-diagnostics block cannot +reintroduce HTTP 400 responses for non-ASCII Cookie fields or leak reserved +inputs upstream. Preparation returns the opaque per-integration decision plus its declared `RequestProcessingRequirements`. The neutral requirements are available before @@ -1203,9 +1597,12 @@ state is never copied into a shared template. Registry preparation remains at both existing locations: adapter boundaries and the idempotent core publisher safety-net boundary used by direct core callers. Request extensions make a second preparation a no-op. Core invokes finalization on the existing response -path. The current GPT-enabled auction-correlation check becomes a neutral -registry/request-state query, so core retains neither a concrete GPT import nor -a new lifecycle call site. +path. GPT diagnostics may request the fixed synchronous post-unified asset phase +and all-response privacy; ordinary head injectors cannot approximate that +position. The current auction-correlation decision is deployment-scoped, not +request-activation-scoped, and becomes a neutral enabled-capability query so +SPA and non-navigation auctions retain correlation without a concrete GPT +import. ## Browser Composition and APS Renderer @@ -1214,8 +1611,10 @@ implementation audits every production value import from an integration or the fixed `creative` prelude into today's `core/` and `shared/` trees. Each import is classified rather than copied blindly: -- Stateful facilities remain owned once by browser core and are exposed through - one versioned `TrustedServerBrowserRuntime` namespace. The initial surface +- Stateful facilities use one versioned `TrustedServerBrowserRuntime` facade. + Browser core owns its API and adoption rules, while bootstrap code that must + run before the unified bundle may create the backing `window.tsjs` object and + selected state first. The initial surface includes logging, context-provider registration and context collection, auction request construction and normalized response parsing, queue installation, slot lookup and rendering helpers, first-impression state, DOM @@ -1234,34 +1633,47 @@ registration, Prebid auction helpers, Testlight queue installation, GPT slot resolution, and the shared script/beacon guards; it is not limited to the APS renderer examples. -The core IIFE initializes exactly one stateful registration object on the -Trusted Server browser namespace before any integration IIFE runs. Integration -bundles consume that object through an external runtime shim and type-only -browser-core declarations; their bundler must not inline the stateful registry -implementation. Artifact tests prove that state registered by an integration -IIFE is visible to the already-loaded core IIFE, including a Permutive context -provider observed by core collection and a renderer observed by GPT and Prebid. +The GPT bootstrap still runs before the unified bundle, creates or adopts +`window.tsjs`, and may install the first-impression state and GPT lifecycle +listeners. The core IIFE must adopt that exact object identity, preserve +pre-bundle claims and listener markers, validate the runtime-facade version, and +initialize only missing registries. It never replaces bootstrap state. Immediate +GPT and deferred Prebid likewise adopt the facade and keep listener installation +idempotent. Integration bundles consume it through an external runtime shim and +type-only browser-core declarations; their bundler must not inline the stateful +registry implementation. Artifact tests load bootstrap, core, immediate GPT, +and deferred Prebid in production order and prove that first-impression claims, +Permutive context, logging configuration, renderer registrations, and APS frame +supersession use the same shared state. + +Every integration IIFE declares the facade version it requires. A mismatched +deferred or stale asset fails closed for that integration with a diagnostic +rather than creating a second registry or mutating an incompatible shape. This +is the runtime version-skew contract for static routes that historically serve +current bytes even when a stale `?v=` value is requested. `trusted-server-integrations-js` builds integration IIFEs. Immediate modules are concatenated in the ordering contract above. Deferred and standalone modules remain separate assets but retain their configuration-relative order and typed identities. -The browser DOM-insertion dispatcher follows the same simple ordering rule as -Rust hooks. Its current numeric priority and ID-lexical sort are removed. -Composition installs the immutable integration ID-to-ordinal mapping before any -IIFE executes. A handler registers under its owning integration ID and executes -by configured ordinal, then by that integration's local registration sequence. -Handler IDs remain diagnostic identities only, and DOM-insertion handlers are -immediate-only so an absent deferred handler cannot observe mutations too late. -Reversing two configured integrations therefore reverses the winner when their -script guards both claim the same candidate; no hidden browser priority or -bundle timing can override TOML order. - -The Rust registration does not carry only a string module ID. Core owns a -neutral immutable `BrowserAsset` contract containing the typed module ID, -embedded bytes, SHA-256 hash, load mode, and trusted script-tag attributes. -Composition produces a `BrowserDocumentAssets` value containing: +The browser DOM-insertion dispatcher follows the same ordering rule as Rust +hooks. Its current numeric priority and ID-lexical sort are removed. Immediate +IIFEs evaluate in configured concatenation order, so their registration +sequence already is the configured sequence; no separate ordinal mapping or +inline mapping script is emitted. A handler registers under its owning +integration ID and executes by registration sequence, then by that +integration's local registration sequence. Handler IDs remain diagnostic +identities only, and DOM-insertion handlers are immediate-only so an absent +deferred handler cannot observe mutations too late. Built-in guards must have +disjoint parsed URL ownership as specified above; configuration order is not a +repair mechanism for overlapping substring matchers. + +The Rust registration does not carry only a string module ID. Core owns an +immutable `CompiledBrowserAsset` containing the typed module ID, embedded bytes, +SHA-256 hash, and load mode. Composition binds configuration-dependent trusted +script-tag attributes separately and produces a `BrowserDocumentAssets` value +containing: - the exact ordered byte parts and concatenated hash for the unified asset; - the ordered deferred and permitted standalone assets with their bytes and @@ -1285,36 +1697,59 @@ output. Request-dependent head variation is permitted only when its neutral `RequestProcessingRequirements` bypass shared-template reuse; request data is never folded into a composition-wide fingerprint. -Publisher template invalidation is broader than the browser document. During -composition, `trusted-server-integrations` hashes a canonical serialization of -the complete composition snapshot: resolved neutral `Settings`, resolved -enabled-integration configuration, and structurally validated retained source -configuration for disabled integrations, all in configured integration and -provider order. Active secret values enter the hash after resolution; inactive -secret references remain unresolved retained source values. Hashing the -complete model deliberately over-invalidates so a future rewriter, -postprocessor, proxy mapping, cookie policy, or other HTML-shaping field cannot -be omitted from a hand-maintained allowlist. Input bytes and resolved secret -values are fed directly to the digest and are never logged, returned, or used -as cache-key plaintext. - -“Canonical” is a versioned encoding contract, not ordinary `Serialize` output. -It preserves the explicitly ordered integration/provider sequences, sorts keys -of semantically unordered maps and elements of semantically unordered sets, -uses length-delimited domain-separated fields, and has golden vectors. Tests -build equivalent `HashMap`/`HashSet` values in different insertion orders and -require the same digest, while reversing an operator-ordered sequence must -change it. - -Core computes the existing template fingerprint from that configuration digest -and `BrowserDocumentAssets.document_fingerprint`. This composite replaces only -the old complete-`Settings` plus global-bundle digest; it does not replace any -other `TemplateCacheKey` dimension. Full URL, request host and scheme, origin -identity, assembly mode, ordered `Vary` values, selected cookie values, and -`TEMPLATE_SCHEMA_VERSION` remain independent key inputs. Transform-shape -changes still bump `TEMPLATE_SCHEMA_VERSION`. Tests prove neutral settings, -non-head integration rewriter settings, external and inline assets, and cookie -policy changes invalidate templates without exposing raw configuration. +Publisher template invalidation is broader than the browser document, but it +does not invent another canonical serializer or hash resolved secret values. +The composition digest is: + +```text +SHA-256( + UTF8("ts-composition-v1\0") || + HEX_DECODE_32(verified_envelope.data_sha256) || + BrowserDocumentAssets.document_fingerprint_raw_32 || + U32_BE(application_schema) || + U32_BE(LEN_UTF8(build_id)) || UTF8(build_id) +) +``` + +The two digests are raw 32-byte values inside this framing; the EdgeZero digest +is decoded from its validated canonical 64-character lowercase hexadecimal +representation. `application_schema` is an unsigned 32-bit integer. `build_id` +is UTF-8 with a four-byte big-endian byte-length prefix. These fixed widths and +the one explicit length prefix make the input unambiguous; implementations do +not hash the displayed formula, hexadecimal text, or delimiter-free strings. + +`data_sha256` is EdgeZero's already-verified canonical hash of stored data +before secret resolution. It covers schema 2 order sidecars, enabled and +disabled source, neutral settings, overlay results, and secret references +without exposing secret values. Raw resolved secrets never enter a template +key. If a future resolved secret changes document bytes rather than only +authorizing an upstream call, its owner must contribute a reviewed non-secret +behavior fingerprint or make the output request-private; it may not hash the +secret itself. Semantically set-valued fields serialize in deterministic order +before EdgeZero hashes them. + +The digest is computed lazily and memoized on the immutable composition so +non-document routes and ineligible template-cache requests pay no repeated +full-config hash cost. Normal request execution does not force it merely for +logging. During a rollout, each adapter's configuration-activation path forces +it once for the activated snapshot and emits one rate-limited schema/digest log; +subsequent template use reads the memoized value. An adapter that cannot retain +the composition still deduplicates this rollout log per schema/data hash and +build ID rather than logging per request. The composition digest is distinct +from the exact concatenated unified-bundle hash, whose existing byte-content +semantics remain unchanged. + +Core uses that configuration digest directly as the template fingerprint's +configuration-and-document contribution; it does not append +`BrowserDocumentAssets.document_fingerprint` a second time because the digest +already covers it. This replaces only the old complete-`Settings` plus +global-bundle digest; it does not replace any other `TemplateCacheKey` +dimension. Full URL, request host and scheme, origin identity, assembly mode, +ordered `Vary` values, selected cookie values, and `TEMPLATE_SCHEMA_VERSION` +remain independent key inputs. Transform-shape changes still bump +`TEMPLATE_SCHEMA_VERSION`. Tests prove neutral settings, non-head integration +rewriter settings, external and inline assets, and cookie policy changes +invalidate templates without exposing raw configuration. Browser core currently imports APS renderer logic directly. Replace that reverse dependency with one neutral renderer registration mechanism. A @@ -1343,6 +1778,10 @@ type-keyed handler registry and neutral calls into it: dispatch handler while retaining the current slot-scoped replay protection. The Prebid `adId` path and server-bid path remain distinct neutral entry points so one cannot consume the other's authority accidentally. +- The direct `/auction` renderer-bid path in browser core also dispatches + through this registry; it is not a third APS-aware implementation. Renderer + failure reasons cross the neutral boundary as stable reason codes for the GPT + bridge without adding an APS enum to core. An enabled APS integration whose provider can emit APS renderer descriptors includes its immediate APS browser module. The core IIFE and fixed creative @@ -1364,7 +1803,10 @@ missing or rejecting renderer suppresses that bid with no generic-creative or native-Prebid fallback; unrelated bids and the page continue. A duplicate type fails Rust composition, while a defensive browser-side duplicate poisons that type rather than using last-registration-wins. No renderer route, DOM, message, -or beacon side effect occurs before the selected handler accepts the payload. +response, or beacon side effect is newly created before the selected handler +accepts the payload. This rule does not remove the baseline APS supersession +bookkeeping or GPT's fail-closed event propagation, which may occur while +deciding whether a payload is admissible. Carrier scrubbing, failed admission, bounded capacity and TTL, source binding, atomic consumption, replay rejection, and `markWinningBidAsUsed` are part of the compatibility contract rather than APS-private implementation details that @@ -1392,15 +1834,18 @@ location used by both Rust and browser tests. The move includes a repository-wide inventory of production executable browser programs assembled by integration Rust, not only files that already end in `.js`. The current DataDome, Didomi, GPT, Prebid, and Sourcepoint config -initializers; Sourcepoint `_sp_` property trap; and GPT-diagnostics -activation/history bootstrap all have integration-owned static program bodies -or typed templates in `trusted-server-integrations-js`. Integration Rust may -serialize safe data, invoke the generated typed template renderer, and assemble -script tags; it does not retain handwritten browser algorithms in Rust string -literals. Exact rendered inline bytes and hashes participate in the document -fingerprint. A source/artifact guard fails when a new production executable -integration script is introduced directly in Rust without an explicitly -reviewed data-only exception. +initializers; Sourcepoint response patches and `_sp_` property trap; and +GPT-diagnostics activation/history bootstrap move byte-for-byte with their +owners. Core-owned `build_bids_script`, `build_seam_script`, and +`build_ad_slots_script` remain core-owned neutral document programs. + +This split does not introduce a general template language. Integration-owned +program bodies live in `trusted-server-integrations-js`; integration Rust may +serialize safe data and fill a narrowly generated owner-specific renderer that +preserves the baseline escaping model and output bytes. Exact rendered inline +bytes and hashes participate in the document fingerprint. A source/artifact +guard fails when a new production executable integration algorithm is +introduced directly in Rust without an explicitly reviewed data-only exception. ## Error Handling @@ -1422,9 +1867,33 @@ parse failure continues to warn and fall back to local ranking; renderer failure follows the per-bid rule above. Moving a concrete call behind a registry must not change that hook's policy or introduce a panic. -Core visibility changes remain narrow. Helpers move with their integration -when possible. Core exposes a new public item only when a neutral cross-crate -contract requires it. +Core visibility changes are governed by a reviewed allowlist and a public-API +snapshot gate. Helpers move with their integration when possible. The +implementation plan names every module and item widened for Prebid, APS, and +the other integrations, classifies it as a neutral contract or moves it +outward, and fails CI on an unreviewed new export. Auction-engine inputs such as +transport headers and response-admission diagnostics are specified as part of +the neutral OpenRTB contract rather than exposed accidentally one helper at a +time. + +Core tests do not add a dev-dependency on `trusted-server-integrations`, which +would form a cycle. Vendor-specific unit tests and fixtures move with their +owner. Core tests use neutral stubs behind `test-utils`; the implementation plan +inventories current vendor-bearing fixtures, registry constructors, and test +call sites and checks that moved tests are still selected by a native or +target-matched gate. + +Some names remain in core because they are established wire/configuration +contracts rather than implementation ownership. The reviewed retained-name +allowlist includes `creative_opportunities.slot.providers.{aps,prebid}`, the +browser `tsjs.scheduleInitialAdInit` and GAM `hb_*` handshake, and the historical +`prebid_eids` neutral identity module. Their current serialized forms and CLI +editing behavior remain. Conversely, `TrustedServerError::Prebid`, concrete +Prebid request extensions, the JS-asset-proxy header constant, the +`"adserver_mock"` plan literal, DataDome-only staging input, vendor benches, and +the concrete core test fixture move outward or become a named neutral contract. +A mechanical guard rejects any additional concrete vendor token in core unless +it is added to this allowlist with a wire-compatibility reason. ## Compatibility Contract @@ -1434,7 +1903,9 @@ provider-ID order to qualified configuration order. It preserves: - Integration IDs. - Existing routes and endpoint behavior. -- Existing integration hook behavior, now ordered by configuration. +- Existing integration hook behavior in milestone 1, except for the explicitly + listed shared-browser-state corrections; schema 2 then changes applicable + hook order to configuration order. - Auction plan compilation semantics after configuration normalization, except for the documented provider-priority source. - OpenRTB request, response, routing, timeout, notification, and response @@ -1443,14 +1914,16 @@ provider-ID order to qualified configuration order. It preserves: telemetry schema. Provider response order and a locally selected equal-price winner may change when configuration order differs from the old lexical order. -- Existing provider identity fields, with values migrated from local IDs such - as `pbs-main` to qualified IDs such as `prebid.pbs-main`. +- Existing provider identity fields. Schema 1 retains local external values; + schema 2 migrates them from values such as `pbs-main` to qualified values such + as `prebid.pbs-main` at the documented cutover. - Managed Prebid User ID aliases, collision checks, consent gating, opaque LiveRamp envelopes, OpenRTB EID production, EC partner ingestion, and admin diagnostics. - External Prebid bidder, User ID, and analytics-module selection, manifests, hashes, SRI values, and runtime codes. -- Cache privacy, full-buffer decisions, cookie-key and bypass policy, and every +- Cache privacy, streaming/buffer decisions, origin-representation policy, + cookie-key and bypass policy, and every existing publisher template-key dimension. - Current CLI ad-template diagnostics, audit/generator recovery behavior, and Prebid bundle mutation behavior. @@ -1472,14 +1945,22 @@ The intentional compatibility breaks are: is rejected. - APS providers no longer activate without an enabled `[integrations.aps]` parent. -- `js_asset_proxy` is no longer implicitly first; migration examples and - procedures place it first to retain existing behavior unless the operator - reorders it. -- Provider failure responses and mediator input stop using the two remaining - lexical recovery sorts and use configuration order everywhere. +- A retained enabled Prebid parent with neither a provider nor browser-activating + URL becomes a valid staged no-op; the activation matrix and migration tool + make this explicit rather than inheriting an integration-specific default. +- Attribute and script rewriters use schema-2 configuration order instead of + the complete legacy sequence. The migration tool shows this order and its + overlap diagnostics; it does not falsely describe `js_asset_proxy` as the + baseline first rewriter. +- The unordered abandon/failure provider path begins using plan order. - Conflicting trusted attributes for the unified script tag fail composition instead of warning and keeping the first value; identical duplicates still collapse. +- Browser state that is accidentally duplicated across self-contained IIFEs is + unified behind the runtime facade: Permutive context reaches core collection, + integration logging follows `tsjs.setConfig`, and APS renderer/frame state is + shared across core, GPT, and Prebid. These are explicit baseline bug fixes, + not invisible "pure moves." - Stored data gains the application schema and explicit order sidecars. The rollout decoder, not the steady-state schema, provides temporary old-blob compatibility. @@ -1510,15 +1991,19 @@ The first merge milestone contains workstreams 1 through 3 and retains the current operator schema and provider activation semantics through a temporary normalization adapter. It delivers the crate boundary without an operator cutover. The second milestone contains workstream 4, introduces stored schema -2 and the rollout decoder, and atomically replaces the temporary source -normalizer with the new source parser. Neutral contracts from workstream 1 may -land as behavior-inert preparatory commits, but milestone 1 is not complete or +2 and the rollout decoder, and replaces the temporary source normalizer with +the new source parser. Neutral contracts from workstream 1 may land as +behavior-inert preparatory commits, but milestone 1 is not complete or deployable as the new architecture until workstreams 1 through 3 all meet its exit criteria. -Intermediate commits may add unused neutral contracts or new crates, but no -merged state may have two active catalogs, two simultaneously interpreted -provider inventories, or adapter-specific composition paths. This scope does -not include an external plugin ecosystem. + +Intermediate revisions keep exactly one active catalog while moving +integrations incrementally. The catalog may temporarily delegate individual +entries to explicitly allowlisted implementations still in core, and the +browser manifest may temporarily point individual entries at the legacy source +root. Each move removes one transitional entry. There are never two catalogs or +two interpreted provider inventories, and copied-but-unused duplicate source +trees are forbidden. This scope does not include an external plugin ecosystem. The milestone-one normalizer is a compatibility boundary, not a second catalog. It accepts only the current operator and stored shape, resolves current @@ -1533,10 +2018,13 @@ Milestone exit criteria are independent: - **Milestone 1 — crate boundary:** only the current operator and stored schema are accepted; fixed integration-builder order, lexical provider priority, - implicit APS activation, and all current browser, CLI, cache, and adapter - behavior remain unchanged. Every live consumer uses the new composition root, - old concrete sources and the old catalog are deleted together, and the full - repository gates pass. + implicit APS activation, CLI behavior, cache policy, adapter behavior, and + Fastly settings-only failure paths remain unchanged. The browser-state + corrections listed in the compatibility contract and one-time artifact/hash + invalidation are explicit milestone deltas. DOM-handler ordering and trusted + attribute conflict policy remain at baseline until milestone 2. Every live + consumer uses the new composition root, every concrete source has moved, all + transitional delegations are deleted, and the full repository gates pass. - **Milestone 2 — ordered configuration cutover:** the new operator source is the only writable shape; the dual stored-schema reader is deployed; qualified identities and order sidecars are used end to end; every normal and recovery @@ -1548,16 +2036,10 @@ Milestone exit criteria are independent: ## Migration Sequence -Implementation may use small commits, but the merged workspace must never have -two active integration or provider inventories. - -Steps 1 through 5 comprise milestone 1. Steps 1 and 2 are independently -mergeable, behavior-inert preparation; within that milestone, steps 3 through 5 -form the atomic ownership cutover. Preparatory commits may compile and -parity-test copied code in an unused new crate while the old catalog remains -authoritative, but the final milestone switch rewires every consumer and -deletes the old concrete sources together. No deployable revision selects some -integrations or browser assets from each catalog. +Implementation uses reviewable commits, but the merged workspace must never +have two active integration or provider inventories. Steps 1 through 6 comprise +milestone 1; each crate/source move is independently testable under the one +transitional catalog. 1. Add neutral capability, processing-requirement, browser-asset, OpenRTB profile/exchange, and mediator contracts to core. Extend the `test-utils` @@ -1565,41 +2047,54 @@ integrations or browser assets from each catalog. 2. Replace the closed APS and Prebid profile variants and the mediator's legacy `AuctionProvider` use while implementations are still in core. Convert GPT diagnostics and DataDome call sites to the neutral lifecycle contracts. -3. Create `trusted-server-integrations-js`, move all integration-owned browser - sources, unit/artifact tests, GPT bootstrap, the APS renderer document, - every integration-owned executable inline template, registry inputs, and - shared fixtures; complete the cross-root import audit; remove every - browser-core APS import; retire DOM-handler priority sorting; and make the - composed asset set authoritative for bytes and hashes. Retain cross-adapter - system tests in `trusted-server-integration-tests` and update their paths and - load order. -4. Create `trusted-server-integrations` with the explicit static catalog. Move - ordinary integrations first, then move DataDome and GPT diagnostics after - their lifecycle seams, APS and Prebid after the profile seam, and - `adserver_mock` after the mediator seam. Add the new `openrtb` adapter. -5. Move `TrustedServerAppConfig`, all integration-specific configuration, +3. Create both new crates and update Cargo aliases, native/WASM checks, JS + discovery/typecheck/lint/format commands, Dependabot inputs, and test package + selection in the same change. Update compiled documentation snippets and + crate-path documentation required for that workspace state. Create the + explicit Rust catalog as the one active catalog, initially delegating through + a shrinking, reviewed set of transitional core exports. Create the one + browser manifest with an equally explicit shrinking map to legacy source + paths. No production code is copied into an unused duplicate tree. +4. Move ordinary Rust integrations one owner at a time, then DataDome and GPT + diagnostics after their lifecycle seams, APS and Prebid after the profile + seam, and `adserver_mock` after the mediator seam. Add `openrtb`. Each move is + a rename-focused change that moves its unit tests/fixtures, removes its + transitional export, runs the public-API guard, and proves the moved tests + are selected. +5. Move browser integrations one owner at a time with their unit/artifact tests, + GPT bootstrap, APS renderer document, integration-owned executable inline + programs, registry inputs, and shared fixtures. Each move removes one legacy + manifest path and updates its JS gates immediately. Complete the cross-root + import audit, remove browser-core APS imports, adopt the shared runtime + facade, and make composed assets authoritative. Keep baseline DOM priority + ordering until milestone 2. Cross-adapter system tests remain in + `trusted-server-integration-tests` and gain bootstrap/core/immediate/deferred + load-order coverage. +6. Move `TrustedServerAppConfig`, all integration-specific configuration, validation, inactive-secret preprocessing, and secret metadata into the - integrations crate. Rewire the CLI to the pure source-validation entry point - while keeping host-only EdgeZero command wrappers in the CLI, and rewire all - adapters to the single runtime composition entry point while retaining the - existing operator schema through the temporary normalizer. This is the - behavior-preserving crate-split milestone. -6. Add the TOML source pre-pass, ordered in-memory maps, application schema 2, + integrations crate. Land the EdgeZero typed-config source/command extension, + repin all EdgeZero workspace dependencies together, rewire the CLI to the + source-view APIs, and rewire all adapters to the single runtime composition entry + point while retaining the existing operator schema through the shared legacy + converter. Remove all transitional catalog/source entries. +7. Add the TOML source pre-pass, ordered in-memory maps, application schema 2, object-shaped storage with explicit order sidecars, strong qualified - provider IDs, the dual stored-schema reader, and the breaking - integration-owned provider schema. Atomically replace and remove the - milestone-one old-source normalizer so this CLI accepts only the new source - inventory. -7. Replace every lexical provider recovery sort with compiled-plan ordinals, - update adapter backend correlation naming, and activate configuration-order - semantics only after their parity and failure-path tests pass. -8. Update examples, fixtures, operator documentation, migration diagnostics, - CI aliases, browser scripts, and the rollout runbook. Deploy the dual reader, - then push schema 2 according to the rollout section. -9. Remove the read-only schema-1 blob decoder only in the later release defined - by the rollout contract. - -Steps 6 through 8 are one deployable milestone-two cutover. They may be + provider IDs, the shared schema-one reader, `ts config migrate`, and the + integration-owned provider schema. Replace and remove only the legacy source + entry point from ordinary config commands; the isolated migration parser may + read legacy source only to emit a schema-2 candidate. +8. Activate configuration-order provider and hook semantics, chained script + rewriting, disjoint DOM claims, strict trusted-attribute conflicts, external + qualified IDs, and plan-order recovery only after focused baseline and + failure-path tests pass. +9. Update operator examples, fixtures, migration diagnostics, guides, and the + adapter-specific rollout runbook. Repository/CI path updates associated with + moved code are already complete from steps 3 through 6. Deploy the dual + reader, then cut over schema 2 according to the rollout section. +10. Remove the read-only schema-1 blob decoder only in the later release defined + by the rollout contract. + +Steps 7 through 9 are one deployable milestone-two cutover. They may be implemented as separately reviewed commits, but schema 2 must not merge or deploy while lexical recovery ordering, CLI consumers, or operator guidance still implement the old contract. @@ -1616,17 +2111,19 @@ to preserve `cfg(test)` imports. ### Discovery and dependency tests - The static Rust catalog contains exactly the sixteen expected definitions, - and every valid `src//mod.rs` directory appears exactly once. + and every valid `src/integrations//mod.rs` directory appears exactly once. - Invalid, missing, or duplicate catalog IDs fail tests or compilation. - JavaScript-only and Rust-only directories are accepted. - A typed Rust reference to an absent browser module fails compilation. - Embedded bundle hashes match built bytes. - Core has no dependency on either integrations crate. - `trusted-server-integrations` has no `edgezero-cli` dependency and compiles in - every native and WASM adapter graph; host-only path/snapshot/delegation code - remains in `trusted-server-cli`. + every native and WASM adapter graph; the `toml_edit` source module is absent + from WASM graphs and host command dispatch remains in `trusted-server-cli`. - Browser core imports no integration source. - Adapters and CLI import no concrete integration module. +- A public-API snapshot rejects exports not present in the reviewed extraction + allowlist, and core tests introduce no dev-dependency cycle. - A dedicated native test/clippy gate executes the integrations catalog and host-only completeness tests; relying on adapter dependency builds is not sufficient to run them. @@ -1636,6 +2133,11 @@ to preserve `cfg(test)` imports. - TOML parent-table order becomes the integration-owned source-model order. - The `toml_edit` pre-pass and typed `toml`/EdgeZero parser use the same TOML language generation and agree on parity fixtures outside `[integrations]`. +- A dependency-graph gate proves the host CLI graph enables + `toml/preserve_order` through the integrations crate's direct feature request, + while adapter graphs consume only explicit stored sidecars and never depend + on TOML or JSON object iteration order. Running the storage round trip with + and without `serde_json/preserve_order` yields identical runtime order. - Nested provider declaration order is retained. - A parent integration or provider table declared after one of its descendants fails before typed deserialization. @@ -1643,29 +2145,44 @@ to preserve `cfg(test)` imports. - TOML-to-envelope-to-runtime round trips preserve both order sidecars exactly, independent of JSON object-member order. - Order sidecars reject missing, duplicate, extra, or mismatched map keys. -- Schema-1 stored blobs normalize through the transition reader; unknown schema - values fail before secrets are resolved. +- Schema-1 stored blobs normalize through the transition reader; schema N+1 and + every other unknown schema value fail before secrets are resolved. Golden + rollback tests prove every binary retained for the rollback window rejects a + schema-2 root marker and cannot mistake it for schema 1. +- The shared schema-one converter preserves omitted integration defaults, + ignores baseline-accepted unknown integration IDs with one rate-limited + warning, retains local external IDs, and reproduces globally interleaved + lexical provider priority before qualification. - Config-store loading produces the same registry, JavaScript, and provider order that the CLI validated. - Registry metadata and CLI provider diagnostics report configured ordinals; lookup-map or alphabetic iteration cannot masquerade as execution order. -- CLI validation never constructs runtime capabilities from unresolved secret - key names; the post-resolution runtime phase rejects unresolved values. +- CLI validation compiles and discards the complete secret-independent plan, + including routes, bidder ownership, mediator capability, and duplicate + claims, without constructing runtime capabilities from unresolved secret key + names. Diff/push fail selected-target validation. Validate reports a result + for every supported target declared in the manifest, and `--strict` fails on + any target rejection. Every secret-independent runtime rejection has a + matching target-neutral error or named-target CLI fixture. - Read-only CLI consumers see the effective overlay through - `ValidatedSourceConfig`; mutators and generators operate on file-only bytes, - retain invalid-baseline recovery where currently supported, apply the same - complete validation to baseline and candidate, and never persist overlay - values. Invalid candidate plus valid baseline refuses; two invalid values - retain the current non-disclosing warning and write escape hatch. -- Validate/diff/push delegate exact private snapshots of both the config and - manifest bytes used by the pre-pass; concurrent edits cannot substitute - different pushed bytes or change the adapter/store target. Snapshot paths are - absent from user-facing success and error output. + `SourceConfigView` without deploy validation; deploy commands use + `ValidatedSourceConfig` after the overlay. Mutators and generators operate on + file-only bytes, retain invalid-baseline recovery where currently supported, + and never persist overlay values. Invalid candidate plus valid baseline + refuses; two invalid values retain the current non-disclosing warning and + write escape hatch. +- Validate/diff/push use the EdgeZero typed-config extension, proving the + pre-pass, command/target validation, and typed serialization consume one exact + read and one effective typed value even under concurrent source replacement. + No command creates config or manifest snapshots. - Prebid bundle selection and managed-module requirements come from the - integration-owned partial source view, not a CLI-local schema or hard-coded - registry path. Bundle modules and generated hash/SRI metadata can be staged - without an external URL, remain runtime-inert, and are patched atomically - over an otherwise-invalid unrelated baseline. + integration-owned `PartialSourceConfigView`, not a CLI-local + schema or hard-coded registry path. Bundle modules and generated hash/SRI + metadata can be staged without an external URL, remain runtime-inert, and are + patched atomically over an otherwise-invalid unrelated baseline. The command + fails with a placement example when the explicit `[integrations.prebid]` + parent is absent and never appends a parent that silently chooses runtime + order. - Disabled integrations may retain structurally valid provider settings, contribute no providers or capabilities, and do not reorder enabled neighbors. - Disabled placeholder values that current examples rely on, including the @@ -1675,8 +2192,9 @@ to preserve `cfg(test)` imports. store-reference, collision, and adapter validation; omitted active-only references are accepted, and inactive paths are not value-resolved at runtime. -- Bidder and mediator references to disabled integrations fail, including while - the global auction is disabled. +- Bidder and mediator references to explicitly disabled known integrations are + omitted from the compiled plan with one warning, including while the global + auction is disabled; absent and unknown targets still fail. - Nested-only, unknown, missing-enabled, descendant-before-parent, and mixed old/new configurations fail with actionable messages. - Qualified provider references resolve correctly and reject missing or @@ -1686,6 +2204,8 @@ to preserve `cfg(test)` imports. with stable qualified identities. - Provider launch, response, mediator-input, and local equal-price tie order follows integration then local-provider declaration order. +- Every later launch receives its actual remaining logical budget, and tests + cover adapter timeout bucket boundaries even without deadline expiration. - Transport-failure recovery paths retain plan order and never fall back to lexical provider sorting. - APS, Prebid, and `openrtb` fixtures cover enabled parents, disabled-parent @@ -1695,27 +2215,49 @@ to preserve `cfg(test)` imports. standard OpenRTB, and the `adserver_mock` mediator. - Qualified IDs that alias under Axum's legacy normalization receive distinct correlation names or fail target validation before deployment. +- `ts config migrate --dry-run` preserves comments and permissions, emits + frozen explicit defaults and environment-variable path mappings, handles + server-only Prebid and implicit APS, reports priority changes, rejects mixed + input, and never performs a remote write. +- Ordering diagnostics cover provider details, ts-debug, telemetry `is_win`, + mediator `ext.bidder_responses`, backend names, and CLI provider listings in + addition to launch and response order. ### Capability and behavior parity tests +- A focused differential harness captures the `a4e01eb55` baseline for a matrix + covering all integrations, APS/Prebid/standard providers including globally + interleaved IDs, deferred Prebid, GPT diagnostics, and the #1135 Next.js + streaming path. It compares hook order, module ID lists, head inserts, trusted + tag attributes, route tables, plan/provider order, mediator input order, and + relevant CLI views. It compares semantic IDs/order rather than rebuilt bundle + bytes and records the explicit milestone deltas separately. - The static catalog contains all current integration IDs plus `openrtb`. - APS and Prebid register page/browser and auction capabilities without core importing their types. - `adserver_mock` registers and is selected through the mediator capability. - Route tables and duplicate detection retain behavior. -- DataDome privacy and buffering behavior remains unchanged. +- DataDome preserves unconditional HTML origin-representation handling without + selecting buffering, and scopes privacy to body-carrying synthesized HTML. - GPT diagnostics preparation, bootstrap injection, finalization, and caching remain unchanged on every adapter path. -- GPT diagnostics preparation is idempotent across adapter preparation and the - core publisher safety net, and auction correlation uses neutral registry - request state rather than a concrete GPT import. +- GPT-diagnostics reserved query/cookie normalization runs and remains + idempotent when the integration is absent, disabled, or enabled. All four + adapters cover multiple Cookie fields and non-ASCII bytes before generic + cookie handling. Activation-dependent preparation/finalization stays gated, + while auction correlation uses deployment capability rather than request + activation. - APS and Prebid request construction, transport, parsing, response admission, and auction results remain equivalent to current `origin/main` behavior - except for the documented provider-priority change. + except for the documented provider-priority, external-identity, and shared + browser-state corrections. - Prepared response parsers consume profile-owned request state without `Any`, downcasts, vendor enums, or cross-provider state reuse. +- Prepared outcomes cover both a bound exchange and no-impressions `Skip`, and + renderer descriptor JSON plus targeting bid identity remains wire-compatible. - Bidder routing, backend naming, notification suppression, telemetry identity, - and mediator behavior remain equivalent apart from documented ordering. + and mediator behavior remain equivalent apart from documented ordering and + schema-2 qualified-identity changes. - Active DataDome secrets are presence-checked and resolved through unchanged object paths; inactive protection and bypass secrets are removed before the shared resolver. @@ -1726,14 +2268,20 @@ to preserve `cfg(test)` imports. partner ingestion, and admin diagnostics without adding a LiveRamp catalog definition or an identity-provider capability system. - Request-processing requirements preserve DataDome origin bypass, full-body - buffering, and final private caching, and prevent request-private GPT - diagnostics state from entering ESI templates. + representation, streaming/buffering selection, scoped private caching, and + origin-readthrough vetoes, and prevent request-private GPT diagnostics state + from entering ESI templates. - Warm-hit and cold-store matrices combine DataDome/GPT requirements with key, bypass, malformed, unlisted, absent, and empty cookies; integration privacy can only restrict the existing cookie/cache decision. - The dedicated mediator capability preserves request construction, ordered - response input, bounded transport, parsing, and local-ranking fallback without - exposing the legacy `AuctionProvider` trait. + response input, bounded transport, parsing, and launch/parse-error fallback + without exposing the legacy `AuctionProvider` trait. A parsed non-2xx error + response retains its current zero-winner/no-fallback behavior. +- Fastly's JA4 gate and failed-startup finalization still receive the + settings-only view when full composition fails; reusable capability objects + are immutable and request-stateless, with document buffers created per HTML + processor. ### Browser tests @@ -1743,9 +2291,11 @@ to preserve `cfg(test)` imports. duplicate state-owner signatures. Permutive registration through its IIFE is visible to core context collection; Prebid auction helpers and Testlight queue behavior still use the single installed runtime. -- Colliding DOM-insertion handlers run in configured order, and reversing two - integration tables reverses their winner. Numeric priority and lexical - handler ID cannot affect the result; a deferred DOM handler fails composition. +- Built-in DOM-insertion guards claim disjoint parsed host/path sets with + exactly one claimant per canonical URL. Composable handlers run in immediate + IIFE registration order; numeric priority and lexical handler ID cannot affect + the result, query-string substrings cannot steal ownership, and a deferred DOM + handler fails composition. - APS is absent from browser core and registers its renderer from its own IIFE. - Core, GPT, and Prebid artifacts contain no private copy of APS renderer state. - Existing APS validation, sandbox, messaging, timeout, and rendering tests pass @@ -1760,38 +2310,50 @@ to preserve `cfg(test)` imports. - Equal duplicate trusted script attributes collapse, while conflicting values fail composition before HTML is served. - Missing or rejecting renderers drop only the renderer-bearing bid with no - fallback or pre-acceptance side effect; duplicate registration poisons the - type or fails composition. + fallback or newly created route/DOM/message/response/beacon side effect; + baseline supersession and fail-closed event handling remain, and duplicate + registration poisons the type or fails composition. - Both server-bid and Prebid-`adId` renderer paths cover carrier scrubbing, failed admission/registration, bounded TTL and capacity, authenticated source and slot binding, atomic consume, replay rejection, renderer-owned Universal Creative response metadata, and `markWinningBidAsUsed` preservation. - Unified, deferred, standalone, and inline assets expose bytes and hashes that match the emitted document fingerprint and static responses. -- Canonical composition-digest golden vectors are stable across construction - order for unordered maps/sets, change when configured integration/provider - order changes, and distinguish active resolved secrets from retained inactive - references without exposing either input. +- Composition-digest vectors combine the verified EdgeZero data hash, + document-assets fingerprint, schema, and build ID using the specified raw-byte + and length framing; reject hex-text or ambiguous concatenation variants; + change when configured integration/provider order changes; never include + resolved secret values; and are lazily memoized. Rollout activation forces + and logs the digest once per snapshot, while ordinary non-document requests do + not. Set-valued source fields serialize deterministically. - Changing any integration setting that affects generated head output changes the document fingerprint; request-dependent head variation bypasses shared template reuse through processing requirements. - GPT bootstrap, APS renderer document, Sourcepoint trap, GPT-diagnostics - bootstrap, and DataDome/Didomi/GPT/Prebid/Sourcepoint inline templates resolve + bootstrap, Sourcepoint response patches, and + DataDome/Didomi/GPT/Prebid/Sourcepoint inline programs resolve byte-for-byte from their integration-owned package locations. A guard rejects handwritten - production integration browser algorithms in Rust string literals. + production integration browser algorithms in Rust string literals without + introducing a general template framework. - External Prebid artifacts preserve bidder, User ID, and analytics category selection, manifest/hash/SRI generation, managed-name alias and collision checks, `identityLinkIdSystem` requirements, consent behavior, and runtime codes after registry and shim paths move. -- Owner-specific output directories and manifests reject stale or partial - output, and concurrent neutral/integration Cargo builds exercise the one - cross-process toolchain lock. +- Owner-specific private `$OUT_DIR` directories and manifests reject stale or + partial output; concurrent neutral/integration Cargo builds share only a lock + around dependency mutation and cannot discover one another's output. - Clean and incremental Cargo builds prove that changing a sibling integration source reruns the integration embed build and changes its manifest/hash while leaving an unrelated neutral artifact unchanged. - Cross-adapter Playwright tests remain in `trusted-server-integration-tests` and verify core, creative, APS, GPT, and Prebid load order using the moved assets. +- An artifact test loads GPT bootstrap, core, immediate GPT, and deferred Prebid + in production order, proves object identity/listener idempotence and + first-impression preservation, and rejects an incompatible facade version. +- Sibling-root gates resolve `vitest`, `prebid.js`, and an exported Prebid module + through the canonical project; a failing typecheck canary proves moved type + tests are actually selected, and Prettier/ESLint use explicit canonical roots. ### Repository gates @@ -1803,9 +2365,13 @@ documentation formatting. The explicit package lists in every Fastly build/check/clippy/test alias include both new Rust crates where applicable; the CLI and codegen host lint gates remain intact; and a dedicated host gate runs catalog completeness and integration test support. The migrated Fastly-SDK -guard continues to scan the moved integration sources. +guard walks the complete moved integration directory and checks the dependency +graph rather than copying the baseline's incomplete handwritten file list. +Each crate/source move updates these selectors in the same change; a coverage +gate proves moved Rust and JavaScript tests were discovered. -The path migration covers repository automation as well as compiled code: +The path migration covers repository automation as well as compiled code and +paths assembled dynamically at runtime: GitHub workflows and PR templates, Dependabot, `AGENTS.md`, `.claude` agents and commands, the CLI Prebid builder, browser and template-cache smoke scripts, TypeScript/Vitest/format/lint configuration, crate READMEs, reader-facing docs, @@ -1818,7 +2384,39 @@ archived specs are not rewritten as though they described the new layout. A repository path guard rejects active code or tooling that still points to `trusted-server-js/lib/src/integrations` or concrete `trusted-server-core/src/integrations/` paths, except for an explicitly -allowlisted historical reference. Dependabot remains rooted at the one lockfile. +allowlisted transitional or historical reference. It checks string literals and +known path joins, not only exact static paths. Dependabot keeps one entry for +the browser lockfile; unrelated Node projects retain their own entries. The +Cargo graph also records the direct `trusted-server-openrtb` +and integration-test dependencies explicitly and removes unused direct browser +crate dependencies from adapters. + +## Baseline Defects and Scope Containment + +The review found baseline defects that this design must not accidentally encode +as desired architecture. They land as focused prerequisite commits or as the +explicit milestone corrections named above, not as unrelated additions to the +crate-move pull requests: + +- replace nondeterministic serialization of + `auction.allowed_context_keys` with a deterministic set before using the + EdgeZero data hash for deployment or template identity; +- make generic Cookie handling robust without depending solely on an enabled + diagnostics integration, while retaining reserved-input removal and Cookie + merge semantics in the catalog normalizer; +- unify the browser context/logging/renderer state behind the adopted runtime + facade and pin the intentional behavior changes with artifact tests; +- preserve baseline acceptance of unknown integration IDs only in schema one, + while schema 2 rejects them; +- chain GTM/Next.js script rewriting over current text and replace Lockr's + substring claim with parsed URL ownership before configuration order becomes + authoritative. + +Other baseline issues discovered during review—Prebid bundle file mode, stale +Fastly guard lists, shared `dist` races, unused dependencies, and mediator +non-2xx test coverage—receive focused fixes or migration-gate updates. They are +not reasons to add a plugin system, a second configuration inventory, or an +unrelated public API to this design. ## Risks and Mitigations @@ -1855,10 +2453,10 @@ only the TOML parser. An old binary cannot consume application schema 2, and a binary rollback after config cutover would otherwise fail at startup. -Mitigation: deploy the dual reader before pushing schema 2, archive the prior -envelope, fence schema-1 CLI writers during and after cutover, assert the stored -schema after the push, require restoration before binary rollback, and remove -the legacy reader only in a later release. +Mitigation: use the adapter-specific version/store isolation table, archive a +complete schema-one generation, deploy the dual reader before schema 2, fence +old writers, prove the active binding and settings-dependent behavior, and +remove the legacy reader only after every platform's rollback drill passes. ### Configuration order silently changes auction priority @@ -1880,23 +2478,23 @@ public item. CLI validation and adapter startup could use different catalogs or schemas. -Mitigation: both call the same catalog-backed source-validation API, and only -adapter startup continues through the post-secret runtime composition API. No -secondary validation inventory is allowed. Adapters receive the already -composed settings, plan, registry, browser assets, and configuration digest -rather than reconstructing any of them. Non-runtime CLI commands consume the -validated source view rather than deserializing integration fragments locally. +Mitigation: structural read-only commands use `SourceConfigView`; deploy +commands and adapters share the complete secret-independent validation kernel; +only adapter startup continues through resolved-secret construction. No +secondary validation inventory is allowed. Adapters receive the settings, +plan, orchestrator, registry, browser assets, target result, and digest rather +than reconstructing them. ### Stale or incorrectly ordered browser artifacts Splitting Rust ownership while sharing one Node workspace can embed previous output, race build scripts, or load APS too late. -Mitigation: hold one cross-process lock across install, cleanup, build, -discovery, manifest verification, and copy; use owner-specific output -directories; retain stale-output refusal; hash built bytes; load core and the -creative prelude first; reject deferred APS composition; and run artifact-level -renderer, fingerprint, and ordering tests. +Mitigation: write each build directly into a private owner `$OUT_DIR`, lock only +dependency mutation, configure every sibling-root resolver/tool explicitly, +retain stale-output refusal, hash built bytes, load core and the creative +prelude first, reject deferred APS composition, and run artifact-level renderer, +fingerprint, version-skew, and ordering tests. ## Acceptance Criteria @@ -1905,11 +2503,11 @@ The change is complete when: 1. Both new crates are workspace members and statically linked by the CLI and every adapter where required. 2. All fifteen current concrete Rust implementation units live under - `trusted-server-integrations/src//`. + `trusted-server-integrations/src/integrations//`. 3. The standard provider configuration is supplied by the built-in Rust-only `openrtb` integration, making sixteen static definitions in total. 4. All integration-owned browser sources, assets, unit/artifact fixtures, and - unit/artifact tests, including production executable inline templates, live + unit/artifact tests, including production executable inline programs, live under `trusted-server-integrations-js`; cross-adapter system tests remain in the integration-test crate. 5. Rust definitions use one explicit compile-checked catalog with a directory @@ -1925,37 +2523,46 @@ The change is complete when: 10. The old `[auction.providers]`, `profile`, `profile_config`, and fixed `protocol` fields are rejected with targeted migration guidance. 11. The compiled auction plan retains current-baseline behavior after - normalization, except for the explicit change from lexical to - configuration-order provider priority. + schema-one normalization through an independent flat provider sequence; + schema 2 makes the explicit change from lexical to configuration-order + provider priority. 12. Browser core imports no concrete integration, and APS rendering works through one immediate registration without private copies in core, GPT, or - Prebid bundles; the complete shared-state facade and DOM dispatcher obey - configuration order; and GPT retains its synchronous-tag bootstrap - contract. -13. `TrustedServerAppConfig`, `ValidatedSourceConfig`, integration secret + Prebid bundles; the shared-state facade adopts GPT bootstrap state, DOM + ownership is disjoint, composable handler registration obeys configuration + order, and GPT retains its synchronous-tag bootstrap contract. +13. `TrustedServerAppConfig`, `SourceConfigView`, + `PartialSourceConfigView`, `ValidatedSourceConfig`, integration secret handling, and final runtime composition are owned by `trusted-server-integrations`; core has no concrete config or loader - dependency, the CLI has no duplicate integration schema, and host-only - EdgeZero wrappers remain in the CLI rather than the WASM-linked crate. + dependency, the CLI has no duplicate integration schema, and the host-only + TOML pre-pass is feature-gated out of WASM graphs. EdgeZero's typed-config + source/command extension replaces filesystem snapshots and all EdgeZero + dependencies share its reviewed immutable revision. 14. The CLI source phase and adapter runtime phase use the same catalog-backed schemas and pure validation definitions, and all adapters receive one post-secret-resolution - settings/plan/registry/browser-assets/configuration-digest composition. + settings/plan/orchestrator/registry/browser-assets/target/digest + composition plus the preserved settings-only degraded view. 15. OpenRTB request-local state crosses the transport boundary through a - prepared response parser without `Any` or vendor enum variants in core. + prepared exchange-or-skip outcome and bound response parser without `Any` + or vendor enum variants in core; renderer descriptors retain their wire + shape through a neutral type. 16. Explicit `enabled`, parent-before-descendant, disabled-retention, activation matrix, local-ID, and qualified-ID rules have end-to-end tests, including a - server-only Prebid migration that does not activate browser behavior. + server-only Prebid migration that does not activate browser behavior and + known-disabled reference pruning that acts as a kill switch. 17. Schema-1 blobs remain readable for the documented rollout release, schema-2 - blobs preserve existing secret paths, and binary rollback requires verified - restoration of the archived schema-1 envelope through a drill-tested - adapter-specific mechanism; old schema-1 CLI writers are fenced after - cutover and stored-schema regression is monitored as a rollback event. -18. Browser assets carry bytes and hashes through composition; publisher - template fingerprints combine the versioned canonical composition digest - with the exact document-assets fingerprint; all existing URL, host, scheme, - origin, assembly, Vary, cookie, and schema-version key dimensions remain; - and request-dependent variants bypass shared reuse. + blobs preserve existing secret paths, and binary rollback uses drill-tested + adapter-specific version/store isolation or verified restoration where + isolation is unavailable; old schema-1 CLI writers are fenced after cutover + and stored-schema regression is monitored as a rollback event. +18. Browser assets carry bytes and hashes through composition; the versioned, + unambiguously framed composition digest already includes the exact + document-assets fingerprint and is included once in publisher template + fingerprints; all existing URL, host, scheme, origin, assembly, Vary, + cookie, and schema-version key dimensions remain; and request-dependent + variants bypass shared reuse. 19. Core test support, the Fastly-SDK migration guard, Cargo aliases, CI, Dependabot, repository automation, browser and cache smoke scripts, documentation and snippet tests, and the CLI Prebid builder cover the new @@ -1969,6 +2576,13 @@ The change is complete when: another integration definition or provider framework. 22. Both milestone exit criteria and the full repository verification gates pass. +23. `ts config migrate --dry-run` produces a validated schema-2 candidate, + reports defaults, environment-overlay renames and priority changes, and + never writes remotely. +24. Reserved GPT-diagnostics inputs and malformed Cookie fields are normalized + on baseline routes even when diagnostics is absent or disabled. +25. The adapter-specific rollout proves schema/binding state through logs and a + settings-dependent probe without adding a public status endpoint. ## Deferred Work @@ -1980,5 +2594,6 @@ The following require separate designs and real consumers: - New identity, EC, geo, device, and permission-signal provider systems. - Permission and jurisdiction policy changes. - Non-OpenRTB auction provider factories. -- Upstream EdgeZero composition and host-service changes. +- Upstream EdgeZero composition and host-service changes beyond the narrow + typed-config source/command extension required above. - Moving CLI audit detection metadata into integration directories. From 3fedcf75f56e2be8b2ee17cc4d01a4841db58af5 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 24 Sep 2026 19:55:04 -0700 Subject: [PATCH 11/13] Address second review of integrations split design --- ...-09-17-split-integrations-crates-design.md | 927 ++++++++++++------ 1 file changed, 645 insertions(+), 282 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index 38b95b0d9..4867d51a2 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -46,11 +46,15 @@ discovery order will never affect execution. For auction providers, that order is also operational priority: it controls launch and response order, mediator input order, and local equal-price tie-breaking. -The compatibility baseline is the behavior shipped on `origin/main` at -`a4e01eb55`, not either prior pull request discussed below. This design changes -only the configuration, activation, and ordering behavior called out explicitly -in this document; all other current runtime, browser, CLI, and cache behavior is -preserved. +Repository discovery started from `origin/main` at `a4e01eb55`, not either +prior pull request discussed below. That commit contains the known P1 output +defects listed as prerequisites here and is not an acceptable compatibility +golden. Implementation branching is blocked until the prerequisite fixes land; +the recorded baseline is then the resulting `origin/main` commit plus captured +Next.js/GTM/RSC and browser-runtime output goldens. This design changes only the +configuration, activation, and ordering behavior called out explicitly in this +document; all other corrected-baseline runtime, browser, CLI, and cache behavior +is preserved. This remains one design, but it has two merge milestones. The crate and runtime boundary moves first without changing operator configuration. The ordered, @@ -182,11 +186,57 @@ PR #1135 is part of the current baseline. Its parser-aware streaming Next.js processor, test support, and cross-adapter parity case move with the integration; the removed HTML post-processor is not recreated by this work. -Before either milestone branches for implementation, its baseline commit and -locked EdgeZero revision are recorded again and every baseline-dependent -inventory in this document is rechecked. Open or previously approved pull -requests never override a decision in this specification merely because of -their review status. +### Open Defect and In-Flight Work Disposition + +The following items are open as of 2026-09-24. They are coordination inputs, +not design authorities. “Prerequisite” means the focused fix lands on `main` +and this specification records a new baseline before extraction begins; the +crate split does not absorb that bug fix into a move commit. + +| Item | Disposition | +| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| #1196 | Prerequisite. Restore cross-IIFE context/log state on the current layout, remove or debug-gate Creative's unconditional log-level bump, and add a production-artifact test. The milestone-one typed facade supersedes any transitional `Symbol.for` storage without reverting behavior. | +| #1197 | Prerequisite. Make set-valued configuration serialization deterministic before EdgeZero or template hashes depend on it. | +| #1198 | Prerequisite. Add the deterministic build digest defined below to the current template key before extraction; the split reuses the same contract. | +| #1199 | Prerequisite. Replace substring DOM ownership with parsed host/path ownership on the current layout. | +| #1200 | Prerequisite. Remove the shared-`dist` partial-build race before two Rust crates consume browser outputs; milestone one then adopts owner-private outputs and the lock contract below. | +| #1201 | Milestone two. New source and schema 2 reject unknown catalog IDs and fields. The temporary schema-1 reader deliberately retains baseline acceptance with a warning and is not “fixed” retroactively. | +| #1202 | Prerequisite. Make `ts prebid bundle` use the permission-preserving atomic writer and non-disclosing parse errors; the moved command retains that corrected behavior. | +| #1203 | Prerequisite and adopted decision. A parsed non-2xx mediator response is a mediation failure and falls back to local ranking as specified below. | +| #1204 | Prerequisite. Every Rust CI job that can trigger a browser build installs the pinned Node toolchain and dependencies; obsolete direct Rust browser dependencies are removed. | +| #1205 | Proposed remedy superseded. This specification keeps operator order and uses the pure script-source claim validation below instead of an unconditional hidden first phase. The issue must be revised to that contract or closed. | +| #1206, #1207 | Prerequisites resolved by #1208. Their output regressions are not accepted as differential-harness goldens. | +| #1208 | Prerequisite. Compose script-text rewriters on the current layout and add Next.js/GTM output goldens before the baseline is captured. | + +PR #1052 may contribute neutral renderer diagnostics, but APS-named browser +core members do not enter core; they move with APS or become generic renderer +reason fields. PR #1175 and EdgeZero PR #381 are rollout prerequisites: the +typed-config extension and Fastly store procedure build on their reviewed +environment-selector line or an equivalent merged successor, never on v0.0.8. + +This specification partially supersedes the following earlier design documents +only where they conflict with the new ownership, configuration, or fingerprint +contracts: + +- `2026-08-10-config-first-auction-provider-architecture-design.md`: its + provider inventory, selection, and ordering syntax are superseded; reusable + neutral auction-engine findings remain evidence to reverify. +- `2026-07-15-gam-ts-cohort-attribution-design.md` and + `2026-08-19-gam-attribution-review-resolution-design.md`: attribution wire + behavior remains, while concrete GPT code and assets move to their integration + owner. +- `2026-09-08-1138-per-cookie-template-cache-policy-design.md`: cookie/Vary + policy remains, while complete-settings fingerprinting is replaced by the + composition/build digest defined here. +- `2026-07-24-prevent-duplicate-gpt-slot-requests-design.md`: duplicate-slot + behavior remains, while concrete GPT state and browser ownership move outward. + +Before either milestone branches for implementation, all prerequisite issues in +the disposition table must have landed, then the baseline commit, output +goldens, and locked EdgeZero revision are recorded again and every +baseline-dependent inventory in this document is rechecked. Open or previously +approved pull requests never override a decision in this specification merely +because of their review status. ## Rejected Alternatives and Rationale @@ -219,6 +269,28 @@ implementation cannot quietly reintroduce the rejected mechanism. **Decision:** `[integrations.]` selects the statically cataloged integration, while nested instance names identify reusable provider configurations. Capability types remain Rust contracts, not configuration. +- **Keeping local provider IDs as the permanent global identity.** Stable + external values during the dual-reader rollout are required; that concern is + accepted. **Objection:** local IDs can collide across owning integrations and + cannot safely key one cross-integration plan. **Decision:** the plan always + uses `QualifiedProviderId`, while an explicit external label preserves local + schema-1 values and changes to qualified values only at the schema-2 cutover. +- **Keeping concrete browser integrations in the neutral JS crate or splitting + them into one package per vendor.** One tested Node dependency graph is + required. **Objection:** leaving vendor sources in the neutral crate preserves + the ownership inversion, while per-vendor packages multiply lockfiles, + resolution rules, and release surfaces without independent consumers. + **Decision:** use one `trusted-server-integrations-js` Rust/source crate for + concrete integration browser code and one canonical Node project shared with + neutral browser core. +- **Keeping the existing mediator trait object unchanged.** One reusable + mediator implementation and stable fallback behavior are required. + **Objection:** the existing object couples configuration selection, request + preparation, transport, response parsing, and provider-specific state, which + forces concrete knowledge and downcasts across the core boundary. + **Decision:** use a typed mediator registration, prepared request, and bound + response parser while retaining the existing wire protocol and the corrected + fallback policy. - **An operator-written `[auction] provider_order` list.** Provider priority must represent schema-1's globally interleaved lexical order; that concern is accepted. **Objection:** a second operator list would duplicate every provider @@ -235,9 +307,11 @@ implementation cannot quietly reintroduce the rejected mechanism. order contract false and still would not reproduce baseline Prebid removal behavior, because Prebid precedes `js_asset_proxy` today. **Decision:** legacy compatibility paths reproduce the complete baseline sequence. Schema 2 - exposes chained replacement and terminal removal in declaration order and - diagnoses known overlaps rather than silently overriding the operator's - order. + exposes chained replacement and terminal removal in declaration order. Pure + per-definition script-source claims make an earlier conflicting owner a + validation error, so the operator must place `js_asset_proxy` first for a + deliberate per-URL override instead of receiving a silent order-dependent + bypass. - **Treating PR #1016, PR #1084, or earlier review statements as design authority.** Their code and tests can reveal compatibility constraints, and conflicting in-flight work needs an explicit disposition. **Objection:** @@ -254,10 +328,11 @@ implementation cannot quietly reintroduce the rejected mechanism. project with explicit, tested resolver and tool-root configuration for the sibling sources. Moving the root remains a fallback only if that contract cannot be made reliable. -- **A new public configuration-status endpoint.** Runtime settings must be +- **A new configuration-status endpoint, public or authenticated.** Runtime settings must be loaded and the expected schema must be observable during rollout; that - concern is accepted. **Objection:** a new endpoint would add an authentication - and public-API surface unrelated to the crate split. + concern is accepted. **Objection:** even an authenticated endpoint would add + a new route, authorization contract, and public support surface unrelated to + the crate split. **Decision:** use adapter-native version/binding inspection, startup schema-and-digest logging, and an existing authenticated or settings-dependent probe. @@ -357,12 +432,13 @@ The following terms are distinct: - **Integration configuration:** the single ordered operator block at `[integrations.]` that activates and configures the definition. - **Auction provider instance:** one named endpoint and policy configuration - below an integration, such as `aps.main`. Multiple instances may use the same + below an integration, such as `aps.aps-main`. Multiple instances may use the same integration implementation. - **Local provider ID:** the provider name within one integration, such as - `main`. + `aps-main`. - **Qualified provider ID:** the strong, globally unique pair of an integration - ID and local provider ID, serialized as `.`. + ID and local provider ID, such as `aps.aps-main`, serialized as + `.`. - **Integration registry:** core runtime state containing the enabled page, request, response, and browser capabilities in configuration order. - **Auction plan:** core runtime state containing the validated configured @@ -523,6 +599,11 @@ not added to core's `Settings`. Composition consumes the integration portion into capabilities and returns a neutral runtime `Settings` value containing only state that core execution engines understand. +The public-API snapshot classifies exports as stable neutral contract, +integration-owned facade, or milestone-one transitional delegation. A +transitional export must name its replacement and owning move step; adding one +requires review, and the class must be empty before milestone 1 exits. + The public API has four explicit levels: 1. `SourceConfigView` performs the TOML pre-pass, typed parse, catalog @@ -550,14 +631,16 @@ The public API has four explicit levels: This distinction makes "runtime-only construction" precise: it does not move any currently push-time, secret-independent plan failure to startup. Diff and -push have one selected adapter and fail when its target validation fails. -`config validate` has no adapter selector: it runs target validation for every -supported adapter declared by the manifest and prints a named result per target. -Ordinary validate fails on target-neutral errors and reports target-specific -failures as warnings so a valid Fastly configuration is not rejected merely -because the same multi-provider plan cannot run on Cloudflare or Spin; -`config validate --strict` fails if any declared target fails. A fixture rejected -by runtime composition for a secret-independent reason must produce the same +push have one selected adapter and fail when its target validation fails. The +existing EdgeZero `config validate --strict` meaning is unchanged: it enforces +the manifest-completeness and handler-path checks already owned by EdgeZero. +Trusted Server target-plan validation is exposed separately as a repeatable +`ts config validate --target ` option. With no `--target`, the command +runs target-neutral Trusted Server checks plus existing EdgeZero validation; +with one or more targets, every selected target is checked and any target +failure makes the command fail. Auction-testing documentation and smoke +commands use this same target spelling. A fixture rejected by runtime +composition for a secret-independent reason must produce the same target-neutral error or the same named target result in the CLI. The complete source and validated views retain the typed operator configuration @@ -602,12 +685,15 @@ verified-data boundary. The integration crate composes it in this order: ```text config-store bytes → core chunk reconstruction and envelope verification - → settings-only neutral view + → application-schema dispatch + → schema-2 sidecar/map bijection or schema-1 normalization → core-owned neutral/global inactive-secret preprocessing → catalog-owned integration inactive-secret preprocessing → aggregated core + integration secret resolution - → catalog-aware config validation + → resolved settings-only neutral view + → catalog-aware resolved-value validation → core AuctionPlan compilation + → target validation → plan-dependent capability construction → core IntegrationRegistry and orchestrator construction → browser asset composition and document fingerprint @@ -622,16 +708,17 @@ dependency; depends on `edgezero-cli`. The current locked EdgeZero revision does not expose enough context for this -contract. EdgeZero therefore gains one narrow typed-config extension with two -default no-op stages. The source stage receives the selected app-config path and -exact raw bytes before deserialization. The effective-config stage receives a -borrow of the overlay-applied typed value plus the command kind and target -context EdgeZero already resolved: one adapter for diff/push and the manifest's -declared supported adapter set for validate. It does not select or mutate a -target. EdgeZero reads the file once, runs the source-aware pre-pass, constructs -one typed value, invokes command validation on that value, and serializes that -same value. Validate, diff, and push therefore cannot validate one app-config -read or typed value and serialize another. +contract. Building on EdgeZero PR #381 and Trusted Server PR #1175, EdgeZero +therefore gains one narrow typed-config extension with two default no-op +stages. Target resolution occurs before either hook. One exact source read is +shared by the source pre-pass, environment overlay, secret checks, typed +callback, diff, and serialization; a callback cannot reopen the path. Hook +context contains the command kind, the one selected target for diff/push or +the explicitly requested target set for validate, app environment-variable +prefix, `--no-env`, EdgeZero's existing `strict` flag, raw path and bytes, and a +borrow of the overlay-applied typed value. EdgeZero invokes command validation +and serializes that same value. Validate, diff, and push therefore cannot +validate one app-config read or typed value and serialize another. All workspace `edgezero-*` dependencies are then repinned together from v0.0.8 to one immutable, reviewed tag or commit containing this extension. The @@ -641,7 +728,10 @@ replaces the filesystem snapshot wrapper entirely: config commands do not write temporary operator or manifest copies, require a writable checkout, rewrite logged paths, or rely on destructor cleanup around `process::exit` and signals. If the extension cannot land and the workspace cannot repin to its reviewed -revision, milestone 2 is blocked rather than restoring the snapshot design. +revision, milestone 1 is blocked rather than restoring the snapshot design. +Push-time selected-target validation is an explicit milestone-one CLI delta +needed to place all deploy validation behind the new composition root; it does +not enable schema 2 or configuration-order semantics. Read-only `config ad-templates` and `audit ad-templates` commands load the effective `SourceConfigView` with the existing optional environment overlay when @@ -660,6 +750,11 @@ an already-invalid baseline retains the current warning and atomic-write escape hatch without disclosing source values. Final validate, diff, and push always use the complete `ValidatedSourceConfig` path. +The migration updates ad-template lint/help text and `--explain` rule counts in +the same commit so neither describes `[auction.providers]` or reports a stale +number of checks. Documentation-snippet and exact-message tests pin the new +integration-owned paths and counts. + `ts prebid bundle` obtains typed bidder, User ID, analytics, and managed-module requirements through an integration-owned `PartialSourceConfigView` rather than a duplicate CLI schema. It @@ -667,15 +762,20 @@ intentionally does not require unrelated app configuration or an `external_bundle_url` to be deploy-valid: `bundle.modules`, `external_bundle_sha256`, and `external_bundle_sri` are inert staging metadata, while `external_bundle_url` activates the browser capability. The command -retains its current ability to build first, atomically patch hash/SRI metadata, -and tell the operator to upload and set the URL. It validates the structural +retains its current ability to build first, patch hash/SRI metadata through the +permission-preserving `write_file_atomically` path, and tell the operator to +upload and set the URL. It validates the structural pre-pass and affected Prebid subtree before writing; full app validation remains the contract of config validate/push. Because parent position is runtime order, the command no longer invents or appends a missing `[integrations.prebid]` parent. It requires an existing parent with explicit `enabled`, or exits with a placement example; it may create only owned descendants beneath that parent. Provider diagnostics display qualified providers in configuration order rather -than alphabetizing a detached map. +than alphabetizing a detached map. TOML parse failures report only the path and +line/column; diagnostics never echo source lines or configuration values. The +command canonicalizes both the Node package root and every integration-owned +source input and rejects realpaths that do not belong to the same workspace and +worktree checkout. `ts audit generate` may retain detector and edit metadata for concrete integrations under this design's audit-reorganization non-goal. That metadata is @@ -716,33 +816,45 @@ owned unit/artifact tests live under `trusted-server-integrations-js`. The shared configuration does more than include that sibling source root: - Vite and Vitest resolve bare packages through the canonical project's - exports-aware resolver and deduplicate stateful browser-core entry points. + exports-aware resolver. Runtime state is externalized behind the facade; the + build does not depend on bundler deduplication of stateful entry points. - TypeScript includes both source roots and supplies exact package paths where Node's ancestor walk cannot reach the canonical `node_modules`. -- Vitest names the sibling test directory, includes its runtime and type tests, - and has a deliberately failing typecheck canary so `ignoreSourceErrors` or an - empty glob cannot make the gate pass silently. -- ESLint uses an explicit common base path and source globs. Prettier always +- Vitest names the sibling test directory and includes its runtime tests. A + separate canary command intentionally typechecks a fixture with one expected + error and asserts that exact diagnostic, so an empty glob or disabled source + checking cannot pass silently. The canonical package provides a real + `npm run typecheck` script for both roots. +- ESLint is invoked from their common ancestor with explicit source globs; it + does not require moving `package.json`. Prettier always receives the canonical `--config` path for sibling files. - Generated external-Prebid entries live under the canonical project or use an explicit resolver that preserves the `prebid.js` package `exports` map; a directory alias that bypasses package exports is forbidden. +- Because the current `gpt_bootstrap.js` fails the canonical formatter and + legacy browser lint rules, its move preserves output behind one exact-path + Prettier ignore and one documented, exact-path ESLint legacy override. The + shared rules are not weakened; modernizing that file is separate work. Separate build targets emit neutral and integration artifacts directly into private owner-specific directories below their Cargo `$OUT_DIR` and validate per-target manifests before embedding them. They never clean, discover, or copy -from one shared `dist` directory. A cross-process lock covers dependency -installation or mutation only; completed dependency trees may be read by -parallel owner builds. This removes stale/partial discovery races without -serializing every adapter build. +from one shared `dist` directory. The dependency installer takes an exclusive +cross-process lock; Vite, Vitest, TypeScript, ESLint, Prettier, both embed +builds, and the CLI Prebid builder take a shared lock for their whole process, +so `npm ci` cannot remove `node_modules` beneath a reader. Owner manifests +record a digest of their complete source inputs and +reject stale output. This removes stale/partial discovery races while allowing +parallel readers. Every Rust and Node participant uses the same canonical lock +path and compatible shared/exclusive protocol; per-crate locks are invalid. The integration build discovers immediate directories containing `index.ts` and emits one IIFE per entry point. An IIFE may call the external versioned browser runtime facade and therefore is not described as self-contained. Its Cargo build embeds each output and its SHA-256 hash. CI, browser integration scripts, and the CLI Prebid builder use the single workspace root rather than maintaining a second -dependency graph; Dependabot continues to watch the one browser lockfile in -addition to the repository's unrelated Node projects. +dependency graph. Dependabot's existing browser entry continues to watch this +one lockfile; the existing docs entry is unchanged. Canonical npm build, typecheck, lint, format, and test commands include both source roots explicitly, and CI invokes those commands rather than core-only @@ -762,7 +874,20 @@ paths explicitly and has no hard-coded `src/integrations/prebid` assumption. The CLI resolves the canonical package root for dependencies and obtains the integration-owned input paths and typed module requirements from the integrations facade; it does not locate a registry through its own relative -path constant. +path constant. The facade returns repository-relative paths, never absolute +compile-checkout paths. The CLI resolves both the package root and inputs under +one selected runtime repository/worktree root, canonicalizes them, and rejects +any realpath that escapes or crosses that checkout. A two-worktree test proves +it cannot fall back to a compile-time `CARGO_MANIFEST_DIR` in the other checkout. + +When a browser build is required, a missing `npm` or failed dependency setup is +a hard error. `TSJS_SKIP_BUILD=1` is a local-development escape hatch only when +every expected owner manifest and artifact exists and its recorded input digest +matches; CI never sets it. Build scripts emit `rerun-if-env-changed` for +`TSJS_SKIP_BUILD`, `TSJS_TEST`, and every tool environment variable that changes +output, in addition to complete `rerun-if-changed` input coverage. Every Rust CI +job that can build either browser crate installs the repository-pinned Node +version and dependencies, satisfying prerequisite #1204. The generated Rust API exposes typed module identifiers rather than accepting unchecked strings. A Rust registration referencing a missing browser module @@ -813,6 +938,35 @@ definition; duplicate registration fails composition. Because provider tables have no profile discriminator, an integration that owns `auction.providers` must register exactly one OpenRTB profile capability. +Before constructing executable capabilities, each definition produces a pure +`IntegrationDeclaration` from its validated, unresolved-secret source view. It +contains provider/profile facts, mediator presence, browser modules, reserved +inputs, exact or pattern-based route claims, and ordered script-source claims. +These declarations are the shared input to CLI and runtime validation; they do +not allocate clients, read secrets, or execute a rewriter. + +A script-source claim is a pure +`claims_script_src(url) -> Replace | Remove | None` predicate with its owning +integration ID and source field. Native integration claims use the same parsed +URL/pattern semantics as their executable attribute rewriter. During schema-2 +validation, every `js_asset_proxy` asset URL is evaluated against claims from +earlier enabled integrations. Any earlier `Replace` or `Remove` conflicts with +the asset's `enabled` or `blocked` policy and fails validate, diff, push, and +startup, naming both integration tables and the URL. Placing `js_asset_proxy` +before the native owner is the explicit, visible per-URL override; terminal +blocking or replacement then follows the ordinary chain. Schema-1 compatibility +uses the frozen legacy sequence and does not apply new schema-2 overlap +strictness. `ts config migrate` evaluates its schema-2 candidate and refuses to +write a conflicting result until the operator reorders the parent or removes +the contradictory asset. + +Route claims likewise carry method plus canonical path/pattern ownership and +are derived from pure configuration, including Prebid `script_patterns` and +`js_asset_proxy` routes. Duplicate routes therefore fail in +`ValidatedSourceConfig` as well as runtime composition. A future integration +whose route shape depends on a resolved secret needs a separate contract; it +may not defer an otherwise source-visible route collision to startup. + The OpenRTB profile boundary has three stages: 1. `OpenRtbProfileDefinition` is the catalog-level capability. It supplies the @@ -876,13 +1030,15 @@ neutral boundary has three parts: normalized mediation result. Core retains mediator transport, deadline enforcement, telemetry, and the -existing outcome policy. Dispatch or parse errors warn and fall back to local -ranking. A successfully parsed non-2xx `adserver_mock` error response is a -mediator response with zero winners and does not fall back; that current -distinction is covered by a contract test. The current mediator endpoint policy, -including an allowed HTTP endpoint where supported today, is preserved. Its -response correlation remains the current mediator integration identity rather -than being silently rewritten as an auction-provider identity. +corrected prerequisite #1203 outcome policy. Dispatch errors, parse errors, and +all non-2xx responses are mediation failures: they warn and fall back to local +ranking. Non-2xx telemetry records stable reason `http_status`, the numeric +status, and `fallback_used = true`; it never fabricates a mediator winner, and +the original ordered provider outcomes and locally selected winner remain +attributed to their providers. The current mediator endpoint policy, including +an allowed HTTP endpoint where supported today, is preserved. Its response +correlation remains the current mediator integration identity rather than being +silently rewritten as an auction-provider identity. `adserver_mock` owns only request construction and response interpretation. The old mediator use of `Arc` is deleted when its last caller migrates. @@ -922,9 +1078,10 @@ capabilities or providers. Configuration cannot activate APS or Prebid through an auction plan while omitting its parent. References to an absent or unknown parent fail validation. References to an explicitly disabled, known integration remain in source as a kill switch: they are omitted from the compiled route or -mediator plan with one aggregated warning and ordered diagnostics, and become -subject to full validation again when the parent is enabled. No unknown ID is -silently treated this way. +mediator plan with one aggregated warning and ordered diagnostics. Their IDs, +references, and structure are still fully validated; only active-value +semantics resume when the parent is enabled. No unknown ID is silently treated +this way. Provider-owning integrations use the following explicit activation rules. They do not add capability names or implementation discriminators to operator @@ -944,11 +1101,13 @@ Thus a current server-only Prebid provider migrates beneath an enabled Prebid parent without activating Prebid's page/browser behavior. Browser-only Prebid continues to be selected by the same required external-bundle URL that current startup validation already uses. The closed browser-only source-field set is -`account_id`, `debug`, `script_patterns`, `client_side_bidders`, -`excluded_gam_ad_unit_path_suffixes`, and `managed_user_ids`. Any explicitly -configured or effectively non-default value in that set without +`account_id`, `debug`, `timeout_ms`, `script_patterns`, `client_side_bidders`, +`excluded_gam_ad_unit_path_suffixes`, and `managed_user_ids`. Any defaulted +typed value in that set that differs from its declared default without `external_bundle_url` fails validation rather than activating a partial browser -path; an implicit default script-pattern list alone does not activate or fail. +path. Validation is value-based, not source-presence-based: explicitly spelling +the default is equivalent to omitting it, and an implicit default script-pattern +list alone does not activate or fail. CLI-only `bundle.modules`, `external_bundle_sha256`, and `external_bundle_sri` may be staged without a URL and never activate runtime browser behavior. For APS, a configured provider necessarily activates its @@ -1062,9 +1221,9 @@ repeat under different integrations without collision. Provider identity uses three strong types rather than broadening the existing local identifier: -- `IntegrationId` matches `^[a-z][a-z0-9_]{0,62}$`. The underscore permits the - existing Rust module IDs such as `adserver_mock`; dots are forbidden. Its - maximum length is 63 ASCII bytes. +- `IntegrationId` matches `^[a-z][a-z0-9]*(?:_[a-z0-9]+)*$`, with a maximum of + 63 ASCII bytes. This permits existing IDs such as `adserver_mock` while + forbidding dots, leading/trailing underscores, and repeated underscores. - `LocalProviderId` retains the current `^[a-z][a-z0-9-]{0,62}$` grammar; dots are forbidden. Its maximum length is 63 ASCII bytes. @@ -1072,17 +1231,25 @@ local identifier: exactly one dot separator, and has a maximum serialized length of 127 ASCII bytes. -`QualifiedProviderId` is the type used by bidder routes, provider plans, -backend discriminators, auction responses, diagnostics, and telemetry. Its -canonical `Display` and serde representation is `.`. No -consumer reconstructs it with string concatenation, truncates it, or treats a -local provider ID as globally unique. Adapter target validation continues to -predict and reject backend-name collisions using the complete qualified -identity. A platform backend name is not itself the provider identity. Where an -adapter's normalization is lossy, as with Axum mapping dots, hyphens, and -underscores to the same character, its correlation name includes a stable -digest of the full qualified ID. Target validation rejects any remaining final -name collision. Tests cover aliases such as `a_b.c` and `a.b-c`. +`QualifiedProviderId` is the internal identity used by bidder routes, provider +plans, and lookup indexes. Its canonical `Display` and serde representation is +`.`. `ExternalProviderLabel` is the separate validated +string used on backend discriminators, auction responses, diagnostics, +telemetry, mediator correlation, and other externally keyed maps: it contains +the legacy local ID for schema 1 and the qualified ID for schema 2. Every such +map is built from an explicit `ExternalProviderLabel -> plan ordinal` index; +label collisions fail validation and no lookup uses `unwrap_or(usize::MAX)` or +map iteration as a fallback ordering rule. + +No consumer reconstructs either type with ad hoc string concatenation, +truncates it, or treats a local provider ID as globally unique. Adapter target +validation predicts and rejects backend-name collisions using the complete +qualified identity. A platform backend name is not itself the provider +identity. Where an adapter's normalization is lossy, as with Axum mapping dots, +hyphens, and underscores to the same character, its correlation name includes +a stable digest of the full qualified ID. Target validation rejects any +remaining final name collision. Tests cover aliases such as `a_b.c` and +`a.b-c`. Internal qualified identity and externally serialized identity cut over at different times. The schema-1 compatibility reader constructs a qualified @@ -1138,13 +1305,17 @@ The contract is: flattened plan. With the existing strict-greater-than price comparison, the first configured provider retains an equal-price tie during local winner selection. Dispatch checks the remaining shared deadline immediately before - each back-to-back launch, so every later launch observes a smaller logical - budget and may also cross an adapter timeout bucket; order is budget-visible - even when the deadline does not expire. -10. Shared browser dispatchers execute handlers by the owning integration's - configured ordinal and then definition-local registration order. Wall-clock - registration timing, numeric priority, and lexical handler ID are not - alternate ordering mechanisms. + each back-to-back launch. A later provider receives + `min(remaining_deadline, provider_timeout)`; elapsed time reduces the shared + remainder but may not change the logical budget when the provider's own + shorter timeout still caps it. Order remains launch-, response-, mediator-, + and tie-visible even when budget values happen to match. +10. Shared browser dispatchers execute immediate handlers by IIFE registration + sequence and then definition-local registration order. Because immediate + IIFEs are concatenated in configuration order and must register + synchronously during evaluation, this sequence is the configured order + without a second ordinal channel. Numeric priority, lexical handler ID, and + later wall-clock callbacks are not alternate ordering mechanisms. Ordered runtime introspection carries the configured ordinal with each integration and provider. Lookup indexes may use maps, but iterating a @@ -1163,30 +1334,33 @@ sourcepoint, osano, google_tag_manager, datadome, gpt, gpt_diagnostics Milestone-one normalization and schema-one decoding use that complete sequence; they do not move `js_asset_proxy` ahead of Prebid. Schema 2 instead chains attribute rewriters in operator-visible integration order: a replacement becomes -the next rewriter's input and a removal is terminal. Operators are told that an -earlier native rewrite can prevent a later exact-original-URL proxy rule from -matching, and that moving `js_asset_proxy` ahead of Prebid can prevent the -publisher-Prebid remover from recognizing a rewritten URL. Overlap tests cover -GPT, Google Tag Manager, DataDome, Sourcepoint, Permutive, Lockr, Testlight, and -Prebid in both legacy-compatible and deliberately reordered configurations. No -engine-only override is hidden from TOML. - -Script-text rewriters also receive the current text produced by the preceding -rewriter rather than having independent `lol_html` handlers repeatedly replace -the original chunk. DOM-insertion guards claim canonical parsed host/path -ownership, not substring coincidences in a query string. Built-in ownership -sets are disjoint and tested with exactly one claimant for every canonical URL; +the next rewriter's input and a removal is terminal. The pure claim validation +defined above prevents an earlier native owner from silently bypassing a later +`js_asset_proxy` asset policy. Moving `js_asset_proxy` before that owner is the +only supported explicit override; the TOML diff then shows the precedence +change. Overlap tests cover blocked and enabled assets against GPT, Google Tag +Manager, DataDome, Sourcepoint, Permutive, Lockr, Testlight, and Prebid in both +legacy-compatible and deliberately reordered configurations. No engine-only +override is hidden from TOML. + +Before either crate move, prerequisite #1208 makes script-text rewriters receive +the current text produced by the preceding rewriter rather than having +independent `lol_html` handlers repeatedly replace the original chunk. That +composition applies to schema-1 and schema-2 blobs; milestone two changes only +which integration order supplies the already-composed chain. Prerequisite #1199 +similarly makes DOM-insertion guards claim canonical parsed host/path ownership, +not substring coincidences in a query string. Built-in ownership sets are +disjoint and tested with exactly one claimant for every canonical URL; configured order applies to genuinely composable handlers, not to select the winner of an ownership bug. Browser load mode remains a lifecycle phase, not a second operator priority: immediate code necessarily evaluates before deferred code. Config order is -preserved within each phase and remains the stored ordinal used by any shared -dispatcher after a module registers. A hook that must arbitrate during initial -document mutation, including a DOM-insertion guard, is immediate-only; -composition rejects it on a deferred asset. Diagnostics display each module's -fixed load mode so this phase boundary is visible rather than inferred from -timing. +preserved within each phase. A browser capability that declares a DOM-insertion +handler has `requires_initial_dom_dispatch = true`; composition accepts that +capability only with `load_mode = immediate`, and the IIFE must register the +handler synchronously before returning. Diagnostics display each module's fixed +load mode so this phase boundary is visible rather than inferred from timing. All recovery paths obey the same provider ordinals. Existing stable plan-index sorts remain. The abandon/failure path that can emit providers from `HashMap` @@ -1197,13 +1371,13 @@ Inline-table and dotted-key shorthand may not define an integration parent or provider parent. Requiring ordinary table headers makes activation, ownership, and order visible in one form and lets the pre-pass produce targeted errors. -The integrations crate directly enables the `preserve_order` feature on the -workspace's single resolved `toml` package; it does not rely on an unrelated -dependency to activate that feature through Cargo unification. EdgeZero's -`toml::Value` then sees the same ordered map implementation. A test asserts that -typed integration/provider order equals the independent `toml_edit` pre-pass -order. The integration-owned source model uses ordered map types with explicit -iteration semantics, never `HashMap` or `BTreeMap`. +The workspace already receives `toml/preserve_order` transitively through +`handlebars`; this design does not pretend that feature is absent or depend on +Cargo feature toggling for correctness. A test asserts that typed +integration/provider order equals the independent `toml_edit` pre-pass order. +The integration-owned source model uses ordered map types with explicit +iteration semantics, never `HashMap` or `BTreeMap`, and stored runtime order +comes only from sidecars. Before typed deserialization, the host-only `source-config` module of `trusted-server-integrations` parses the source with `toml_edit`. The dependency @@ -1238,7 +1412,9 @@ construction, consent, and store writes after the pre-pass succeeds. The EdgeZero extension uses EdgeZero's existing path and target resolution: an explicit `--app-config` wins; otherwise the path is `/.toml`; diff/push provide their resolved adapter; and -validate provides the supported adapter set declared by the manifest. Parity +the Trusted Server validate hook receives only the repeatable `--target` +selections, possibly empty. EdgeZero separately retains the manifest-declared +target set for its existing validation and `--strict` checks. Parity tests cover exact-byte validation and serialization, command/target context, explicit and default paths, manifest paths with and without parent directories, concurrent app-config replacement, and `--no-env`. The environment overlay can @@ -1290,10 +1466,12 @@ three boundaries. This is an explicit asymmetric serde boundary. EdgeZero's typed CLI deserializes and validates `TrustedServerAppConfig`, then its manual `Serialize` implementation emits the schema-2 storage DTO. Runtime reads that DTO rather -than deserializing it back into the operator type. `integration_order` is always -present. `provider_order` is required whenever the corresponding `providers` -object is present, including when both are empty; both are absent when an -integration has no provider collection. +than deserializing it back into the operator type. The DTO stores normalized +typed values after defaults, not source-field presence; the retained +`toml_edit` view alone preserves comments and editing intent. `integration_order` +is always present. `provider_order` is required whenever the corresponding +`providers` object is present, including when both are empty; both are absent +when an integration has no provider collection. `trusted_server_schema`, `integration_order`, and `provider_order` are reserved storage fields. They are generated by serialization and rejected if supplied in @@ -1310,8 +1488,9 @@ Object-shaped storage keeps secret paths such as `integrations.datadome.server_side_key_secret_name` valid. Core and integration secret metadata traverse that resolution view before ordered settings are constructed. Integration-owned inactive-secret preprocessing also receives the -object-shaped integration entry by ID and writes any resolved value back to the -same entry; it never searches an `{ id, config }` sequence. End-to-end tests +object-shaped integration entry by ID and removes inactive references; the +shared resolver later writes active resolved values back to the same entry. +Neither searches an `{ id, config }` sequence. End-to-end tests prove both active DataDome secret fields are presence-checked, resolved, and removed when inactive. Before storage, EdgeZero's static secret metadata still validates every reference actually present in source, including a reference @@ -1390,14 +1569,22 @@ Validation fails for: - A referenced browser module absent from the generated browser catalog. - An unsupported capability combination. -A disabled integration may retain structurally valid provider configuration; -those providers are not added to the plan. A globally disabled auction may -likewise retain otherwise valid enabled integration and provider configuration -so operators can prepare configuration before enabling the auction. References -from bidder routing or mediator selection to an explicitly disabled integration -are retained but omitted from the compiled plan with one aggregated warning. -This makes `enabled = false` a one-line integration kill switch without making -typos or absent definitions valid. The global auction switch remains the +A disabled integration may retain provider configuration, but every retained +child ID and reference is resolved and structurally validated before pruning; +a typo or missing child fails even while disabled. Active-only semantic checks +remain skipped. A globally disabled auction may likewise retain otherwise +valid enabled integration and provider configuration so operators can prepare +configuration before enabling the auction. + +References to an explicitly disabled known integration are retained in a +validated suppression set and omitted from executable capabilities with one +aggregated warning. A pruned bidder route produces the distinct diagnostic +outcome `disabled_by_integration`, not `unroutable`, and its bidder stays +suppressed from Prebid client-side fallback. A selected disabled mediator +intentionally falls back to local ranking with a warning instead of the +baseline startup error; this is an explicit behavior change. This makes +`enabled = false` a one-line integration kill switch without making typos or +absent definitions valid. The global auction switch remains the cross-integration emergency stop. The design does not add a second generic capability-specific enablement system; server-only Prebid remains an enabled parent without browser-activating fields. @@ -1434,14 +1621,20 @@ comment-preserving `toml_edit` transformation, never a push. It: - maps every profile to its owning integration, flattens `profile_config`, drops the fixed protocol, qualifies cross-integration references, and creates each required parent before its descendants; -- emits retained/required integration parents in the deterministic migration - sequence `prebid, aps, js_asset_proxy, testlight, nextjs, permutive, lockr, -didomi, sourcepoint, osano, google_tag_manager, datadome, gpt, -gpt_diagnostics, openrtb, adserver_mock`, filtered to configured entries. The - first fourteen reproduce the complete baseline hook-registration sequence; - `openrtb` and the mediator have no competing page-hook position. This does not - preserve the previously cosmetic order of legacy `[integrations.*]` tables or - assume `js_asset_proxy` was globally first; +- first derives the complete legacy flat provider sequence in lexical local-ID + order. Provider-owning parents with active providers are promoted to a prefix + before `js_asset_proxy`, ordered by the first occurrence of one of their + providers in that sequence; each parent's provider tables retain their legacy + local order. If each owner's entries were contiguous, this exactly preserves + legacy priority. If owners were interleaved, grouping is unavoidable: the + report prints every old and new ordinal and every reordered pair, then asks + for confirmation before writing. Remaining parents follow the frozen legacy + hook sequence `prebid, aps, js_asset_proxy, testlight, nextjs, permutive, +lockr, didomi, sourcepoint, osano, google_tag_manager, datadome, gpt, +gpt_diagnostics, openrtb, adserver_mock`, excluding promoted entries and + filtering to configured entries. This does not preserve the previously + cosmetic order of legacy `[integrations.*]` tables or assume + `js_asset_proxy` was globally first; - writes explicit `enabled` using a frozen table of the baseline defaults rather than guessing one value for every integration; the baseline-true set is Prebid, GPT, Didomi, Lockr, and Permutive, and retained blocks for other @@ -1450,17 +1643,25 @@ gpt_diagnostics, openrtb, adserver_mock`, filtered to configured entries. The - handles server-only Prebid and implicit APS activation explicitly, and warns before enabling a retained disabled APS block whose `rendering_mode` would change renderer-route behavior; -- emits providers in the new integration/provider declaration order, reports - every legacy globally interleaved priority that schema 2 cannot retain, and - shows the resulting qualified sequence before writing; -- prints old-to-new environment-overlay variable paths and fails when an - app-prefixed environment variable targets a retired path; +- for a server-only Prebid migration, enables the parent, comments out rather + than activates inactive legacy browser-only non-default fields including + `debug`, `client_side_bidders`, and `timeout_ms`, and reports each edit. It + preserves inert `bundle.modules`, `external_bundle_sha256`, and + `external_bundle_sri` staging fields. The emitted candidate must pass the new + value-based activation validation; +- prints old-to-new environment-overlay variable paths. Migration warns and + maps stale names when possible; `--no-env` skips even the environment-name + scan and reports that it did so. Ordinary validate, diff, and push fail any + app-prefixed variable that still targets a retired path; - preserves file permissions and uses the existing permission-preserving atomic writer. Missing-`enabled` diagnostics name the exact old default even when operators -migrate by hand. The shipped example, `config init`, audit drafts, operator -guides, and environment-overlay examples contain a complete migrated shape. +migrate by hand. An exact golden test migrates the shipped example and proves +that its previously inactive Prebid browser-only values stay commented/inert, +its bundle staging metadata is preserved, and the result validates. The shipped +example, `config init`, audit drafts, operator guides, and environment-overlay +examples contain a complete migrated shape. Old `[auction.providers]`, `profile`, `profile_config`, and `protocol` fields fail with an actionable message naming the new integration-owned location or @@ -1494,13 +1695,16 @@ New CLI writes schema 2 only. Rollout is adapter-specific: -| Adapter | Forward cutover | Rollback isolation and evidence | -| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Fastly | Export and reconstruct the complete schema-one envelope, including every referenced chunk. Create a new physical Config Store, seed it with schema one, and link it as `trusted_server_config` only in the service version containing the dual reader. After settings-dependent probes pass at representative POPs, push schema 2 to that new store. | The prior service version remains linked to the untouched schema-one store, so reactivation cannot observe schema two. Config GC is forbidden for either rollback generation until the rollback window closes. Control-plane confirmation alone is insufficient because store visibility is eventual. | -| Cloudflare | Create a new Worker version whose `TRUSTED_SERVER_CONFIG` variable contains schema 2; `ts config push` to KV is not treated as a runtime cutover. | Worker rollback restores the previous code and schema-one binding together. Verify the bound version, startup schema/digest log, and a settings-dependent route. | -| Spin | Use a named versioned KV key/store only when the deployment platform can export, restore, and select it atomically with the component version. | The default remote Spin path is blocked from schema-2 rollout until a concrete control-plane export/restore drill exists; liveness alone is not evidence because the startup-error router stays healthy. | -| Axum | Use the migrated local file/environment as a development-only cutover. | Retain the archived schema-one file and restart the matching binary/config pair. | +| Adapter | Forward cutover | Rollback isolation and evidence | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Fastly | A native Fastly entry copy, not `ts` reserialization, reconstructs the complete source root and every referenced chunk. It writes identical chunk keys/values to a new physical Config Store first, reads and reconstructs them there, verifies the archived `BlobEnvelope.sha256`, and writes the pointer/root entry last. Production and staging stores/selectors are copied and verified separately under PR #1175/EdgeZero PR #381. The dual-reader service version alone links the new store; representative POPs must observe the expected setting and a settings-dependent route before schema 2 is pushed. Missing or not-yet-propagated chunks block cutover; milestone 1 explicitly maps this condition to the adapter's transient startup/503 path instead of today's configuration/500, and partial decode is forbidden. | Rollback is only reactivation of the prior service version, which remains linked to the untouched schema-one store. Never redeploy an old release or bind an old binary to the new store. Config GC is forbidden for either rollback generation until the rollback window closes; control-plane confirmation alone is insufficient because visibility is eventual. | +| Cloudflare | Create a new Worker version whose `TRUSTED_SERVER_CONFIG` variable contains schema 2; `ts config push` to KV is not treated as a runtime cutover. | Worker rollback restores the previous code and schema-one binding together. Verify the bound version, startup schema/digest log, and a settings-dependent route. | +| Spin | Use a named versioned KV key/store only when the deployment platform can export, restore, and select it atomically with the component version. | The default remote Spin path is blocked from schema-2 rollout until a concrete control-plane export/restore drill exists; liveness alone is not evidence because the startup-error router stays healthy. | +| Axum | Use the migrated local file/environment as a development-only cutover. | Retain the archived schema-one file and restart the matching binary/config pair. | +Before rollout, the candidate dual reader decodes an archived production +schema-one envelope and compares normalized settings, plan, integration and +provider order, and every `ExternalProviderLabel` with the old binary's output. The common release order is: archive and verify schema one; deploy the dual-reader code against schema one; verify a settings-dependent route and startup log reporting application schema and non-secret composition digest; @@ -1511,12 +1715,13 @@ authenticated diagnostics may expose the same evidence, but no new public status route is introduced. Store/version isolation makes downgrade structurally safe where the platform -supports it. The operational old-CLI fence remains defense in depth, not the -only protection: an old writer has no release credentials or destination -mapping for the new Fastly store/version or Cloudflare binding. Schema-one -reappearance after cutover is an explicit rollback event. The compatibility -reader is removed only in a later release after every supported deployment has -completed and drilled its rollback window. +supports it. The operational old-writer fence is defense in depth, not a +credential guarantee or the only protection. The dual reader still accepts a +schema-one write on the new generation but records one structured +schema-regression/rollback event. Schema-one reappearance after cutover is an +explicit rollback event. The compatibility reader is removed only in a later +release after every production adapter, including Spin, has completed and +drilled its rollback window; a blocked Spin cutover blocks reader cleanup. ## Required Neutral Lifecycle Boundaries @@ -1580,13 +1785,17 @@ for every compiled definition whether that integration is absent, disabled, or enabled. The GPT-diagnostics reserved-input normalizer preserves the baseline behavior -that is independent of activation: it captures then removes `ts_console`, -removes `__Host-ts-console`, merges retained Cookie fields, and drops Cookie -fields that cannot be represented as visible ASCII. It runs before EC setup, -generic cookie parsing/classification, template-key construction, and origin -forwarding at every existing adapter boundary, with the direct publisher call -remaining an idempotent safety net. This static normalization contract is not -an enabled runtime capability. Therefore an absent GPT-diagnostics block cannot +that is independent of activation: it captures console activation from the +`ts_console` query and `__Host-ts-console` cookie into neutral +`ReservedInputState`, then removes those reserved inputs, merges retained Cookie +fields, and drops Cookie fields that cannot be represented as visible ASCII. +The integrations composition boundary exposes one request wrapper that runs +this catalog normalization before EC setup, template-key construction, origin +forwarding, or generic classification. All four adapters and direct core test +entry points use that wrapper. Core's generic Cookie parser remains a lenient, +idempotent safety net, but cannot call the concrete catalog because dependency +direction forbids it. This static normalization contract is not an enabled +runtime capability. Therefore an absent GPT-diagnostics block cannot reintroduce HTTP 400 responses for non-ASCII Cookie fields or leak reserved inputs upstream. @@ -1594,10 +1803,10 @@ Preparation returns the opaque per-integration decision plus its declared `RequestProcessingRequirements`. The neutral requirements are available before the existing template-cache/private decision; under ESI, request-private opaque state is never copied into a shared template. Registry preparation remains at -both existing locations: adapter boundaries and the idempotent core publisher -safety-net boundary used by direct core callers. Request extensions make a -second preparation a no-op. Core invokes finalization on the existing response -path. GPT diagnostics may request the fixed synchronous post-unified asset phase +the integrations-owned wrapper; request extensions make a repeated neutral +preparation a no-op where an adapter boundary can be re-entered. Core invokes +finalization on the existing response path. GPT diagnostics may request the +fixed synchronous post-unified asset phase and all-response privacy; ordinary head injectors cannot approximate that position. The current auction-correlation decision is deployment-scoped, not request-activation-scoped, and becomes a neutral enabled-capability query so @@ -1627,7 +1836,11 @@ is classified rather than copied blindly: relative imports across the two crate source roots. Runtime calls use a stateless accessor mapped by the integration build to the -already-installed namespace. The integration build fails on any undeclared +already-installed namespace. Facade version 1 is exact, not a compatible range; +each install callback resolves `getRuntimeV1()` lazily when it runs instead of +capturing an eager imported object. A mismatch logs one diagnostic, skips that +integration, and leaves its installed/shim flag unset so a failed Prebid load +cannot look successful. The integration build fails on any undeclared cross-root value import. This covers current imports such as Permutive context registration, Prebid auction helpers, Testlight queue installation, GPT slot resolution, and the shared script/beacon guards; it is not limited to the APS @@ -1637,7 +1850,10 @@ The GPT bootstrap still runs before the unified bundle, creates or adopts `window.tsjs`, and may install the first-impression state and GPT lifecycle listeners. The core IIFE must adopt that exact object identity, preserve pre-bundle claims and listener markers, validate the runtime-facade version, and -initialize only missing registries. It never replaces bootstrap state. Immediate +initialize only missing registries. The legacy bootstrap state has no version, +so adoption shape-validates it. GPT may intentionally replace legacy +`adInit`/`scheduleInitialAdInit` functions as the facade is installed; object and +first-impression state identity are preserved, not function identity. Immediate GPT and deferred Prebid likewise adopt the facade and keep listener installation idempotent. Integration bundles consume it through an external runtime shim and type-only browser-core declarations; their bundler must not inline the stateful @@ -1646,6 +1862,24 @@ and deferred Prebid in production order and prove that first-impression claims, Permutive context, logging configuration, renderer registrations, and APS frame supersession use the same shared state. +Prerequisite #1196 removes or debug-gates Creative's unconditional log-level +bump before extraction. The default remains `warn`, and an explicit publisher +`warn` setting is never overwritten; a production-artifact test pins both. +Restoring shared context makes Permutive context visible to core collection, +which is an intentional bug fix with a consent caveat: server-side +`allowed_context_keys` remains only a key allowlist, not a browser consent gate. +This extraction adds no new browser consent check and does not redesign consent +policy. + +Browser core publishes the queue API without draining preloaded `requestAds()` +calls. Composition appends one neutral finalizer after every immediate +integration IIFE; only that finalizer drains the preloaded queue, after +Permutive and all other synchronous registrations are complete. A production +order artifact test queues a call before loading the bundle and proves its +context providers are present. For one deprecation window, the facade also +backs the compatibility alias `window.tsjs.apsPrebidRenderers` used by bundles +predating #967. + Every integration IIFE declares the facade version it requires. A mismatched deferred or stale asset fails closed for that integration with a diagnostic rather than creating a second registry or mutating an incompatible shape. This @@ -1680,20 +1914,24 @@ containing: hashes; - the fixed creative prelude; - a deterministic document fingerprint covering injected asset IDs, hashes, - order, attributes, immutable inline-asset bytes, and stable configuration - fingerprints for generated inline head output. + order, attributes, and the exact static or configuration-rendered inline + bytes. Trusted attributes from immediate assets are merged onto the unified script -tag. Duplicate names with different values fail composition; equal duplicates -collapse to one attribute. +tag through validated name/value constructors. Invalid names or values and +duplicate names with different values fail release-mode composition; equal +duplicates collapse to one attribute. None of these checks is only a debug +assertion. Static serving, cache-busting URLs, and immutable-cache validation consume `BrowserDocumentAssets`; core no longer performs a crate-global `all_module_ids()` lookup. Its document fingerprint includes only assets that can affect the composed document and includes GPT bootstrap bytes. Each head injector whose generated inline output varies with integration configuration -supplies the exact immutable bytes or a deterministic contribution for that -output. Request-dependent head variation is permitted only when its neutral +supplies the exact immutable bytes for that output. The verified EdgeZero +envelope hash already covers configuration, so injectors do not invent +additional configuration fingerprints. Request-dependent head variation is +permitted only when its neutral `RequestProcessingRequirements` bypass shared-template reuse; request data is never folded into a composition-wide fingerprint. @@ -1704,7 +1942,7 @@ The composition digest is: ```text SHA-256( UTF8("ts-composition-v1\0") || - HEX_DECODE_32(verified_envelope.data_sha256) || + HEX_DECODE_32(verified_envelope.sha256) || BrowserDocumentAssets.document_fingerprint_raw_32 || U32_BE(application_schema) || U32_BE(LEN_UTF8(build_id)) || UTF8(build_id) @@ -1718,7 +1956,7 @@ is UTF-8 with a four-byte big-endian byte-length prefix. These fixed widths and the one explicit length prefix make the input unambiguous; implementations do not hash the displayed formula, hexadecimal text, or delimiter-free strings. -`data_sha256` is EdgeZero's already-verified canonical hash of stored data +`BlobEnvelope.sha256` is EdgeZero's already-verified canonical hash of stored data before secret resolution. It covers schema 2 order sidecars, enabled and disabled source, neutral settings, overlay results, and secret references without exposing secret values. Raw resolved secrets never enter a template @@ -1726,16 +1964,48 @@ key. If a future resolved secret changes document bytes rather than only authorizing an upstream call, its owner must contribute a reviewed non-secret behavior fingerprint or make the output request-private; it may not hash the secret itself. Semantically set-valued fields serialize in deterministic order -before EdgeZero hashes them. +before EdgeZero hashes them. When `trusted_server_schema` is absent, the +application schema contribution is exactly `1`. + +`build_id` is a deterministic build-time digest: + +```text +SHA-256("ts-build-v1\0" || LENGTH_FRAMED_SORTED(PATH || RAW_BYTES)) +``` + +Inputs include core and integrations Rust sources plus both neutral +`trusted-server-js` and `trusted-server-integrations-js` Rust sources, build +scripts, and manifests; neutral and integration browser sources and their +configuration/build scripts; embedded assets and templates; root `Cargo.toml`, +`Cargo.lock`, and pinned toolchain/version files; the canonical `package.json` +and `package-lock.json`; and any target or feature value that can alter emitted +output. Every file uses a normalized workspace-relative UTF-8 path label; +non-file inputs use stable labels such as `target-triple` and +`cargo-features`. Labels and byte values each have an explicit big-endian +length, and records are sorted bytewise by label. One canonical build-ID +generator supplies the value embedded by every composition consumer; generated +output and other self-referential derived artifacts are excluded from its input +set. The build +scripts emit matching +`rerun-if-changed`/`rerun-if-env-changed` lines and generate the digest into the +artifact. Individual asset hashes are build-time inputs; the unified +concatenated hash is a composition-time output. Each deployed artifact forces a +cold template namespace once when this `build_id` changes. The digest is computed lazily and memoized on the immutable composition so non-document routes and ineligible template-cache requests pay no repeated full-config hash cost. Normal request execution does not force it merely for logging. During a rollout, each adapter's configuration-activation path forces -it once for the activated snapshot and emits one rate-limited schema/digest log; -subsequent template use reads the memoized value. An adapter that cannot retain -the composition still deduplicates this rollout log per schema/data hash and -build ID rather than logging per request. The composition digest is distinct +it once for the activated snapshot and emits one rate-limited schema/digest log. +A process/adapter `CompositionObservationCache`, keyed only by application +schema, envelope hash, and build ID, retains at most 32 entries with LRU +eviction and deduplicates observations without retaining configuration values; +adapters that cannot retain the composition use the same cache instead of +logging per request. The once-per-snapshot structured activation log includes +ordered active integration IDs and ordered browser asset IDs/hashes. Runtime +provider-order evidence comes from existing ordered `/auction` +`ext.orchestrator.provider_details`. Together these make rollout registry/order +verification executable without a new endpoint. The composition digest is distinct from the exact concatenated unified-bundle hash, whose existing byte-content semantics remain unchanged. @@ -1746,8 +2016,9 @@ already covers it. This replaces only the old complete-`Settings` plus global-bundle digest; it does not replace any other `TemplateCacheKey` dimension. Full URL, request host and scheme, origin identity, assembly mode, ordered `Vary` values, selected cookie values, and `TEMPLATE_SCHEMA_VERSION` -remain independent key inputs. Transform-shape changes still bump -`TEMPLATE_SCHEMA_VERSION`. Tests prove neutral settings, non-head integration +remain independent key inputs. `TEMPLATE_SCHEMA_VERSION` changes only when the +cache-entry serialization or interpretation format changes; transform/output +code changes are already covered by `build_id`. Tests prove neutral settings, non-head integration rewriter settings, external and inline assets, and cookie policy changes invalidate templates without exposing raw configuration. @@ -1779,28 +2050,40 @@ type-keyed handler registry and neutral calls into it: The Prebid `adId` path and server-bid path remain distinct neutral entry points so one cannot consume the other's authority accidentally. - The direct `/auction` renderer-bid path in browser core also dispatches - through this registry; it is not a third APS-aware implementation. Renderer - failure reasons cross the neutral boundary as stable reason codes for the GPT - bridge without adding an APS enum to core. + through this registry; it is not a third APS-aware implementation. Its + neutral handler receives the validated `impid` targeting/slot ID, width, + height, and creative ID. The parser preserves the exact current fallback + order: numeric `bid.w`/`bid.h`, then descriptor dimensions, then `300x250`; + string `bid.crid`, then descriptor creative ID, then `-`. A + malformed descriptor is treated as absent and an otherwise valid `adm` + follows the ordinary generic-creative path, matching the baseline. Only after + a descriptor is valid and renderer dispatch is selected does a missing or + rejecting handler suppress the bid without generic fallback. Provider + admission/drop classification remains unchanged and failures expose stable + neutral reason codes. + Renderer failure reasons cross the neutral boundary as stable reason codes + for the GPT bridge without adding an APS enum to core. An enabled APS integration whose provider can emit APS renderer descriptors includes its immediate APS browser module. The core IIFE and fixed creative prelude load first, so APS registration is complete before a bid can render. -APS renderer registration is an immediate-only capability; composition rejects +APS module activation is explicit in the activation matrix above. APS renderer +registration is an immediate-only capability; composition rejects a deferred APS renderer. Its current rendering mode continues to be read while the synchronous unified script tag is executing, and the APS-owned trusted attribute remains on that tag. A future switch to a standalone or deferred APS asset requires replacing `document.currentScript` configuration first. -GPT is also immediate. Its current bootstrap reads `document.currentScript` +GPT is also immediate. Its current GPT module reads `document.currentScript` during module evaluation, so `data-ts-gam-attribution` remains on the unified synchronous tag and an artifact-level test proves the value is available at evaluation time. Moving source ownership must not silently make GPT deferred or move that attribute to a later tag. -Renderer failure is scoped to the owning renderer-bearing bid or message. A -missing or rejecting renderer suppresses that bid with no generic-creative or -native-Prebid fallback; unrelated bids and the page continue. A duplicate type +Renderer failure is scoped to the owning valid renderer-bearing bid or message. +A missing or rejecting renderer suppresses that bid with no generic-creative +or native-Prebid fallback; malformed direct-auction descriptors retain the +ordinary `adm` behavior above, and unrelated bids and the page continue. A duplicate type fails Rust composition, while a defensive browser-side duplicate poisons that type rather than using last-registration-wins. No renderer route, DOM, message, response, or beacon side effect is newly created before the selected handler @@ -1834,16 +2117,24 @@ location used by both Rust and browser tests. The move includes a repository-wide inventory of production executable browser programs assembled by integration Rust, not only files that already end in `.js`. The current DataDome, Didomi, GPT, Prebid, and Sourcepoint config -initializers; Sourcepoint response patches and `_sp_` property trap; and -GPT-diagnostics activation/history bootstrap move byte-for-byte with their -owners. Core-owned `build_bids_script`, `build_seam_script`, and -`build_ad_slots_script` remain core-owned neutral document programs. +initializers, Sourcepoint `_sp_` property trap, and GPT-diagnostics +activation/history bootstrap move with their owners. Sourcepoint response +patches are Rust regex rewriting behavior rather than standalone browser +programs, so that patch logic stays with the Rust Sourcepoint integration while +only executable fragments/assets move. Core-owned `build_bids_script`, +`build_seam_script`, and `build_ad_slots_script` remain core-owned neutral +document programs. This split does not introduce a general template language. Integration-owned program bodies live in `trusted-server-integrations-js`; integration Rust may serialize safe data and fill a narrowly generated owner-specific renderer that -preserves the baseline escaping model and output bytes. Exact rendered inline -bytes and hashes participate in the document fingerprint. A source/artifact +preserves the baseline escaping model. This permits the existing narrowly +scoped safe substitutions but does not expose generic templating. Static and +configuration-rendered inline bytes and hashes participate in the document +fingerprint. Request-dependent GPT-diagnostics bytes bypass shared-template +reuse and do not enter the composition-wide document fingerprint. Canonical +formatting may change generated bytes during the move, so behavior and reviewed +goldens—not a blanket byte-for-byte promise—are the acceptance contract. A source/artifact guard fails when a new production executable integration algorithm is introduced directly in Rust without an explicitly reviewed data-only exception. @@ -1883,17 +2174,30 @@ inventories current vendor-bearing fixtures, registry constructors, and test call sites and checks that moved tests are still selected by a native or target-matched gate. +The integrations crate exposes test-only +`routes_with_source_config`/`routes_with_composition_and_services` helpers (or +an equivalent `TestCompositionFactory` under `test-utils`) for adapter route +tests and cross-adapter parity. Core unit tests use only neutral stub +registrations. The #1135 end-to-end case remains in +`trusted-server-integration-tests/tests/parity.rs`; only integration-owned unit +fixtures move outward. This keeps test construction on the production +composition path without creating a core-to-integrations dev-dependency cycle. + Some names remain in core because they are established wire/configuration contracts rather than implementation ownership. The reviewed retained-name allowlist includes `creative_opportunities.slot.providers.{aps,prebid}`, the -browser `tsjs.scheduleInitialAdInit` and GAM `hb_*` handshake, and the historical -`prebid_eids` neutral identity module. Their current serialized forms and CLI -editing behavior remain. Conversely, `TrustedServerError::Prebid`, concrete -Prebid request extensions, the JS-asset-proxy header constant, the -`"adserver_mock"` plan literal, DataDome-only staging input, vendor benches, and -the concrete core test fixture move outward or become a named neutral contract. -A mechanical guard rejects any additional concrete vendor token in core unless -it is added to this allowlist with a wire-compatibility reason. +browser `tsjs.scheduleInitialAdInit` and `trustedServer` namespace, the GAM +`hb_*` handshake and slot fields, `nextjs-static`, `TrustedServerError::Gam`, +`rsc_flight`, APS/Prebid slot parameter types, and the historical `prebid_eids` +neutral identity module. Their current serialized forms and CLI editing +behavior remain. Conversely, `TrustedServerError::Prebid`, concrete Prebid +request extensions, the JS-asset-proxy header constant, the `"adserver_mock"` +plan literal, DataDome-only staging input, vendor benches, and the concrete core +test fixture move outward or become a named neutral contract. A mechanical +guard scans production Rust/JS beneath core, excluding tests, fixtures, +generated output, and exact allowlisted paths/tokens. It rejects any additional +concrete vendor token unless added with a wire-compatibility reason; it is not a +raw substring ban across the repository. ## Compatibility Contract @@ -1911,12 +2215,10 @@ provider-ID order to qualified configuration order. It preserves: - OpenRTB request, response, routing, timeout, notification, and response admission behavior. - Auction price comparison, mediation protocol, renderer descriptors, and - telemetry schema. Provider response order and a locally selected equal-price + provider telemetry schema, apart from the explicit #1203 mediator-failure + fields below. Provider response order and a locally selected equal-price winner may change when configuration order differs from the old lexical order. -- Existing provider identity fields. Schema 1 retains local external values; - schema 2 migrates them from values such as `pbs-main` to qualified values such - as `prebid.pbs-main` at the documented cutover. - Managed Prebid User ID aliases, collision checks, consent gating, opaque LiveRamp envelopes, OpenRTB EID production, EC partner ingestion, and admin diagnostics. @@ -1926,13 +2228,21 @@ provider-ID order to qualified configuration order. It preserves: cookie-key and bypass policy, and every existing publisher template-key dimension. - Current CLI ad-template diagnostics, audit/generator recovery behavior, and - Prebid bundle mutation behavior. -- The route and behavioral parity of Fastly, Axum, Cloudflare, and Spin. + corrected prerequisite #1202 Prebid bundle mutation behavior. +- The route and behavioral parity of Fastly, Axum, Cloudflare, and Spin, except + for the explicitly listed Fastly missing-chunk error-class correction. The intentional compatibility breaks are: - Auction providers move from `[auction.providers]` beneath their owning integration and references become qualified. +- External provider labels change at the schema-2 cutover from local values + such as `pbs-main` to qualified values such as `prebid.pbs-main`; schema 1 + retains the legacy labels during the dual-reader window. +- The corrected #1203 mediator-failure telemetry adds `reason = "http_status"`, + numeric `http_status`, and `fallback_used = true` on the structured mediator + outcome/log surface for non-2xx fallback; provider outcome telemetry remains + unchanged. - The fixed legacy `protocol = "openrtb-2.6"` field is removed from operator source rather than copied into each integration-owned provider. - Every retained integration parent requires an explicit `enabled` value, @@ -1948,6 +2258,11 @@ The intentional compatibility breaks are: - A retained enabled Prebid parent with neither a provider nor browser-activating URL becomes a valid staged no-op; the activation matrix and migration tool make this explicit rather than inheriting an integration-specific default. +- References to a known disabled integration are pruned as a kill switch: + bidder routes report `disabled_by_integration` and stay suppressed from + client-side fallback, while a disabled selected mediator warns and uses local + ranking instead of the baseline startup error. Absent and unknown targets + still fail. - Attribute and script rewriters use schema-2 configuration order instead of the complete legacy sequence. The migration tool shows this order and its overlap diagnostics; it does not falsely describe `js_asset_proxy` as the @@ -1960,7 +2275,10 @@ The intentional compatibility breaks are: unified behind the runtime facade: Permutive context reaches core collection, integration logging follows `tsjs.setConfig`, and APS renderer/frame state is shared across core, GPT, and Prebid. These are explicit baseline bug fixes, - not invisible "pure moves." + not invisible "pure moves." Permutive's newly restored context can now be + emitted for allowlisted keys without a new browser consent gate; + `allowed_context_keys` is not represented as consent enforcement, and a + consent-policy redesign is explicitly out of scope. - Stored data gains the application schema and explicit order sidecars. The rollout decoder, not the steady-state schema, provides temporary old-blob compatibility. @@ -2018,13 +2336,20 @@ Milestone exit criteria are independent: - **Milestone 1 — crate boundary:** only the current operator and stored schema are accepted; fixed integration-builder order, lexical provider priority, - implicit APS activation, CLI behavior, cache policy, adapter behavior, and - Fastly settings-only failure paths remain unchanged. The browser-state + implicit APS activation, operator source semantics, cache policy, adapter behavior, and + Fastly settings-only failure paths remain unchanged except that a missing or + not-yet-propagated config chunk becomes the documented transient startup/503 + outcome instead of configuration/500. The EdgeZero typed hook, + `config validate --target`, and selected-target diff/push validation are the + explicit milestone-one CLI deltas. The browser-state corrections listed in the compatibility contract and one-time artifact/hash - invalidation are explicit milestone deltas. DOM-handler ordering and trusted - attribute conflict policy remain at baseline until milestone 2. Every live - consumer uses the new composition root, every concrete source has moved, all - transitional delegations are deleted, and the full repository gates pass. + invalidation are explicit milestone deltas. Corrected script chaining, + disjoint parsed DOM ownership, and release-mode trusted-attribute conflict + validation are prerequisites and apply to schema 1 as well as schema 2; only + their ordering source changes later. Every live consumer uses the new + composition root, every concrete source has moved, the public-API snapshot's + explicitly classified transitional-export set is empty, and the full + repository gates pass. - **Milestone 2 — ordered configuration cutover:** the new operator source is the only writable shape; the dual stored-schema reader is deployed; qualified identities and order sidecars are used end to end; every normal and recovery @@ -2050,11 +2375,18 @@ transitional catalog. 3. Create both new crates and update Cargo aliases, native/WASM checks, JS discovery/typecheck/lint/format commands, Dependabot inputs, and test package selection in the same change. Update compiled documentation snippets and - crate-path documentation required for that workspace state. Create the - explicit Rust catalog as the one active catalog, initially delegating through - a shrinking, reviewed set of transitional core exports. Create the one - browser manifest with an equally explicit shrinking map to legacy source - paths. No production code is copied into an unused duplicate tree. + crate-path documentation required for that workspace state. Move + `TrustedServerAppConfig`, integration validation/secret metadata, and the + composition facade to the integrations crate; rewire every adapter and the + CLI to that one composition root while retaining the legacy schema through + the shared converter. Create the explicit Rust catalog as the one active + catalog, initially delegating through a shrinking, reviewed class of + transitional core exports, and delete core's old `builders()` and concrete + `with_plan` composition paths. Create the one browser manifest with an + equally explicit shrinking map to legacy source paths. No production code is + copied into an unused duplicate tree. The EdgeZero extension and selected- + target push validation land here so config consumers cannot bypass the new + root. 4. Move ordinary Rust integrations one owner at a time, then DataDome and GPT diagnostics after their lifecycle seams, APS and Prebid after the profile seam, and `adserver_mock` after the mediator seam. Add `openrtb`. Each move is @@ -2066,27 +2398,27 @@ transitional catalog. programs, registry inputs, and shared fixtures. Each move removes one legacy manifest path and updates its JS gates immediately. Complete the cross-root import audit, remove browser-core APS imports, adopt the shared runtime - facade, and make composed assets authoritative. Keep baseline DOM priority - ordering until milestone 2. Cross-adapter system tests remain in + facade, and make composed assets authoritative. Under schema 1, immediate + concatenation reproduces the corrected frozen legacy order through the new + registration-sequence dispatcher; schema 2 later changes only the order + source. Cross-adapter system tests remain in `trusted-server-integration-tests` and gain bootstrap/core/immediate/deferred load-order coverage. -6. Move `TrustedServerAppConfig`, all integration-specific configuration, - validation, inactive-secret preprocessing, and secret metadata into the - integrations crate. Land the EdgeZero typed-config source/command extension, - repin all EdgeZero workspace dependencies together, rewire the CLI to the - source-view APIs, and rewire all adapters to the single runtime composition entry - point while retaining the existing operator schema through the shared legacy - converter. Remove all transitional catalog/source entries. +6. Remove the final Rust and browser transitional exports/path mappings, prove + the public-API snapshot's transitional class is empty, and run the complete + schema-one differential and repository gates. This is cleanup and + verification of the composition switch made in step 3, not a second switch. 7. Add the TOML source pre-pass, ordered in-memory maps, application schema 2, object-shaped storage with explicit order sidecars, strong qualified provider IDs, the shared schema-one reader, `ts config migrate`, and the integration-owned provider schema. Replace and remove only the legacy source entry point from ordinary config commands; the isolated migration parser may read legacy source only to emit a schema-2 candidate. -8. Activate configuration-order provider and hook semantics, chained script - rewriting, disjoint DOM claims, strict trusted-attribute conflicts, external - qualified IDs, and plan-order recovery only after focused baseline and - failure-path tests pass. +8. Switch the already-composed hook and provider pipelines from frozen legacy + order to schema-2 configuration order, and activate schema-2 external + qualified labels and plan-order recovery only after focused baseline and + failure-path tests pass. Script chaining, disjoint DOM ownership, and trusted + attribute validation do not wait for this step. 9. Update operator examples, fixtures, migration diagnostics, guides, and the adapter-specific rollout runbook. Repository/CI path updates associated with moved code are already complete from steps 3 through 6. Deploy the dual @@ -2133,11 +2465,11 @@ to preserve `cfg(test)` imports. - TOML parent-table order becomes the integration-owned source-model order. - The `toml_edit` pre-pass and typed `toml`/EdgeZero parser use the same TOML language generation and agree on parity fixtures outside `[integrations]`. -- A dependency-graph gate proves the host CLI graph enables - `toml/preserve_order` through the integrations crate's direct feature request, - while adapter graphs consume only explicit stored sidecars and never depend - on TOML or JSON object iteration order. Running the storage round trip with - and without `serde_json/preserve_order` yields identical runtime order. +- A dependency-graph gate records the existing unified `toml/preserve_order` + feature while adapter graphs consume only explicit stored sidecars and never + depend on TOML or JSON object iteration order. Storage tests permute JSON + object members and prove identical runtime order; they do not claim to toggle + a unified Cargo feature within one graph. - Nested provider declaration order is retained. - A parent integration or provider table declared after one of its descendants fails before typed deserialization. @@ -2160,10 +2492,12 @@ to preserve `cfg(test)` imports. - CLI validation compiles and discards the complete secret-independent plan, including routes, bidder ownership, mediator capability, and duplicate claims, without constructing runtime capabilities from unresolved secret key - names. Diff/push fail selected-target validation. Validate reports a result - for every supported target declared in the manifest, and `--strict` fails on - any target rejection. Every secret-independent runtime rejection has a - matching target-neutral error or named-target CLI fixture. + names. Diff/push fail selected-target validation. Validate with no `--target` + runs target-neutral Trusted Server checks plus unchanged EdgeZero validation; + each explicit repeatable `--target` is reported and any selected-target + rejection fails. EdgeZero `--strict` retains its existing manifest meaning. + Every secret-independent runtime rejection has a matching target-neutral + error or named-target CLI fixture. - Read-only CLI consumers see the effective overlay through `SourceConfigView` without deploy validation; deploy commands use `ValidatedSourceConfig` after the overlay. Mutators and generators operate on @@ -2182,7 +2516,8 @@ to preserve `cfg(test)` imports. patched atomically over an otherwise-invalid unrelated baseline. The command fails with a placement example when the explicit `[integrations.prebid]` parent is absent and never appends a parent that silently chooses runtime - order. + order. Tests preserve the original file mode and prove parse diagnostics + contain path and line/column but no source content. - Disabled integrations may retain structurally valid provider settings, contribute no providers or capabilities, and do not reorder enabled neighbors. - Disabled placeholder values that current examples rely on, including the @@ -2194,7 +2529,9 @@ to preserve `cfg(test)` imports. runtime. - Bidder and mediator references to explicitly disabled known integrations are omitted from the compiled plan with one warning, including while the global - auction is disabled; absent and unknown targets still fail. + auction is disabled; absent and unknown targets still fail. Pruned routes + report `disabled_by_integration` and remain suppressed from Prebid client-side + fallback, while a disabled selected mediator warns and uses local ranking. - Nested-only, unknown, missing-enabled, descendant-before-parent, and mixed old/new configurations fail with actionable messages. - Qualified provider references resolve correctly and reject missing or @@ -2217,21 +2554,24 @@ to preserve `cfg(test)` imports. correlation names or fail target validation before deployment. - `ts config migrate --dry-run` preserves comments and permissions, emits frozen explicit defaults and environment-variable path mappings, handles - server-only Prebid and implicit APS, reports priority changes, rejects mixed - input, and never performs a remote write. + server-only Prebid and implicit APS, preserves exact legacy priority when + provider owners are contiguous, reports every old/new ordinal and reordered + pair when grouping cannot preserve an interleaving, rejects mixed input, and + never performs a remote write. The shipped-example golden validates. - Ordering diagnostics cover provider details, ts-debug, telemetry `is_win`, mediator `ext.bidder_responses`, backend names, and CLI provider listings in addition to launch and response order. ### Capability and behavior parity tests -- A focused differential harness captures the `a4e01eb55` baseline for a matrix - covering all integrations, APS/Prebid/standard providers including globally - interleaved IDs, deferred Prebid, GPT diagnostics, and the #1135 Next.js - streaming path. It compares hook order, module ID lists, head inserts, trusted - tag attributes, route tables, plan/provider order, mediator input order, and - relevant CLI views. It compares semantic IDs/order rather than rebuilt bundle - bytes and records the explicit milestone deltas separately. +- After every prerequisite lands, a focused differential harness records the + exact baseline commit and captures actual output goldens for a matrix covering + all integrations, APS/Prebid/standard providers including globally interleaved + IDs, deferred Prebid, GPT diagnostics, Next.js/GTM script rewriting, RSC + streaming, and the #1135 path. It compares hook order, rendered output, + module ID lists, head inserts, trusted tag attributes, route tables, + plan/provider order, mediator input order, and relevant CLI views. Rebuilt + artifact hashes are recorded separately from behavior deltas. - The static catalog contains all current integration IDs plus `openrtb`. - APS and Prebid register page/browser and auction capabilities without core importing their types. @@ -2277,11 +2617,14 @@ to preserve `cfg(test)` imports. - The dedicated mediator capability preserves request construction, ordered response input, bounded transport, parsing, and launch/parse-error fallback without exposing the legacy `AuctionProvider` trait. A parsed non-2xx error - response retains its current zero-winner/no-fallback behavior. + response records `http_status` and falls back to local ranking without a + fabricated mediator winner, as fixed by prerequisite #1203. - Fastly's JA4 gate and failed-startup finalization still receive the settings-only view when full composition fails; reusable capability objects are immutable and request-stateless, with document buffers created per HTML - processor. + processor. A Fastly adapter test proves a missing referenced config chunk + takes the milestone-one transient startup/503 path and never partially + decodes the envelope. ### Browser tests @@ -2292,10 +2635,12 @@ to preserve `cfg(test)` imports. visible to core context collection; Prebid auction helpers and Testlight queue behavior still use the single installed runtime. - Built-in DOM-insertion guards claim disjoint parsed host/path sets with - exactly one claimant per canonical URL. Composable handlers run in immediate - IIFE registration order; numeric priority and lexical handler ID cannot affect - the result, query-string substrings cannot steal ownership, and a deferred DOM - handler fails composition. + exactly one claimant per canonical URL. In milestone 1, composable handlers + run in immediate IIFE registration order sourced from the corrected frozen + legacy sequence; in milestone 2, the same dispatcher receives schema-2 + configuration order. Numeric priority and lexical handler ID cannot affect + either result, query-string substrings cannot steal ownership, and a deferred + DOM handler fails composition. - APS is absent from browser core and registers its renderer from its own IIFE. - Core, GPT, and Prebid artifacts contain no private copy of APS renderer state. - Existing APS validation, sandbox, messaging, timeout, and rendering tests pass @@ -2307,12 +2652,16 @@ to preserve `cfg(test)` imports. unified tag; a deferred APS renderer is rejected during composition. - GPT remains immediate and reads `data-ts-gam-attribution` from the unified tag through `document.currentScript` at evaluation time. -- Equal duplicate trusted script attributes collapse, while conflicting values - fail composition before HTML is served. +- Equal duplicate trusted script attributes collapse, while invalid names, + invalid values, and conflicting duplicates fail release-mode composition + before HTML is served. - Missing or rejecting renderers drop only the renderer-bearing bid with no fallback or newly created route/DOM/message/response/beacon side effect; baseline supersession and fail-closed event handling remain, and duplicate registration poisons the type or fails composition. +- Direct-auction tests pin `w`/`h`/descriptor/default dimension precedence, + `crid`/descriptor/seat-plus-impid creative-ID precedence, malformed-descriptor + generic-`adm` fallback, and valid-descriptor fail-closed dispatch. - Both server-bid and Prebid-`adId` renderer paths cover carrier scrubbing, failed admission/registration, bounded TTL and capacity, authenticated source and slot binding, atomic consume, replay rejection, renderer-owned Universal @@ -2326,22 +2675,28 @@ to preserve `cfg(test)` imports. resolved secret values; and are lazily memoized. Rollout activation forces and logs the digest once per snapshot, while ordinary non-document requests do not. Set-valued source fields serialize deterministically. +- Build-ID generator vectors are deterministic across repeated builds and + independently perturb core Rust, integration Rust, neutral browser source, + integration browser source, `Cargo.lock`, target triple, and feature inputs; + each relevant change alters the ID while file ordering does not. - Changing any integration setting that affects generated head output changes the document fingerprint; request-dependent head variation bypasses shared template reuse through processing requirements. - GPT bootstrap, APS renderer document, Sourcepoint trap, GPT-diagnostics - bootstrap, Sourcepoint response patches, and - DataDome/Didomi/GPT/Prebid/Sourcepoint inline programs resolve byte-for-byte - from their integration-owned package locations. A guard rejects handwritten - production integration browser algorithms in Rust string literals without - introducing a general template framework. + bootstrap, and DataDome/Didomi/GPT/Prebid/Sourcepoint executable inline + programs resolve from their integration-owned package locations and pass + reviewed behavior/output goldens. Sourcepoint's Rust regex response patches + stay with its Rust owner. A guard rejects handwritten production integration + browser algorithms in Rust string literals without introducing a general + template framework. - External Prebid artifacts preserve bidder, User ID, and analytics category selection, manifest/hash/SRI generation, managed-name alias and collision checks, `identityLinkIdSystem` requirements, consent behavior, and runtime codes after registry and shim paths move. - Owner-specific private `$OUT_DIR` directories and manifests reject stale or - partial output; concurrent neutral/integration Cargo builds share only a lock - around dependency mutation and cannot discover one another's output. + partial output; installation holds the exclusive dependency lock while every + Vite, Vitest, TypeScript, ESLint, Prettier, Cargo browser build, and CLI + Prebid reader holds the shared lock for its entire process. - Clean and incremental Cargo builds prove that changing a sibling integration source reruns the integration embed build and changes its manifest/hash while leaving an unrelated neutral artifact unchanged. @@ -2350,7 +2705,13 @@ to preserve `cfg(test)` imports. Prebid load order using the moved assets. - An artifact test loads GPT bootstrap, core, immediate GPT, and deferred Prebid in production order, proves object identity/listener idempotence and - first-impression preservation, and rejects an incompatible facade version. + first-impression preservation, rejects an incompatible facade version without + setting the Prebid shim flag, and verifies the temporary + `apsPrebidRenderers` compatibility alias. A queued pre-load `requestAds()` is + drained only by the post-immediate finalizer and observes Permutive context. +- Production logging artifacts prove the default remains `warn`, an explicit + publisher `warn` is not overwritten, and Creative does not raise the shared + level. - Sibling-root gates resolve `vitest`, `prebid.js`, and an exported Prebid module through the canonical project; a failing typecheck canary proves moved type tests are actually selected, and Prettier/ESLint use explicit canonical roots. @@ -2385,8 +2746,8 @@ A repository path guard rejects active code or tooling that still points to `trusted-server-js/lib/src/integrations` or concrete `trusted-server-core/src/integrations/` paths, except for an explicitly allowlisted transitional or historical reference. It checks string literals and -known path joins, not only exact static paths. Dependabot keeps one entry for -the browser lockfile; unrelated Node projects retain their own entries. The +known path joins, not only exact static paths. Dependabot's existing browser and +docs entries remain; this split adds no second browser entry. The Cargo graph also records the direct `trusted-server-openrtb` and integration-test dependencies explicitly and removes unused direct browser crate dependencies from adapters. @@ -2490,8 +2851,9 @@ than reconstructing them. Splitting Rust ownership while sharing one Node workspace can embed previous output, race build scripts, or load APS too late. -Mitigation: write each build directly into a private owner `$OUT_DIR`, lock only -dependency mutation, configure every sibling-root resolver/tool explicitly, +Mitigation: write each build directly into a private owner `$OUT_DIR`, use an +exclusive dependency-mutation lock and full-process shared locks for every +browser/tooling reader, configure every sibling-root resolver/tool explicitly, retain stale-output refusal, hash built bytes, load core and the creative prelude first, reject deferred APS composition, and run artifact-level renderer, fingerprint, version-skew, and ordering tests. @@ -2529,8 +2891,9 @@ The change is complete when: 12. Browser core imports no concrete integration, and APS rendering works through one immediate registration without private copies in core, GPT, or Prebid bundles; the shared-state facade adopts GPT bootstrap state, DOM - ownership is disjoint, composable handler registration obeys configuration - order, and GPT retains its synchronous-tag bootstrap contract. + ownership is disjoint, composable handler registration obeys frozen legacy + order for schema 1 and configuration order for schema 2, and GPT retains its + synchronous-tag bootstrap contract. 13. `TrustedServerAppConfig`, `SourceConfigView`, `PartialSourceConfigView`, `ValidatedSourceConfig`, integration secret handling, and final runtime composition are owned by @@ -2582,7 +2945,7 @@ The change is complete when: 24. Reserved GPT-diagnostics inputs and malformed Cookie fields are normalized on baseline routes even when diagnostics is absent or disabled. 25. The adapter-specific rollout proves schema/binding state through logs and a - settings-dependent probe without adding a public status endpoint. + settings-dependent probe without adding any new status endpoint. ## Deferred Work From 19baffa08ee164c179893aea6c9c9a234d3b7edb Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 24 Sep 2026 23:50:34 -0700 Subject: [PATCH 12/13] Address third review of integrations split design --- ...-09-17-split-integrations-crates-design.md | 842 +++++++++++------- 1 file changed, 537 insertions(+), 305 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md index 4867d51a2..7d3756eb4 100644 --- a/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md +++ b/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md @@ -182,9 +182,12 @@ inventory, discriminator, provider-naming, and crate-layout decisions are superseded for in-tree integrations by this specification. That disposition is coordination, not evidence for the architecture chosen here. -PR #1135 is part of the current baseline. Its parser-aware streaming Next.js -processor, test support, and cross-adapter parity case move with the integration; -the removed HTML post-processor is not recreated by this work. +PR #1135 is part of the current baseline. The parser-aware streaming Next.js +implementation and integration-owned fixtures move with their owner. Its +cross-adapter end-to-end case remains in +`trusted-server-integration-tests/tests/parity.rs` and consumes +integration-owned test support; the removed HTML post-processor is not +recreated by this work. ### Open Defect and In-Flight Work Disposition @@ -193,26 +196,45 @@ not design authorities. “Prerequisite” means the focused fix lands on `main` and this specification records a new baseline before extraction begins; the crate split does not absorb that bug fix into a move commit. -| Item | Disposition | -| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| #1196 | Prerequisite. Restore cross-IIFE context/log state on the current layout, remove or debug-gate Creative's unconditional log-level bump, and add a production-artifact test. The milestone-one typed facade supersedes any transitional `Symbol.for` storage without reverting behavior. | -| #1197 | Prerequisite. Make set-valued configuration serialization deterministic before EdgeZero or template hashes depend on it. | -| #1198 | Prerequisite. Add the deterministic build digest defined below to the current template key before extraction; the split reuses the same contract. | -| #1199 | Prerequisite. Replace substring DOM ownership with parsed host/path ownership on the current layout. | -| #1200 | Prerequisite. Remove the shared-`dist` partial-build race before two Rust crates consume browser outputs; milestone one then adopts owner-private outputs and the lock contract below. | -| #1201 | Milestone two. New source and schema 2 reject unknown catalog IDs and fields. The temporary schema-1 reader deliberately retains baseline acceptance with a warning and is not “fixed” retroactively. | -| #1202 | Prerequisite. Make `ts prebid bundle` use the permission-preserving atomic writer and non-disclosing parse errors; the moved command retains that corrected behavior. | -| #1203 | Prerequisite and adopted decision. A parsed non-2xx mediator response is a mediation failure and falls back to local ranking as specified below. | -| #1204 | Prerequisite. Every Rust CI job that can trigger a browser build installs the pinned Node toolchain and dependencies; obsolete direct Rust browser dependencies are removed. | -| #1205 | Proposed remedy superseded. This specification keeps operator order and uses the pure script-source claim validation below instead of an unconditional hidden first phase. The issue must be revised to that contract or closed. | -| #1206, #1207 | Prerequisites resolved by #1208. Their output regressions are not accepted as differential-harness goldens. | -| #1208 | Prerequisite. Compose script-text rewriters on the current layout and add Next.js/GTM output goldens before the baseline is captured. | +| Item | Disposition | +| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| #1196 | Prerequisite. Restore cross-IIFE context/log state on the current layout, remove or debug-gate Creative's unconditional log-level bump, and add a production-artifact test. The milestone-one typed facade supersedes any transitional `Symbol.for` storage without reverting behavior. | +| #1197 | Prerequisite. Make set-valued configuration serialization deterministic before EdgeZero or template hashes depend on it. | +| #1198 | Prerequisite. Add the deterministic build digest defined below to the current template key before extraction, with the canonical generator owned by core build support; the split reuses the same generator for component digests. | +| #1199 | Prerequisite. Replace substring browser guards and Rust/HTML script-source matchers, including Permutive, DataDome, Lockr, and Testlight, with canonical parsed URL ownership on the current layout. Declaration and execution use one positive/negative corpus. | +| #1200 | Prerequisite. Remove the shared-`dist` partial-build race before two Rust crates consume browser outputs; milestone one then adopts owner-private outputs and the lock contract below. | +| #1201 | Milestone two. New source and schema 2 reject unknown catalog IDs and fields. The temporary schema-1 reader deliberately retains baseline acceptance with a warning and is not “fixed” retroactively. | +| #1202 | Prerequisite. Make `ts prebid bundle` use the permission-preserving atomic writer and non-disclosing parse errors; the moved command retains that corrected behavior. | +| #1203 | Prerequisite and adopted decision. A parsed non-2xx mediator response is a mediation failure and falls back to local ranking as specified below. | +| #1204 | Prerequisite. Every Rust CI job that can trigger a browser build installs the pinned Node toolchain and dependencies; obsolete direct Rust browser dependencies are removed. | +| #1205 | Proposed remedy superseded. This specification keeps operator order and uses the pure script-source claim validation below instead of an unconditional hidden first phase. The issue must be revised to that contract or closed. | +| #1206, #1207 | Prerequisites resolved by #1208. Their output regressions are not accepted as differential-harness goldens. | +| #1208 | Prerequisite. Compose script-text rewriters on the current layout and add Next.js/GTM output goldens before the baseline is captured. | +| #1098 | Prerequisite. Make neutral Cookie parsing lenient per header field and pair, preserving valid pairs when another pair is malformed and recording only counts/reasons. The catalog normalizer then removes and merges only its reserved names. This chosen remedy is broader and more explicit than the issue text. | +| #791 | Covered by extraction step 3. The Fastly-SDK guard follows the complete moved source set and dependency graph; this design does not add a second migration guard. | + +Other open work is coordinated without making it architectural authority or an +automatic prerequisite: + +| Item | Coordination contract | +| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| #1169 | Coordinate cache-policy, key-schema, and fixture changes. Preserve its separate origin-shareability decision if it merges before the baseline freeze. | +| #1179 | Reconcile per-document buffer work with #1208 and select one reviewed EdgeZero pin. Reuse remains optional; composition works per request. | +| #1193 | Rebase caching onto the immutable composition and its fingerprint; do not retain a parallel composition or fingerprint path. | +| #1191 | Preserve whichever claim/lifecycle contract lands before the baseline freeze; move its bootstrap/runtime harness and shared-state shape together. | +| #1159 | Preserve stored-request intent, filtered-impression admission, and `Skip` metadata through the prepared-exchange seam, including its server-before-JavaScript rollout requirement. | +| #1185 | External Prebid fork-source selection remains separate. Same-checkout enforcement here covers Trusted Server-owned package and input paths; future vendor-source selection requires an explicit extension. | +| #1002 | Its alternate release catalog, bootstrap transport, and hard cutover do not enter this design. Independently verified renderer tests may be reused; the release lines must otherwise be reconciled explicitly. | +| #1121, #1154 | Preserve accepted diagnostic wire behavior while moving GPT presentation and state outward; core retains only neutral diagnostic facts. | +| #1188 | Independent neutral EC/EID correctness stays in core. Preserve body ingestion and consent checks if it merges; it creates no integration definition. | +| #1107 | Separate proposed product feature, deferred. Extraction implies no trace endpoint or general tracing framework. | PR #1052 may contribute neutral renderer diagnostics, but APS-named browser core members do not enter core; they move with APS or become generic renderer -reason fields. PR #1175 and EdgeZero PR #381 are rollout prerequisites: the -typed-config extension and Fastly store procedure build on their reviewed -environment-selector line or an equivalent merged successor, never on v0.0.8. +reason fields. PR #1175 and EdgeZero PR #381 are milestone-one step-3 +prerequisites: the typed-config extension and Fastly store procedure build on +their reviewed environment-selector line or an equivalent merged successor, +never on v0.0.8. This specification partially supersedes the following earlier design documents only where they conflict with the new ownership, configuration, or fingerprint @@ -308,10 +330,11 @@ implementation cannot quietly reintroduce the rejected mechanism. behavior, because Prebid precedes `js_asset_proxy` today. **Decision:** legacy compatibility paths reproduce the complete baseline sequence. Schema 2 exposes chained replacement and terminal removal in declaration order. Pure - per-definition script-source claims make an earlier conflicting owner a - validation error, so the operator must place `js_asset_proxy` first for a - deliberate per-URL override instead of receiving a silent order-dependent - bypass. + per-definition script-source transitions validate the complete final outcome + for each operator asset. No position, including placing `js_asset_proxy` + first, is a privileged override: an operator may reorder integrations, but + validate/diff/push accept the result only when the entire chain finishes in + the declared asset policy. - **Treating PR #1016, PR #1084, or earlier review statements as design authority.** Their code and tests can reveal compatibility constraints, and conflicting in-flight work needs an explicit disposition. **Objection:** @@ -635,11 +658,11 @@ push have one selected adapter and fail when its target validation fails. The existing EdgeZero `config validate --strict` meaning is unchanged: it enforces the manifest-completeness and handler-path checks already owned by EdgeZero. Trusted Server target-plan validation is exposed separately as a repeatable -`ts config validate --target ` option. With no `--target`, the command -runs target-neutral Trusted Server checks plus existing EdgeZero validation; -with one or more targets, every selected target is checked and any target -failure makes the command fail. Auction-testing documentation and smoke -commands use this same target spelling. A fixture rejected by runtime +`ts config validate --adapter ` option. With no `--adapter`, the +command runs target-neutral Trusted Server checks plus existing EdgeZero +validation; with one or more adapters, every selected target is checked and any +target failure makes the command fail. Auction-testing documentation and smoke +commands use this same adapter spelling. A fixture rejected by runtime composition for a secret-independent reason must produce the same target-neutral error or the same named target result in the CLI. @@ -666,10 +689,14 @@ per-document factories and live in request/processor state, never in a reused registry object. Composition work is O(configuration); exact asset hashes are build-time inputs and template identity is lazy/memoized as described below. -Composition also exposes a narrow settings-only result before capability -construction. Fastly retains its current JA4 gate and failed-startup -finalization paths through that view; a registry or orchestrator failure must -not erase settings those degraded paths already use. +Composition also exposes a narrow settings-only result after the loader checks +that already precede today's `Settings` result: resolved-value validation, +auction-plan compilation, plan-dependent enabled-integration validation, and EC +partner validation. It is captured before target validation, mediator lookup, +route insertion, and executable capability construction. Fastly retains its +current JA4 gate and failed-startup finalization paths through that validated +view; a failure in those later stages must not erase settings those degraded +paths already use. Core retains neutral config-store access, Fastly chunk reconstruction, blob envelope verification, preprocessing for inactive neutral/global secret @@ -690,10 +717,12 @@ config-store bytes → core-owned neutral/global inactive-secret preprocessing → catalog-owned integration inactive-secret preprocessing → aggregated core + integration secret resolution - → resolved settings-only neutral view → catalog-aware resolved-value validation → core AuctionPlan compilation + → catalog-aware plan-dependent and core EC-partner validation + → validated settings-only neutral view → target validation + → mediator resolution → plan-dependent capability construction → core IntegrationRegistry and orchestrator construction → browser asset composition and document fingerprint @@ -709,20 +738,26 @@ depends on `edgezero-cli`. The current locked EdgeZero revision does not expose enough context for this contract. Building on EdgeZero PR #381 and Trusted Server PR #1175, EdgeZero -therefore gains one narrow typed-config extension with two default no-op -stages. Target resolution occurs before either hook. One exact source read is -shared by the source pre-pass, environment overlay, secret checks, typed -callback, diff, and serialization; a callback cannot reopen the path. Hook -context contains the command kind, the one selected target for diff/push or -the explicitly requested target set for validate, app environment-variable +therefore gains narrow `run_*_typed_with_hooks` entry points with source and +command hook objects supplied by the downstream caller; their defaults are +no-op. Trusted Server owns a wrapper argument type that flattens EdgeZero's +non-exhaustive `ConfigValidateArgs` and adds repeatable `--adapter`; it does not +add fields to EdgeZero's public argument type. Each command resolves its +adapter set before either hook: zero or more for validate and the existing one +for diff/push. One exact source read supplies both the raw/source-aware parse +and the typed overlay view, secret checks, command callback, diff, envelope, and +serialization; neither EdgeZero nor a callback may reopen the path. Hook context +contains the command kind, resolved adapter set, app environment-variable prefix, `--no-env`, EdgeZero's existing `strict` flag, raw path and bytes, and a borrow of the overlay-applied typed value. EdgeZero invokes command validation and serializes that same value. Validate, diff, and push therefore cannot validate one app-config read or typed value and serialize another. -All workspace `edgezero-*` dependencies are then repinned together from v0.0.8 -to one immutable, reviewed tag or commit containing this extension. The -extension does not alter manifest parsing, target selection, +All workspace `edgezero-*` dependencies are then repinned together to one +immutable, reviewed successor tag or commit containing both the environment +selector work and this extension. The current baseline happens to use v0.0.8; +the design does not assume that version is the successor's immediate parent. +The extension does not alter manifest parsing, target selection, environment-overlay mechanics, logging, storage, or adapter behavior. It replaces the filesystem snapshot wrapper entirely: config commands do not write temporary operator or manifest copies, require a writable checkout, rewrite @@ -819,12 +854,17 @@ shared configuration does more than include that sibling source root: exports-aware resolver. Runtime state is externalized behind the facade; the build does not depend on bundler deduplication of stateful entry points. - TypeScript includes both source roots and supplies exact package paths where - Node's ancestor walk cannot reach the canonical `node_modules`. + Node's ancestor walk cannot reach the canonical `node_modules`. Milestone 1 + first adds ambient `?inline`/`?raw` module declarations, a production + two-root `tsconfig` with `vite/client`, `DOM.Iterable`, and the current target, + and fixes the existing duplicate `w`/`h` production errors. It then enables a + no-emit typecheck of production sources; legacy Vitest typing is not hidden + behind that production gate. - Vitest names the sibling test directory and includes its runtime tests. A - separate canary command intentionally typechecks a fixture with one expected - error and asserts that exact diagnostic, so an empty glob or disabled source - checking cannot pass silently. The canonical package provides a real - `npm run typecheck` script for both roots. + separate isolated canary command intentionally typechecks a fixture with one + expected error and asserts that exact diagnostic, so an empty glob or disabled + source checking cannot pass silently. The canonical package provides a real + `npm run typecheck` script for both production roots. - ESLint is invoked from their common ancestor with explicit source globs; it does not require moving `package.json`. Prettier always receives the canonical `--config` path for sibling files. @@ -839,14 +879,17 @@ shared configuration does more than include that sibling source root: Separate build targets emit neutral and integration artifacts directly into private owner-specific directories below their Cargo `$OUT_DIR` and validate per-target manifests before embedding them. They never clean, discover, or copy -from one shared `dist` directory. The dependency installer takes an exclusive -cross-process lock; Vite, Vitest, TypeScript, ESLint, Prettier, both embed -builds, and the CLI Prebid builder take a shared lock for their whole process, -so `npm ci` cannot remove `node_modules` beneath a reader. Owner manifests -record a digest of their complete source inputs and -reject stale output. This removes stale/partial discovery races while allowing -parallel readers. Every Rust and Node participant uses the same canonical lock -path and compatible shared/exclusive protocol; per-crate locks are invalid. +from one shared `dist` directory. A checked-in, dependency-free Node runner +outside `node_modules` acquires one atomic-directory lease at the canonical +project root, records PID/nonce/start metadata, and holds it while spawning the +requested command. It reclaims a stale lease only after the recorded process is +absent and a grace interval has elapsed. `npm ci`, Vite, Vitest, TypeScript, +ESLint, Prettier, both Cargo embed builds, and the CLI Prebid builder all run +through it, so commands are serialized and installation cannot replace +`node_modules` beneath a reader. Owner manifests record a digest of their +complete source inputs and reject stale output. All npm scripts, build scripts, +and workflows use the same runner and lock path; a repository guard rejects +bare `npx` or unwrapped browser-tool invocations. Per-crate locks are invalid. The integration build discovers immediate directories containing `index.ts` and emits one IIFE per entry point. An IIFE may call the external versioned @@ -945,27 +988,56 @@ inputs, exact or pattern-based route claims, and ordered script-source claims. These declarations are the shared input to CLI and runtime validation; they do not allocate clients, read secrets, or execute a rewriter. -A script-source claim is a pure -`claims_script_src(url) -> Replace | Remove | None` predicate with its owning -integration ID and source field. Native integration claims use the same parsed -URL/pattern semantics as their executable attribute rewriter. During schema-2 -validation, every `js_asset_proxy` asset URL is evaluated against claims from -earlier enabled integrations. Any earlier `Replace` or `Remove` conflicts with -the asset's `enabled` or `blocked` policy and fails validate, diff, push, and -startup, naming both integration tables and the URL. Placing `js_asset_proxy` -before the native owner is the explicit, visible per-URL override; terminal -blocking or replacement then follows the ordinary chain. Schema-1 compatibility -uses the frozen legacy sequence and does not apply new schema-2 overlap -strictness. `ts config migrate` evaluates its schema-2 candidate and refuses to -write a conflicting result until the operator reorders the parent or removes -the contradictory asset. +A script-source claim is a pure transition +`apply_script_source(context) -> Unchanged | Replace(new_url) | Remove` with its +owning integration ID and source field. `context` carries the exact element and +attribute names, parsed URL, and relevant `rel`/`as` tokens; it therefore does +not confuse arbitrary `src`/`href` attributes with `