From fde55b1d828037ab5f437aa70df83ca663e607fe Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 1 Sep 2026 14:45:53 +0530 Subject: [PATCH 1/8] docs: design mobile ad-render trace endpoint --- ...-mobile-ad-render-trace-endpoint-design.md | 733 ++++++++++++++++++ 1 file changed, 733 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md new file mode 100644 index 000000000..e46f9892b --- /dev/null +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -0,0 +1,733 @@ +# Mobile Ad-Rendering Trace Endpoint Design + +**Status:** Proposed + +**Issue:** [#1050 — Create debug endpoint for mobile user to trace ad rendering](https://github.com/IABTechLab/trusted-server/issues/1050) + +**Related work:** + +- [#1081 — Improvements to TS_CONSOLE for ad observability](https://github.com/IABTechLab/trusted-server/issues/1081) +- [#1074 — Request-phase timing](https://github.com/IABTechLab/trusted-server/pull/1074) +- [#1076 — Auction timing milestones](https://github.com/IABTechLab/trusted-server/pull/1076) +- [#974 — GPT runtime diagnostics](https://github.com/IABTechLab/trusted-server/pull/974) +- [#997 — Trusted Server delivery attribution](https://github.com/IABTechLab/trusted-server/pull/997) + +## 1. Summary + +Add a deployment-controlled, public, privacy-safe `GET /_ts/trace` page for a +mobile end user who needs to reproduce an ad-rendering problem and give support +an exportable diagnostic report. + +The endpoint is both a setup page and a report viewer. On the first visit it +enables the existing GPT diagnostics browser session and explains how to +reproduce the problem. The user then returns to the real publisher page and +reloads it. Trusted Server supplies redacted request context, while the existing +TS Console records GPT, auction, and render evidence. A `View trace results` +action creates one bounded, allowlisted snapshot in same-tab `sessionStorage` +and navigates to `/_ts/trace`. The endpoint reads that snapshot, presents a +mobile-first HTML report, and offers JSON export, copy, and progressive Web +Share actions. + +The design introduces no report database, server-side trace store, report ID, +target URL parameter, telemetry query, or publisher-origin change. It cannot +recover an event that happened before tracing was enabled; the user must +reproduce the problem. + +## 2. Problem and product interpretation + +Issue #1050 names three required data groups: + +1. Network information inspired by `fastly-debug.com`. +2. Auction information per ad slot, similar to the TS tracer and auction + telemetry. +3. End-user cookie information. + +The title additionally establishes two product constraints: the experience is +for a mobile user, and it is reached through an endpoint. A mobile user should +not need browser developer tools, Basic Authentication, a copied trace ID, or a +second copy of the affected page URL. + +A standalone request cannot know what occurred in a previous document. Exact +render evidence exists only while the publisher page is running in the browser. +The design therefore separates two responsibilities without separating the user +experience: + +- `/_ts/trace` owns setup, consolidated presentation, and export. +- TS Console owns observation of the real publisher page. + +The browser-local handoff joins them without introducing a backend report +service. + +## 3. Goals + +- Give a non-technical mobile user one memorable URL: + `https:///_ts/trace`. +- Capture evidence from a real publisher-page reproduction, not a synthetic + auction. +- Display a Fastly-inspired network summary for the traced publisher request. +- Report health for an explicit allowlist of Trusted Server cookies without + exposing their values. +- Present the versioned, allowlisted TS Console evidence for every retained GPT + slot and request cycle. +- Support a full report in a narrow mobile viewport without developer tools. +- Export the same allowlisted model as formatted JSON. +- Keep capture bounded, same-tab, temporary, and inactive by default. +- Preserve normal auction, GPT, rendering, origin, and caching behavior whenever + diagnostics is inactive. +- Keep core behavior platform-neutral while allowing Fastly to provide richer + optional request fields. + +## 4. Non-goals + +- Recovering a failure that occurred before tracing was enabled. +- Permanent history, cross-device retrieval, server upload, or support-ticket + integration. +- A database, distributed trace store, report token, or telemetry lookup. +- A target URL such as `/_ts/trace?target=/article`. +- Replaying an auction or treating a synthetic auction as evidence about a + publisher page. +- Exact parity with every field or active measurement on `fastly-debug.com`. +- Reading the browser's complete cookie jar, third-party cookies, cookie + attributes, or cookies withheld from the request. +- Exposing raw cookie values, EC IDs, EIDs, consent strings, unmasked IP + addresses, internal auction request IDs, creative markup, targeting maps, + cache URLs, or stack traces. +- Reimplementing TS Console auction and creative observability requested by + #1081. +- Querying Tinybird to build an interactive report. +- Adding exact provider-by-slot no-bid explanations before the auction model can + observe those dispositions. +- Direct `POST /auction` browser diagnostics in the first release. + +## 5. Decisions + +### 5.1 Public, redacted endpoint + +`/_ts/trace` is public when explicitly enabled by deployment configuration. It +is not placed under `/_ts/admin`, because the intended user is a layperson on a +phone and the existing Basic Authentication flow is unsuitable for that +journey. + +Public access is safe only because both the page and export use a strict +allowlist. The activation cookie is a feature toggle, not authentication. No +field becomes eligible merely because tracing is active. + +### 5.2 Reuse the existing diagnostics session + +The endpoint reuses `__Host-ts-console` and the existing GPT diagnostics +activation semantics rather than creating a second `ts-trace` session. The +cookie remains host-only, `Secure`, `HttpOnly`, `SameSite=Lax`, and +browser-session scoped. + +`/_ts/trace?enabled=false` clears the activation cookie and browser snapshot. +Other values, duplicate `enabled` parameters, and malformed directives fail +closed and do not mutate session state. + +### 5.3 Browser-local, explicit handoff + +TS Console remains memory-only during observation. It writes a report to +`sessionStorage` only after the user selects `View trace results`. The action: + +1. Builds the same versioned allowlisted snapshot used by export. +2. Adds the redacted request-context envelope. +3. Serializes and validates the size. +4. Stores it under one versioned key in the current tab. +5. Navigates the same tab to `/_ts/trace`. + +Continuous persistence is prohibited. Opening the endpoint in another tab does +not retrieve the snapshot. Closing the tab deletes it according to browser +session-storage semantics. + +### 5.4 Forward reproduction, not historical diagnosis + +The first endpoint visit enables tracing for subsequent eligible document +navigations. The setup page must say plainly that the user needs to return to +the affected page, reload it, and reproduce the problem. + +If the user replaced the affected URL in the address bar with `/_ts/trace`, the +page offers a `Return to previous page` action backed by browser history and +then instructs the user to reload once. The design does not claim that +back-forward-cache restoration caused a new server request. + +Support should preferably give the user the trace URL before reproduction. The +product does not attempt to discover the previous URL through `Referer`, because +address-bar navigation commonly omits it and relying on it would create +inconsistent behavior. + +### 5.5 Separate issue ownership + +#1050 defines the report shell, request context, mobile flow, browser-local +handoff, and export. #1081 remains the owner of creative numbering, auction +classification, bidder/price policy, terminology, and normalized auction/render +timing. + +The trace report consumes TS Console's public versioned export contract. It +does not read TS Console internals or create an alternate slot correlation +engine. + +## 6. User experience + +### 6.1 First visit: no captured report + +`GET /_ts/trace` returns a mobile-first HTML page with: + +- Title: `Trusted Server ad diagnostics`. +- State: `Trace ready` after the response establishes the session cookie. +- A short explanation that no previous ad failure can be recovered. +- Network and cookie health for the setup request, labeled `Setup request`. +- Primary action: `Return to previous page` when browser history permits. +- Secondary instructions: return to the affected page, reload once, reproduce + the problem, then select `View trace results`. +- Action to disable tracing. + +The page must not imply that setup-request network facts or an empty auction +section describe the affected page. + +### 6.2 Active publisher page + +The existing TS Console remains available. On mobile it gains a prominent +`View trace results` action. Selecting it never changes ad behavior; it only +snapshots retained observations and navigates after serialization succeeds. + +If the snapshot cannot be stored, the page remains in place, announces the +failure, and keeps the existing direct JSON export available. + +### 6.3 Report visit + +When a valid snapshot exists, `/_ts/trace` renders: + +1. Report summary and capture time. +2. Network and request section for the traced publisher document. +3. Trusted Server cookie-health section. +4. Auction and rendering section grouped by numbered slot. +5. Coverage and ambiguity section. +6. Export actions. +7. `Clear report and end tracing` action. + +The setup request's facts are not merged into or substituted for missing traced +page facts. Missing fields display `Unavailable`; missing evidence displays +`Not observed` or `Unknown`, following TS Console terminology. + +### 6.4 Mobile and accessibility requirements + +- Support viewport widths down to 320 CSS pixels without horizontal page + scrolling. +- Use a full-document report rather than the current floating 460-pixel panel. +- Use at least 44-by-44 CSS pixel primary touch targets. +- Keep export/end-trace actions reachable without covering report content. +- Use semantic headings, lists, tables only where they remain readable on a + narrow viewport, visible focus styles, and an `aria-live` status region. +- Do not rely on hover, color alone, badges alone, or precise pointer input. +- Preserve browser zoom and safe-area insets. +- Prefer native text and controls over a framework or new UI dependency. + +## 7. Architecture + +```text +First GET /_ts/trace + | + |-- core route builds setup request context + |-- response sets __Host-ts-console + |-- HTML explains forward reproduction + v +Real publisher document reload + | + |-- adapter supplies optional network facts + |-- core computes allowlisted cookie health + |-- core injects redacted TraceRequestContextV1 + |-- existing TS Console observes GPT and TS delivery + v +User selects "View trace results" + | + |-- JS builds bounded TraceReportV1 + |-- same-tab sessionStorage write + |-- location.assign('/_ts/trace') + v +Second GET /_ts/trace + | + |-- static report shell reads and validates TraceReportV1 + |-- mobile HTML renders sections + |-- local JSON/copy/share actions +``` + +### 7.1 Core responsibilities + +- Define configuration and route behavior. +- Register the route before publisher fallback on every supported adapter. +- Define the platform-neutral request-context and report-envelope schemas. +- Build cookie-health facts through read-only parsing. +- Convert `ClientInfo` and available geo data into the public network allowlist. +- Inject request context only into an active private diagnostics document. +- Apply response privacy and security headers. +- Ensure trace requests never reach the publisher origin. + +### 7.2 Adapter responsibilities + +- Register the named route with exact method handling. +- Populate optional `ClientInfo` fields available on the platform. +- Fastly may supply POP, HTTP version, TLS, JA4, H2 fingerprint, and edge + server data when the SDK exposes them. +- Other adapters return the same schema with unsupported fields absent. +- Adapter-specific errors omit optional facts rather than failing publisher + delivery. + +### 7.3 JavaScript responsibilities + +- Accept the immutable redacted request context at initialization. +- Preserve the existing bounded TS Console observation store. +- Build and validate `TraceReportV1` on explicit user action. +- Store only one report in same-tab `sessionStorage`. +- Render the report shell from the validated model. +- Implement download, copy, progressive Web Share, clearing, expiry, and + accessible status reporting. +- Never upload diagnostic data or issue a telemetry query. + +## 8. Route and configuration contract + +Add an explicit default-off option to the existing integration: + +```toml +[integrations.gpt_diagnostics] +enabled = true +trace_page_enabled = false +``` + +Rules: + +- `trace_page_enabled = true` requires `enabled = true`; invalid combinations + fail configuration validation. +- `GET /_ts/trace` returns the setup/report HTML and establishes the session. +- `GET /_ts/trace?enabled=false` returns the shell, clears the cookie, and asks + the client to clear the stored snapshot. +- `HEAD /_ts/trace` returns the same status and headers without a body but does + not mutate the cookie. +- All other methods return a local `405 Method Not Allowed` with `Allow: GET, +HEAD`. +- Disabled deployments return a local `404` for the exact route and never fall + through to the publisher origin. +- Extra path segments, encoded separators, duplicate parameters, and lookalike + paths do not match. +- The route never creates or refreshes an EC, ingests EIDs, runs an auction, + fetches the publisher origin, or emits auction telemetry. + +The current `?ts_console=1` and `?ts_console=0` activation flow remains +supported for technical users. Both activation surfaces drive the same cookie +and runtime; they must not create two concurrent diagnostic modes. + +## 9. Data contracts + +### 9.1 Request context + +The server injects one immutable `TraceRequestContextV1` into active diagnostic +documents: + +```text +TraceRequestContextV1 + schema_version: 1 + captured_at: RFC 3339 UTC timestamp + page: + origin: publisher origin + path: normalized path + network: + masked_client_ip?: string + country?: string + region?: string + asn?: u32 + http_version?: string + tls_protocol?: string + tls_cipher?: string + tls_ja4?: string + h2_fingerprint?: string + edge_hostname?: string + edge_region?: string + edge_pop?: string + cookies: + ts_ec: CookieHealth + ts_eids: CookieHealth + ts_tester: CookieHealth + diagnostics_session: CookieHealth +``` + +The page field omits query and fragment data. It does not contain origin-facing +URLs, referrers, or arbitrary headers. + +`masked_client_ip` uses a deterministic display-only mask for the current +request: IPv4 keeps at most the first 24 bits and IPv6 keeps at most the first +48 bits. The full address never enters HTML, JavaScript, browser storage, or +export. + +JA4 and H2 fingerprints are optional probabilistic identifiers. They are +included only when the deployment has separately enabled the existing +fingerprint diagnostic capability. Their absence is not an error. + +### 9.2 Cookie health + +```text +CookieHealth + state: + absent | present_valid | present_invalid | duplicate | unavailable + source: request + detail?: allowlisted enum +``` + +Allowed details describe shape, not value, for example `valid_ec_format`, +`malformed`, `oversized`, or `activation_pending_response`. + +The parser must inspect the incoming request before any diagnostics-cookie +sanitization, while preserving existing authoritative-cookie and consent +semantics. Inspection is read-only: it must not generate an EC, touch the +identity graph, sync partner IDs, or extend any cookie lifetime. + +Only Trusted Server-owned cookie names are reported. Arbitrary cookie names and +values are excluded. The endpoint cannot claim knowledge of browser attributes, +expiry, or cookies the browser withheld from the request. + +### 9.3 Report envelope + +```text +TraceReportV1 + schema_version: 1 + captured_at: RFC 3339 UTC timestamp + request_context: TraceRequestContextV1 + gpt_diagnostics: GptDiagnosticsExportV1-or-successor +``` + +The trace envelope owns request context and transport. TS Console continues to +own its nested schema. Compatibility is explicit: the viewer supports a small +documented set of TS Console schema versions and rejects unknown versions with +an actionable message rather than guessing. + +### 9.4 Storage limits and expiry + +- Storage key: a namespaced, versioned constant owned by the diagnostics + module. +- Maximum encoded report size: 512 KiB. +- Maximum report age: 15 minutes from `captured_at`. +- One report per tab; a new explicit snapshot replaces the old report. +- Invalid, oversized, expired, or unsupported reports are removed immediately. +- `Clear report and end tracing` removes the storage entry and clears the + activation cookie. + +These are product limits, not assumptions about browser quota. A storage write +failure is handled even when the report is below the application limit. + +## 10. Auction and rendering evidence + +The report uses TS Console's evidence model. It must preserve the distinction +between: + +- A Trusted Server opportunity. +- A provider response. +- A selected Trusted Server candidate. +- A GPT request and response. +- A non-empty GPT render. +- Trusted Server creative-bridge evidence. +- Creative load and viewability. +- A publisher or client-side refresh. + +The viewer must not infer that Trusted Server rendered an ad merely because GPT +reported a filled slot. Ambiguous and unattributed cycles remain explicit. + +Current diagnostics tokens exist only on delivered winning bids. No-bid, +failed, skipped, hidden, unresolved, and direct `/auction` paths can lack server +correlation. The report displays the available observed facts and `Unknown` +rather than manufacturing a correlation. + +Provider-call telemetry is auction-wide, while bid rows exist only for returned +bids. Exact `provider X was asked for slot Y` and exact no-bid causality require +a future provider-impression disposition model. That instrumentation is not +silently assumed by this design. + +Timing fields introduced by #1074/#1076 are consumed only after they merge and +are propagated through the live diagnostics contract. The report never queries +Tinybird, and it does not combine browser `performance.now()` values with +server-relative timing as though they were one clock. + +Bidder and winning price are included only if #1081 approves them in the public +TS Console export contract. #1050 does not independently weaken the existing +privacy policy. + +## 11. Network scope + +The report is inspired by Fastly Debug, not a clone. Version one uses facts +already present or reasonably addable to the platform request abstraction. + +Supported categories: + +- Masked client address. +- Country, region, and ASN when available. +- HTTP version. +- TLS protocol and cipher. +- Optional JA4 and H2 fingerprints. +- Edge hostname, region, and POP. +- Capture time. + +Explicitly excluded: + +- DNS resolver address and resolver ASN. +- Active bandwidth or speed tests. +- TCP congestion window, next hop, RTT, and retransmit counters. +- DDoS/internal Fastly classifications. +- Arbitrary request headers. +- Full client IP in HTML or export. + +Unsupported optional fields are omitted rather than populated with fabricated +fallbacks. + +## 12. Security and privacy + +### 12.1 Allowlist boundary + +The report serializer constructs a new public model field by field. It never +serializes request structs, cookie parsers, auction requests, telemetry rows, or +browser objects wholesale. + +Forbidden data includes: + +- Raw `Cookie` and `Set-Cookie` headers. +- EC IDs, EIDs, bidder user IDs, and consent strings. +- Unmasked client IP. +- Query strings and fragments. +- Fastly or internal request identifiers that can join to user-bearing logs. +- Internal `AuctionRequest.id`. +- Bid requests/responses, losing-bid payloads, targeting, creative markup, + cache URLs, and stack traces. + +### 12.2 Same-origin script visibility + +Publisher and third-party scripts running on the publisher origin can access +`sessionStorage`. Therefore the stored model must be safe even if read by any +same-origin script. A random storage key, closed shadow root, or public endpoint +does not change this requirement. + +### 12.3 Response hardening + +Both the endpoint and every active diagnostic publisher response are terminally +`private, no-store`. The endpoint also sends: + +- `Content-Type: text/html; charset=utf-8` +- `X-Content-Type-Options: nosniff` +- `Referrer-Policy: no-referrer` +- `Content-Security-Policy` restricting content to the endpoint's own static + assets and prohibiting framing +- A restrictive `Permissions-Policy` + +The endpoint makes no third-party requests. Dynamic JSON embedded in HTML uses +the repository's script-safe serializer and is never concatenated into +executable JavaScript. + +### 12.4 Shared templates and ESI + +Per-request trace context must never enter a shared template or ESI fragment. +The existing diagnostics private/no-store decision remains a load-bearing gate. +Tests must prove that late response-header handlers cannot make traced content +publicly cacheable. + +## 13. Failure handling + +- Disabled route: local privacy-safe `404`. +- Unsupported method: local `405`; never publisher fallback. +- Optional platform fact unavailable: omit the field and continue. +- Cookie parser failure: report `present_invalid` without the value. +- Diagnostics context serialization failure: omit the context, log a bounded + server error, and preserve publisher delivery. +- TS Console capture failure: fail open for advertising and show incomplete + coverage in diagnostics. +- Storage unavailable, quota exceeded, or serialization oversized: remain on + the publisher page, announce the error, and offer direct download. +- Missing snapshot on endpoint: show setup state, not an empty successful + report. +- Expired, malformed, or unknown report schema: clear it and explain that the + user must reproduce again. +- Clipboard or Web Share unavailable: keep JSON download available. +- Export failure: retain the on-screen report and show an accessible error. + +Diagnostic failures must never suppress, delay, add, remove, or reorder GPT +requests, auctions, targeting, or creative rendering. + +## 14. Testing strategy + +### 14.1 Core unit tests + +- Configuration defaults off and rejects trace-page enablement without GPT + diagnostics. +- Exact route, query, method, encoded-path, and fallback behavior. +- Session cookie set/clear attributes and duplicate-directive fail-closed + behavior. +- Endpoint skips EC generation/finalization, EID ingestion, auction, telemetry, + and origin fetch. +- Cookie-health parser covers absent, valid, malformed, duplicate, non-UTF-8, + and oversized inputs without retaining values. +- Request-context serializer masks IPv4/IPv6 and omits query, raw headers, IDs, + and unsupported fields. +- Active responses remain terminally private/no-store under hostile late header + overrides. +- Dynamic HTML/JSON values cannot close elements or create executable script. + +### 14.2 Adapter parity tests + +- Fastly route registration and optional field mapping. +- Axum, Cloudflare, and Spin return the common route/schema with unavailable + fields omitted. +- Named route failures never fall through to publisher origin. +- HEAD and unsupported methods behave identically across adapters. +- Fastly fingerprint fields respect the existing fingerprint-debug gate. + +### 14.3 JavaScript unit tests + +- Explicit snapshot only; no continuous `sessionStorage` writes. +- Size limit, schema validation, expiry, replacement, clearing, and storage + exceptions. +- Same-tab navigation occurs only after a successful write. +- Viewer handles absent optional network facts and every cookie-health state. +- Forbidden fields never enter storage or export fixtures. +- Download filename and MIME type are deterministic. +- Copy and Web Share success, rejection, absence, and fallback behavior. +- 320-pixel layout, keyboard navigation, focus handling, and accessible status + announcements. + +### 14.4 Browser integration tests + +- First endpoint visit sets the session and shows setup state. +- A real fixture reload activates diagnostics and captures multiple slots. +- `View trace results` navigates in the same tab and renders the captured + request context and slot evidence. +- Empty, filled, ambiguous, no-candidate, and unattributed slot states remain + distinct. +- Reloading the trace page retains an unexpired same-tab report. +- A new tab cannot access the original tab's report. +- Disabling clears both cookie and report. +- Back-forward-cache restoration is not described as a fresh traced request; + the setup page tells the user to reload. +- Export JSON matches the displayed versioned model. +- Inactive publisher traffic has no trace assets, storage access, listeners, or + cache-policy change. + +### 14.5 Manual acceptance + +Verify on current iOS Safari and Android Chrome using a representative publisher +fixture: + +- A layperson can follow the page instructions without developer tools. +- Touch targets, scrolling, zoom, safe areas, download, copy, and native share + behavior are usable. +- The user can distinguish setup information from captured-page information. +- A failed share or download does not lose the visible report. + +## 15. Rollout and observability + +- Ship default-off. +- Enable first in a controlled staging publisher configuration. +- Validate response cache headers and CDN behavior before production use. +- Validate with redacted fixtures before real publisher traffic. +- Log only route outcome, schema version, report-present boolean, and bounded + error category. Never log report contents or cookie/network values. +- Roll back by disabling `trace_page_enabled`; existing `?ts_console=1` + diagnostics remain independently configurable. + +## 16. Acceptance criteria + +1. With the feature disabled, exact trace routes return local `404` and ordinary + traffic is unchanged. +2. A mobile user can enable tracing by opening only `/_ts/trace`; no target URL, + credentials, or trace ID is required. +3. The setup page accurately explains that the problem must be reproduced after + activation. +4. A subsequent real publisher-page reload captures redacted request context + and existing TS Console evidence without altering ad behavior. +5. `View trace results` transfers one bounded snapshot in the same tab and opens + the report page without server-side storage. +6. The report separates network, cookie health, auction/render evidence, and + coverage/unknowns. +7. JSON export contains the same versioned allowlisted information shown on the + page. +8. No raw cookies, user IDs, full IPs, consent strings, query strings, internal + auction IDs, targeting, or creative payloads appear in HTML, browser storage, + logs, or export. +9. Trace HTML and active publisher pages remain terminally private/no-store. +10. Missing platform fields, incomplete auction correlation, storage failure, + and unavailable share APIs degrade honestly without affecting advertising. +11. The full report is usable at 320 CSS pixels and with keyboard/screen-reader + navigation. +12. Auction fields owned by #1081 are consumed through its versioned public + contract rather than duplicated in #1050. + +## 17. Implementation sequencing + +This design is one product flow but should be implemented in dependency order: + +1. Core request-context schema, cookie-health classification, configuration, + and endpoint shell. +2. Adapter route parity and Fastly optional network enrichment. +3. TS Console request-context envelope and explicit same-tab snapshot handoff. +4. Mobile viewer, export/copy/share, expiry, and clearing. +5. Integration with the current TS Console schema. +6. Additive adoption of #1081 and #1074/#1076 fields after their contracts + merge. +7. Browser, privacy, cache, and real-device acceptance. + +The implementation plan must not claim completion of #1081 or the open timing +PRs as part of #1050. If those dependencies are unavailable, the report ships +only with current observed auction/render evidence and labels unavailable fields +honestly. + +## 18. Rejected alternatives + +### `/_ts/admin/trace?target=/article` + +Rejected because it requires the user to supply the affected URL twice, adds +target validation and open-redirect risk, and is unsuitable for a layperson. + +### Basic Authentication + +Rejected for the mobile end-user workflow. Authentication also would not make +it safe to inject raw secrets into a publisher page containing third-party +JavaScript. + +### Server-managed trace sessions + +Rejected because they require shared storage, report authorization, expiry, +deletion, and operational infrastructure beyond the issue's needs. + +### Synthetic auction on the endpoint + +Rejected because it does not reproduce the real page's DOM, GPT lifecycle, +consent context, refresh path, or auction timing and could produce misleading +results. + +### Endpoint-only report with no publisher-page integration + +Rejected because a request to `/_ts/trace` cannot observe rendering that +occurred in another document. + +### Query-only in-page console + +The existing `?ts_console=1` flow remains supported, but it is not the complete +answer to #1050: the issue asks for a memorable mobile endpoint and a +Fastly-style consolidated HTML report. The endpoint/viewer builds on rather +than replaces the console. + +### Cross-page URL payload + +Rejected because fragments or query strings containing the report create URL +length, history, logging, referrer, and accidental-sharing risks. + +## 19. Known limitations + +- The user must reproduce the problem after enabling tracing. +- Same-tab storage prevents cross-device and cross-tab sharing; JSON export is + the handoff artifact. +- Publisher-origin scripts can read the stored public-safe report. +- Browser privacy settings may disable storage, clipboard, download, or share + capabilities. +- Current server/browser correlation does not cover every no-bid, skipped, + failed, hidden, unresolved, or direct-auction path. +- Fastly-only transport details do not exist on every adapter. +- The current 30-pixel TS Console controls are not sufficient for this mobile + report; the endpoint uses independent 44-pixel touch targets. +- `fastly-debug.com` fields that require resolver, TCP, or active speed probes + remain out of scope. + +These limitations are displayed in operator documentation and, where relevant, +in the report itself. They are not hidden behind apparently successful empty +states. From 8899dcadc89d1c4db3c5902078fa3a9e2cbf0c42 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 1 Sep 2026 15:33:05 +0530 Subject: [PATCH 2/8] docs: harden mobile ad trace design --- ...-mobile-ad-render-trace-endpoint-design.md | 754 ++++++++++++++---- 1 file changed, 591 insertions(+), 163 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md index e46f9892b..6ed9f8c48 100644 --- a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -16,16 +16,17 @@ Add a deployment-controlled, public, privacy-safe `GET /_ts/trace` page for a mobile end user who needs to reproduce an ad-rendering problem and give support -an exportable diagnostic report. +an exportable diagnostic report. Visiting the page is read-only. The user +intentionally enables or ends tracing with a same-origin POST action. The endpoint is both a setup page and a report viewer. On the first visit it -enables the existing GPT diagnostics browser session and explains how to -reproduce the problem. The user then returns to the real publisher page and -reloads it. Trusted Server supplies redacted request context, while the existing -TS Console records GPT, auction, and render evidence. A `View trace results` -action creates one bounded, allowlisted snapshot in same-tab `sessionStorage` -and navigates to `/_ts/trace`. The endpoint reads that snapshot, presents a -mobile-first HTML report, and offers JSON export, copy, and progressive Web +offers a large `Enable tracing` action and explains how to reproduce the +problem. The user then returns to the real publisher page and reloads it. +Trusted Server supplies redacted request context, while the existing TS Console +records GPT, auction, and render evidence. A `View trace results` action creates +one bounded, allowlisted snapshot in same-tab `sessionStorage` and navigates to +`/_ts/trace`. The endpoint reads that untrusted snapshot, validates it, presents +a mobile-first HTML report, and offers JSON export, copy, and progressive Web Share actions. The design introduces no report database, server-side trace store, report ID, @@ -68,10 +69,12 @@ service. - Report health for an explicit allowlist of Trusted Server cookies without exposing their values. - Present the versioned, allowlisted TS Console evidence for every retained GPT - slot and request cycle. + slot and request cycle that fits the public report bounds, with explicit + omission counts when deterministic size truncation is required. - Support a full report in a narrow mobile viewport without developer tools. - Export the same allowlisted model as formatted JSON. -- Keep capture bounded, same-tab, temporary, and inactive by default. +- Keep the supported capture journey bounded, same-tab, temporary, and inactive + by default without treating browser storage as a security boundary. - Preserve normal auction, GPT, rendering, origin, and caching behavior whenever diagnostics is inactive. - Keep core behavior platform-neutral while allowing Fastly to provide richer @@ -101,7 +104,7 @@ service. ## 5. Decisions -### 5.1 Public, redacted endpoint +### 5.1 Public, redacted endpoint with intentional activation `/_ts/trace` is public when explicitly enabled by deployment configuration. It is not placed under `/_ts/admin`, because the intended user is a layperson on a @@ -112,16 +115,27 @@ Public access is safe only because both the page and export use a strict allowlist. The activation cookie is a feature toggle, not authentication. No field becomes eligible merely because tracing is active. +`GET /_ts/trace` is read-only and never activates or ends tracing. Activation +and deactivation use an in-page same-origin `fetch` POST accepted only when its +fixed custom action header, `Origin`, and Fetch Metadata identify the publisher +origin. Requests with a conflicting or missing signal fail closed. The POST +updates the existing page rather than adding a history entry, so browser Back +can still reach the article. This prevents an unrelated site from silently +toggling diagnostics through a top-level GET while preserving a one-URL, +one-tap mobile workflow. This control does not defend against code already +executing on the publisher origin. + ### 5.2 Reuse the existing diagnostics session The endpoint reuses `__Host-ts-console` and the existing GPT diagnostics activation semantics rather than creating a second `ts-trace` session. The -cookie remains host-only, `Secure`, `HttpOnly`, `SameSite=Lax`, and -browser-session scoped. - -`/_ts/trace?enabled=false` clears the activation cookie and browser snapshot. -Other values, duplicate `enabled` parameters, and malformed directives fail -closed and do not mutate session state. +cookie remains host-only, `Secure`, `HttpOnly`, and `SameSite=Lax`. +`POST /_ts/trace/enable` sets it with a fixed 30-minute `Max-Age` and does not +refresh that lifetime on publisher requests; `POST /_ts/trace/end` clears it. +Neither action accepts state-changing query parameters. The shorter endpoint +lifetime bounds accidental private/no-store operation if a user forgets to end +tracing; the existing technical query flow keeps its existing session-cookie +semantics. ### 5.3 Browser-local, explicit handoff @@ -134,20 +148,26 @@ TS Console remains memory-only during observation. It writes a report to 4. Stores it under one versioned key in the current tab. 5. Navigates the same tab to `/_ts/trace`. -Continuous persistence is prohibited. Opening the endpoint in another tab does -not retrieve the snapshot. Closing the tab deletes it according to browser -session-storage semantics. +Continuous persistence is prohibited. Same-tab navigation is the supported +handoff, not an isolation guarantee: a browser may copy session storage into an +opener-created tab or preserve it during session restore. Same-origin scripts +and service workers can read, replace, or forge the snapshot. The viewer +therefore treats it as untrusted, applies an application-level expiry, and +labels it browser-observed rather than authoritative. ### 5.4 Forward reproduction, not historical diagnosis -The first endpoint visit enables tracing for subsequent eligible document +The user's explicit activation enables tracing for subsequent eligible document navigations. The setup page must say plainly that the user needs to return to the affected page, reload it, and reproduce the problem. If the user replaced the affected URL in the address bar with `/_ts/trace`, the page offers a `Return to previous page` action backed by browser history and -then instructs the user to reload once. The design does not claim that -back-forward-cache restoration caused a new server request. +then instructs the user to reload once. History is only a convenience: it may +lead to a messaging app, search page, or unrelated site. The page includes a +fallback instruction to reopen the affected article on the same hostname and +in the same tab. The design does not claim that back-forward-cache restoration +caused a new server request. Support should preferably give the user the trace URL before reproduction. The product does not attempt to discover the previous URL through `Referer`, because @@ -161,9 +181,11 @@ handoff, and export. #1081 remains the owner of creative numbering, auction classification, bidder/price policy, terminology, and normalized auction/render timing. -The trace report consumes TS Console's public versioned export contract. It +Version one consumes `GptDiagnosticsExportV1` through TS Console's public export +contract and projects it into a distinct redacted `TraceGptDiagnosticsV1`. It does not read TS Console internals or create an alternate slot correlation -engine. +engine. #1081 and #1074/#1076 are additive follow-up work and are not release +gates for this version. ## 6. User experience @@ -172,13 +194,22 @@ engine. `GET /_ts/trace` returns a mobile-first HTML page with: - Title: `Trusted Server ad diagnostics`. -- State: `Trace ready` after the response establishes the session cookie. +- State derived from the setup request: `Tracing is off` unless the server + observed a valid existing diagnostics cookie, including one activated through + the technical query flow. - A short explanation that no previous ad failure can be recovered. - Network and cookie health for the setup request, labeled `Setup request`. -- Primary action: `Return to previous page` when browser history permits. +- Primary action: `Enable tracing`, implemented as an in-page same-origin fetch + POST that does not add a history entry. +- After activation and a successful state-verification request, state: `Tracing +is on — cookie observed by server`. +- After activation, primary action: `Return to previous page` when browser + history permits. - Secondary instructions: return to the affected page, reload once, reproduce the problem, then select `View trace results`. -- Action to disable tracing. +- A recovery instruction to reopen the affected article on the exact same + hostname and in the same tab if browser history is not useful. +- Action to end tracing when it is active. The page must not imply that setup-request network facts or an empty auction section describe the affected page. @@ -189,8 +220,12 @@ The existing TS Console remains available. On mobile it gains a prominent `View trace results` action. Selecting it never changes ad behavior; it only snapshots retained observations and navigates after serialization succeeds. -If the snapshot cannot be stored, the page remains in place, announces the -failure, and keeps the existing direct JSON export available. +If a valid bounded snapshot is built but browser storage rejects it, the page +remains in place, announces the storage failure, and offers a direct download +of that same combined `TraceReportV1` envelope. If projection, validation, or +size bounding fails before a valid report exists, the page reports capture +failure and does not mislabel the existing GPT-only export as an equivalent +fallback. ### 6.3 Report visit @@ -208,6 +243,16 @@ The setup request's facts are not merged into or substituted for missing traced page facts. Missing fields display `Unavailable`; missing evidence displays `Not observed` or `Unknown`, following TS Console terminology. +The report begins with `Browser-observed, unverified diagnostic data`. It does +not claim that the snapshot is authentic or suitable as forensic or security +evidence. + +`Copy` copies formatted JSON. `Share` supplies the same JSON file to the native +Web Share sheet only after an explicit tap and tells the user that the selected +app will receive it. If file sharing is unsupported or rejected, the viewer +keeps Copy and Download available; it does not silently share a URL or upload +the report. + ### 6.4 Mobile and accessibility requirements - Support viewport widths down to 320 CSS pixels without horizontal page @@ -224,11 +269,17 @@ page facts. Missing fields display `Unavailable`; missing evidence displays ## 7. Architecture ```text -First GET /_ts/trace +GET /_ts/trace | - |-- core route builds setup request context + |-- early reserved-route classifier terminates locally + |-- HTML explains forward reproduction; no state mutation + v +POST /_ts/trace/enable after explicit user action + | + |-- validates same-origin request signals |-- response sets __Host-ts-console - |-- HTML explains forward reproduction + |-- client requests /_ts/trace/state + |-- server reports whether the new request carried a valid cookie v Real publisher document reload | @@ -243,7 +294,7 @@ User selects "View trace results" |-- same-tab sessionStorage write |-- location.assign('/_ts/trace') v -Second GET /_ts/trace +Report GET /_ts/trace | |-- static report shell reads and validates TraceReportV1 |-- mobile HTML renders sections @@ -253,7 +304,8 @@ Second GET /_ts/trace ### 7.1 Core responsibilities - Define configuration and route behavior. -- Register the route before publisher fallback on every supported adapter. +- Provide a shared exact-path reserved-route classifier that runs before event + context, filters, auctions, named routes, or publisher fallback. - Define the platform-neutral request-context and report-envelope schemas. - Build cookie-health facts through read-only parsing. - Convert `ClientInfo` and available geo data into the public network allowlist. @@ -263,10 +315,11 @@ Second GET /_ts/trace ### 7.2 Adapter responsibilities -- Register the named route with exact method handling. +- Invoke the reserved-route classifier at the earliest adapter dispatch point + with exact path and method handling. - Populate optional `ClientInfo` fields available on the platform. -- Fastly may supply POP, HTTP version, TLS, JA4, H2 fingerprint, and edge - server data when the SDK exposes them. +- Fastly may supply bounded POP, HTTP version, TLS, and edge-server data when + the SDK exposes them. JA4 and H2 fingerprints are excluded from version one. - Other adapters return the same schema with unsupported fields absent. - Adapter-specific errors omit optional facts rather than failing publisher delivery. @@ -275,11 +328,14 @@ Second GET /_ts/trace - Accept the immutable redacted request context at initialization. - Preserve the existing bounded TS Console observation store. -- Build and validate `TraceReportV1` on explicit user action. -- Store only one report in same-tab `sessionStorage`. +- Build and validate `TraceReportV1` with a redacted + `TraceGptDiagnosticsV1` projection on explicit user action. +- Store only one supported report for the same-tab workflow in + `sessionStorage`, while treating its contents as untrusted. - Render the report shell from the validated model. -- Implement download, copy, progressive Web Share, clearing, expiry, and - accessible status reporting. +- Implement equivalent combined-report download, formatted-JSON copy, + progressive JSON-file Web Share, clearing, expiry, and accessible status + reporting. - Never upload diagnostic data or issue a telemetry query. ## 8. Route and configuration contract @@ -296,23 +352,102 @@ Rules: - `trace_page_enabled = true` requires `enabled = true`; invalid combinations fail configuration validation. -- `GET /_ts/trace` returns the setup/report HTML and establishes the session. -- `GET /_ts/trace?enabled=false` returns the shell, clears the cookie, and asks - the client to clear the stored snapshot. -- `HEAD /_ts/trace` returns the same status and headers without a body but does - not mutate the cookie. -- All other methods return a local `405 Method Not Allowed` with `Allow: GET, -HEAD`. -- Disabled deployments return a local `404` for the exact route and never fall - through to the publisher origin. -- Extra path segments, encoded separators, duplicate parameters, and lookalike - paths do not match. -- The route never creates or refreshes an EC, ingests EIDs, runs an auction, - fetches the publisher origin, or emits auction telemetry. +- Operator documentation beside this option states that the public page makes + the allowlisted presence/validity of four HttpOnly Trusted Server cookies + visible to same-origin JavaScript whenever the feature is enabled. It also + states that masked IP prefixes and coarse geo remain potentially personal or + pseudonymous network data. Enabling the option is the deployment's explicit + acceptance of those bounded disclosures. +- `GET /_ts/trace` returns the setup/report shell without changing cookies or + browser storage at the HTTP layer. After load, the explicitly included viewer + script may remove a rejected or expired local entry. Unrelated query + parameters do not activate or deactivate tracing and are not reflected into + the page or export. +- `HEAD /_ts/trace` returns the GET status and headers without a body or state + mutation. +- `GET /_ts/trace/state` returns private/no-store JSON containing only + `observed_active: true|false`, determined from whether that request carried + exactly one valid diagnostics cookie. `HEAD` returns the same status and + headers without a body. Other methods return local 405 responses. +- `GET` and `HEAD` on exactly `/_ts/trace/assets/v1.js` and + `/_ts/trace/assets/v1.css` return fixed versioned assets. They contain no + request or report data and may use immutable public caching. Other methods + return local 405 responses. These v1 URLs are immutable byte contracts: any + JS or CSS byte change requires a new asset-set URL such as `v2.js`/`v2.css` + and an updated shell reference; a release never replaces bytes at a published + immutable URL. +- `POST /_ts/trace/enable` accepts no query parameters, validates an empty body, + the exact `X-TS-Trace-Action: enable` header, and same-origin request signals; + sets the diagnostics cookie; and returns a small local JSON result. A success + response means only that the server requested the cookie change. +- `POST /_ts/trace/end` applies the same validation, clears the diagnostics + cookie using `X-TS-Trace-Action: end`, and returns a small local JSON result. + After explicit user confirmation, client JavaScript independently attempts + local report deletion and the end POST. Neither result gates the other. +- For both POST paths, absent or exactly-zero `Content-Length` is accepted, + `Transfer-Encoding` is rejected, and the adapter reads at most one byte when + it must verify an absent length. Any body byte or positive/invalid length + returns local `413 Payload Too Large` without draining or processing an + unbounded body. The one-byte read inherits a maximum two-second adapter + request-body deadline; timeout returns local `408 Request Timeout` with no + mutation. +- State-changing POSTs require an `Origin` exactly matching the canonical + request origin and `Sec-Fetch-Site: same-origin`. Missing, conflicting, + malformed, cross-site, or duplicate control values return local `403` without + mutation. This deliberately targets current supported mobile browsers rather + than weakening the check for legacy clients. +- The canonical request origin comes from adapter-owned inbound URL/scheme and + validated authority data, never an arbitrary forwarded header. Both it and + the single parsed `Origin` header are serialized with lowercase host and + default ports removed before exact comparison. Invalid or multi-valued host, + authority, scheme, or origin input fails closed. +- Unsupported methods on a shell or state-changing path return a local 405 + Method Not Allowed response with the path-specific `Allow` header. +- Disabled deployments return a local `404` for the complete trace route set, + including assets, and never fall through to the publisher origin. +- The `/_ts/trace` namespace is reserved. A trailing slash, extra path segment, + unsupported asset name, repeated separator, or lookalike beneath that + namespace returns a local `404`; an encoded separator or ambiguous dot + segment returns a local `400`. None falls through to the publisher origin. + The adapter classifies from its canonical parsed path while retaining enough + raw-path information to reject ambiguous encodings consistently. + +Every adapter implements the following order: + +1. Parse the method, canonical host/origin, path, query, and bounded headers + required for route safety. +2. Classify an exact Trusted Server reserved path. +3. For a trace path, terminate locally after only trace-specific validation and + bounded request-context inspection, including an optional read-only platform + geo lookup used solely for the displayed setup request. +4. For all other paths, continue through the adapter's ordinary event context, + authentication, request filters, geo enrichment, EC/EID processing, named + routes, auction handling, telemetry, and publisher fallback. + +Consequently a trace route never creates or finalizes an ordinary event +context, invokes publisher-configured filters, creates or refreshes an EC, +ingests EIDs, runs an auction, fetches the publisher origin, or emits auction +telemetry. Tests must verify ordering in Fastly, Axum, Cloudflare, and Spin; +ordinary named-route registration alone does not satisfy this contract. + +After an enable or end POST succeeds, the client performs a no-store state GET. +It claims `Tracing is on — cookie observed by server` only when that separate +request reports active, and `Tracing is off — cookie absent on server request` +only when it reports inactive. A mismatch or failed verification is +`Activation unconfirmed` or `Deactivation unconfirmed` and offers an idempotent +retry. These are server-observation statements, not proof that browser state is +authentic: same-origin service workers can forge or suppress the whole exchange. + +Versioned assets may remain in a CDN or browser cache after the feature is +disabled. Cache misses return the configured local 404, but rollback relies on +the uncached shell, state, and action routes being disabled; inert cached assets +alone cannot activate tracing or access a report page. The current `?ts_console=1` and `?ts_console=0` activation flow remains supported for technical users. Both activation surfaces drive the same cookie -and runtime; they must not create two concurrent diagnostic modes. +and runtime; they must not create two concurrent diagnostic modes. That +pre-existing query flow has its existing top-level-navigation activation risk; +#1050 neither expands it to the new trace GET nor claims to remediate it. ## 9. Data contracts @@ -325,9 +460,6 @@ documents: TraceRequestContextV1 schema_version: 1 captured_at: RFC 3339 UTC timestamp - page: - origin: publisher origin - path: normalized path network: masked_client_ip?: string country?: string @@ -336,8 +468,6 @@ TraceRequestContextV1 http_version?: string tls_protocol?: string tls_cipher?: string - tls_ja4?: string - h2_fingerprint?: string edge_hostname?: string edge_region?: string edge_pop?: string @@ -348,17 +478,27 @@ TraceRequestContextV1 diagnostics_session: CookieHealth ``` -The page field omits query and fragment data. It does not contain origin-facing -URLs, referrers, or arbitrary headers. +The request-context envelope intentionally contains no page URL, path, +referrer, query, or fragment. During the field-by-field trace projection, +`GptDiagnosticsExportV1.page.origin` is retained after validation and its +`pathname` is replaced with the literal `/[redacted]`. The trace viewer accepts +only that literal. Version one therefore does not store or export an exact page +path. Any future route-template policy requires a new schema and privacy review +because paths can contain accounts, emails, preview tokens, and other secrets. `masked_client_ip` uses a deterministic display-only mask for the current request: IPv4 keeps at most the first 24 bits and IPv6 keeps at most the first 48 bits. The full address never enters HTML, JavaScript, browser storage, or -export. +export. These prefixes can still be personal or pseudonymous network data; the +report labels them as approximate network identifiers and the operator privacy +decision covers them explicitly. -JA4 and H2 fingerprints are optional probabilistic identifiers. They are -included only when the deployment has separately enabled the existing -fingerprint diagnostic capability. Their absence is not an error. +All platform strings are normalized to printable characters and bounded before +they enter logs, HTML, storage, or export. Country uses at most 2 ASCII +characters; region and POP 32 UTF-8 bytes; HTTP/TLS enumerations 32 bytes; and +edge hostname/region 128 bytes. Values that fail their field contract are +omitted and produce only a bounded error category. JA4 and H2 fingerprints are +not members of `TraceRequestContextV1`. ### 9.2 Cookie health @@ -367,20 +507,62 @@ CookieHealth state: absent | present_valid | present_invalid | duplicate | unavailable source: request - detail?: allowlisted enum + detail?: + valid_ec_format | valid_eids_format | valid_tester_value + | valid_diagnostics_value | malformed | oversized + | unsupported_value | multiple_values + | header_too_large | header_not_utf8 ``` -Allowed details describe shape, not value, for example `valid_ec_format`, -`malformed`, `oversized`, or `activation_pending_response`. - -The parser must inspect the incoming request before any diagnostics-cookie -sanitization, while preserving existing authoritative-cookie and consent -semantics. Inspection is read-only: it must not generate an EC, touch the -identity graph, sync partner IDs, or extend any cookie lifetime. - -Only Trusted Server-owned cookie names are reported. Arbitrary cookie names and -values are excluded. The endpoint cannot claim knowledge of browser attributes, -expiry, or cookies the browser withheld from the request. +Details describe shape, never value. `absent` has no detail; `duplicate` uses +`multiple_values`; `unavailable` uses `header_too_large` or `header_not_utf8`; +and a valid state uses its cookie-specific valid detail. + +The classifier uses this deterministic contract: + +- Inspect all `Cookie` header fields in wire order, up to a combined 16 KiB. + Exceeding the cap or encountering any non-UTF-8 header makes all four states + `unavailable`; no partial result is presented as authoritative. +- Split each readable header on semicolons and trim optional ASCII whitespace. + A valid pair contains a non-empty RFC 6265 token name, one `=`, and the + remaining bytes as its value; additional `=` bytes belong to the value. Empty + segments and malformed pairs with an unrelated name are ignored. A segment + with no `=` counts as one malformed reserved occurrence only when its first + whitespace-delimited token is exactly a reserved name; a name such as + `ts-ec-extra` remains unrelated. No malformed unrelated pair poisons a + reserved-cookie result. +- Count exact, case-sensitive reserved names before passing values to existing + parsers. Zero occurrences is `absent`; more than one is `duplicate`, + regardless of whether one value would otherwise be valid. Duplicate + precedence is therefore diagnostic rather than first- or last-value + selection. +- Per-value limits are 512 bytes for `ts-ec`, 8 KiB for `ts-eids`, and 16 bytes + each for `ts-tester` and `__Host-ts-console`. A single value beyond its limit + is `present_invalid/oversized`; it does not change the other three states. +- One `ts-ec` occurrence is valid only when the canonical EC cookie validator + accepts its complete value. +- One `ts-eids` occurrence is valid only when the existing bounded Base64/JSON + EID parser accepts its complete value, including its current 8 KiB value cap. +- One `ts-tester` occurrence is valid only when its value is exactly `true`. +- One `__Host-ts-console` occurrence is valid only when its value is exactly + `1`. +- A single rejected value is `present_invalid` with exactly one of the public + details `malformed`, `oversized`, or `unsupported_value`. Parser error text + and the value itself never enter the report or logs. + +The parser inspects the incoming request before diagnostics-cookie sanitation, +while preserving existing authoritative-cookie and consent semantics. It must +scan without using the current lossy `CookieJar` representation, which skips +malformed pairs and cannot preserve duplicate evidence. Inspection is +read-only: it must not generate an EC, touch the identity graph, sync partner +IDs, or extend any cookie lifetime. + +Only those four Trusted Server-owned cookie names are reported. Arbitrary +cookie names and values are excluded. The endpoint cannot claim knowledge of +browser attributes, expiry, or cookies the browser withheld from the request. +Because the result reveals presence and validity of HttpOnly cookies to +same-origin JavaScript, enabling this public feature requires an explicit +operator privacy decision documented beside `trace_page_enabled`. ### 9.3 Report envelope @@ -389,28 +571,132 @@ TraceReportV1 schema_version: 1 captured_at: RFC 3339 UTC timestamp request_context: TraceRequestContextV1 - gpt_diagnostics: GptDiagnosticsExportV1-or-successor + gpt_diagnostics: TraceGptDiagnosticsV1 + truncation: + omitted_request_cycles: u16 + omitted_callback_issues: u16 + omitted_attribution_issues: u16 + omitted_nested_values: u16 ``` -The trace envelope owns request context and transport. TS Console continues to -own its nested schema. Compatibility is explicit: the viewer supports a small -documented set of TS Console schema versions and rejects unknown versions with -an actionable message rather than guessing. +`TraceGptDiagnosticsV1` is a trace-owned projection sourced only from +`GptDiagnosticsExportV1`. It contains: + +- `schema_version: 1` and `source_schema_version: 1`; +- the source `capturedAt` value; +- `page.origin` after validation and `page.pathname` fixed to `/[redacted]`; +- field-for-field allowlisted copies of the current v1 slots, requests, + callback issues, attribution issues, coverage, and metadata, subject to the + bounds and truncation below. + +It is deliberately not named or represented as `GptDiagnosticsExportV1`, +because the fixed pathname and trace-level bounds change the source field +semantics. TS Console continues to own the source schema; the trace envelope +owns its public projection and transport. The initial compatibility matrix is +exactly `TraceReportV1` plus `TraceGptDiagnosticsV1`, sourced from +`GptDiagnosticsExportV1`. The viewer rejects every unknown outer, trace-auction, +or source version with an actionable message rather than guessing. A future TS +Console successor requires an additive source compatibility change and, if the +public projection changes, a new trace-envelope version. + +Origin validation requires a parseable HTTP(S) origin whose canonical +serialization exactly equals `window.location.origin`; credentials, paths, +queries, and fragments are rejected. Outer and source capture times require +strict RFC 3339 UTC strings and must be within 60 seconds of `stored_at_ms`. +`TraceRequestContextV1.captured_at` requires strict RFC 3339 UTC but may be older +because it represents the publisher document request rather than snapshot time. ### 9.4 Storage limits and expiry - Storage key: a namespaced, versioned constant owned by the diagnostics module. -- Maximum encoded report size: 512 KiB. -- Maximum report age: 15 minutes from `captured_at`. -- One report per tab; a new explicit snapshot replaces the old report. -- Invalid, oversized, expired, or unsupported reports are removed immediately. -- `Clear report and end tracing` removes the storage entry and clears the - activation cookie. +- Stored value: `{ stored_at_ms, report }`, where `stored_at_ms` is generated by + the capture code and is not taken from report content. +- Maximum encoded size: 512 KiB, defined as the byte length of the complete + compact UTF-8 `{ stored_at_ms, report }` JSON measured with `TextEncoder` + before storage. Formatted download size and JavaScript UTF-16 string length + are not used for enforcement. +- Maximum age: 15 minutes from `stored_at_ms`. Non-finite, negative, malformed, + more than 60 seconds in the future, or older values are rejected. A backward + wall-clock jump that places the timestamp beyond the tolerated future skew + also invalidates the entry. Expiry is exposure reduction, not a security + guarantee. +- One supported report for the current browsing context; a new explicit + snapshot replaces the old report. Browser opener cloning and session restore + may copy or retain it. +- The runtime validator accepts only the exact outer and trace-auction v1 + schemas, rejects unknown fields, applies the limits below, and checks compact + UTF-8 size before rendering. +- Invalid, oversized, expired, unsupported, or hostile reports are removed when + possible and otherwise ignored. Rendering uses DOM properties and + `textContent`, never report-derived HTML. +- After confirmation, `Clear report and end tracing` always attempts local + deletion, the validated end POST, and state verification as independent + retry-safe steps. Offline or server failure cannot prevent local deletion. + The UI reports server-observed cookie state and local-report state separately. + A distinct `Delete local report` action remains available whenever a report + is displayed, including after an earlier local-deletion failure. These are product limits, not assumptions about browser quota. A storage write failure is handled even when the report is below the application limit. +Runtime limits are part of the v1 contract: + +| Value | Limit | +| --------------------------------------------- | ------------------------------------------------------------ | +| Container nesting | 8 levels | +| Slots | 64 | +| Request cycles | 10 per slot before total-size truncation | +| Callback issues | 128 | +| Attribution issues | 128 | +| Requested slot sizes | 16 per cycle | +| Ad Manager yield-group or company IDs | 8 of each per cycle | +| Creative-failure enums | 16 per cycle | +| Origin | 255 UTF-8 bytes | +| GPT pathname in trace projection | Exact literal `/[redacted]` | +| Slot element ID and ad-unit path | 512 UTF-8 bytes each | +| Trusted Server auction ID and callback reason | 256 UTF-8 bytes each | +| Any other string | 128 UTF-8 bytes | +| Enum | Exact documented value only | +| Identifier, sequence, or counter | Finite safe integer from 0 through `Number.MAX_SAFE_INTEGER` | +| Browser-relative timestamp or duration | Finite number from 0 through `Number.MAX_SAFE_INTEGER` | +| Visibility percentage | Finite number from 0 through 100 | +| Slot dimension | Finite integer from 1 through 100,000 | + +Every accepted string must be valid Unicode and must not contain C0/C1 control +characters or bidirectional override/isolate controls. This applies to browser +source fields as well as platform fields and precedes rendering or export. + +The snapshot builder creates a new field-by-field projection and rejects an +invalid source value rather than stringifying it. It retains only the first +documented number of requested sizes, yield-group IDs, company IDs, and creative +failure enums, recording discarded entries in `omitted_nested_values`; strings +are never silently shortened. It then measures the complete compact UTF-8 +storage wrapper. If it exceeds 512 KiB, it removes the globally oldest request +cycles first while retaining the newest cycle for each slot, then the oldest +callback issues, then the oldest attribution issues, and finally the oldest +remaining request cycles until the report fits. It records every removal in +`truncation`. +A report that still cannot fit after this bounded procedure fails snapshot +creation. The implementation must include a worst-case fixture proving the +result is bounded. + +All omission counters use checked addition. If any source collection would +make a counter exceed `u16::MAX`, projection rejects the source instead of +wrapping or saturating the count. + +For depth accounting, the `TraceReportV1` object—not its storage wrapper—is +level 1; entering either an object or an array increments the level by one; +primitives do not. No accepted report value may enter a ninth container level. +The storage wrapper is validated separately as the exact two-field object +`{ stored_at_ms, report }`. + +For deterministic ordering, a request cycle with no `requestedAtMs` sorts +before a cycle with a timestamp; otherwise cycles sort by `requestedAtMs`, then +`runtimeSlotNumber`, then `requestNumber`. Callback and attribution issues sort +by `timestampMs`, then their original array index. The builder preserves the +relative order of all retained records. + ## 10. Auction and rendering evidence The report uses TS Console's evidence model. It must preserve the distinction @@ -438,19 +724,21 @@ bids. Exact `provider X was asked for slot Y` and exact no-bid causality require a future provider-impression disposition model. That instrumentation is not silently assumed by this design. -Timing fields introduced by #1074/#1076 are consumed only after they merge and -are propagated through the live diagnostics contract. The report never queries -Tinybird, and it does not combine browser `performance.now()` values with -server-relative timing as though they were one clock. +Timing fields introduced by #1074/#1076 are outside the v1 compatibility +matrix. They may be consumed in a later version only after they merge and are +propagated through the public live-diagnostics contract. The report never +queries Tinybird, and it does not combine browser `performance.now()` values +with server-relative timing as though they were one clock. -Bidder and winning price are included only if #1081 approves them in the public -TS Console export contract. #1050 does not independently weaken the existing -privacy policy. +Bidder and winning price are not added by version one. A later version may +consume them only if #1081 approves them in the public TS Console export +contract. #1050 does not independently weaken the existing privacy policy. ## 11. Network scope -The report is inspired by Fastly Debug, not a clone. Version one uses facts -already present or reasonably addable to the platform request abstraction. +The report is inspired by Fastly Debug, not a clone. Version one exposes only +facts with a defined source and privacy boundary. Every field is optional; an +adapter must omit a value it cannot obtain directly and safely. Supported categories: @@ -458,10 +746,28 @@ Supported categories: - Country, region, and ASN when available. - HTTP version. - TLS protocol and cipher. -- Optional JA4 and H2 fingerprints. - Edge hostname, region, and POP. - Capture time. +Initial provenance and adapter support are: + +| Public field | Source | Fastly | Axum | Cloudflare | Spin | +| ------------------------------ | ---------------------------------------------------------------------------------------------- | --------------------------- | ---------------------- | ------------------------------------ | ----------- | +| `masked_client_ip` | `RuntimeServices.client_info.client_ip`, after trusted-client-IP resolution, then core masking | expected | expected | expected | expected | +| `country`, `region` | `RuntimeServices.geo.lookup(client_info.client_ip)` projected to `GeoInfo.country/region` | expected | unavailable by default | country expected, region unavailable | unavailable | +| `asn` | `GeoInfo.asn` | unavailable until populated | unavailable | unavailable until populated | unavailable | +| `http_version` | new bounded adapter mapping from inbound protocol metadata | optional | expected | optional | optional | +| `tls_protocol`, `tls_cipher` | `ClientInfo.tls_protocol/tls_cipher` | expected | unavailable | unavailable | unavailable | +| `edge_hostname`, `edge_region` | `ClientInfo.server_hostname/server_region` | expected | unavailable | unavailable | unavailable | +| `edge_pop` | new bounded adapter mapping from documented runtime metadata | optional | unavailable | optional | unavailable | + +`expected` means the implementation plan must map and test an existing source; +`optional` means the adapter includes it only when its supported SDK exposes a +stable value; `unavailable` means v1 intentionally omits it. In particular, +ASN is currently not populated by the Fastly or Cloudflare geo adapters and +must not be claimed until a concrete source is implemented. New HTTP-version or +POP mappings must be confirmed against the pinned adapter SDK before addition. + Explicitly excluded: - DNS resolver address and resolver ASN. @@ -470,6 +776,7 @@ Explicitly excluded: - DDoS/internal Fastly classifications. - Arbitrary request headers. - Full client IP in HTML or export. +- JA4, H2, or other probabilistic client fingerprints. Unsupported optional fields are omitted rather than populated with fabricated fallbacks. @@ -482,6 +789,12 @@ The report serializer constructs a new public model field by field. It never serializes request structs, cookie parsers, auction requests, telemetry rows, or browser objects wholesale. +The deserializer is an equally strict boundary. It validates the complete +outer and nested schema at runtime before any display, export, copy, or share +operation. Unknown properties, overlong strings, non-finite numbers, excessive +arrays, excessive depth, unsupported versions, and invalid timestamps reject +the report. Validation errors expose only bounded categories. + Forbidden data includes: - Raw `Cookie` and `Set-Cookie` headers. @@ -496,25 +809,52 @@ Forbidden data includes: ### 12.2 Same-origin script visibility Publisher and third-party scripts running on the publisher origin can access -`sessionStorage`. Therefore the stored model must be safe even if read by any -same-origin script. A random storage key, closed shadow root, or public endpoint -does not change this requirement. +`sessionStorage`. They can also replace it, opener-created tabs may receive a +copy, browser session restore may preserve it, and a same-origin service worker +may intercept navigation. Therefore the stored model must be safe even if read +or forged by any same-origin code. A random storage key, closed shadow root, or +public endpoint does not change this requirement. + +Every report view and exported artifact is labeled `Browser-observed, +unverified diagnostic data`. Support documentation says that it helps +troubleshoot rendering but is not proof of a server event, user identity, or +security incident. ### 12.3 Response hardening -Both the endpoint and every active diagnostic publisher response are terminally -`private, no-store`. The endpoint also sends: +The HTML shell, enable/end responses, and every active diagnostic publisher +response are terminally `private, no-store`. The fixed versioned JS/CSS assets +are the sole exception and may be publicly cached because they contain no +request or report data. HTML and JSON endpoint responses also send: -- `Content-Type: text/html; charset=utf-8` +- Path-appropriate `Content-Type`: `text/html; charset=utf-8` for the shell and + `application/json; charset=utf-8` for enable, end, and state results. - `X-Content-Type-Options: nosniff` - `Referrer-Policy: no-referrer` -- `Content-Security-Policy` restricting content to the endpoint's own static - assets and prohibiting framing -- A restrictive `Permissions-Policy` +- The Content Security Policy specified below. +- `Permissions-Policy: camera=(), microphone=(), geolocation=(), payment=(), +usb=()` + +```text +default-src 'none'; script-src 'self'; style-src 'self'; base-uri 'none'; +object-src 'none'; frame-ancestors 'none'; form-action 'none'; connect-src 'self' +``` -The endpoint makes no third-party requests. Dynamic JSON embedded in HTML uses -the repository's script-safe serializer and is never concatenated into -executable JavaScript. +The endpoint makes no third-party requests. Its script and stylesheet are fixed +same-origin static assets. Setup-request values are server-rendered as escaped +text nodes, and the viewer obtains the report only from browser storage. Active +publisher-page context continues to use the repository's script-safe serializer +and is never concatenated into executable JavaScript. If inline executable +assets become necessary, they require a per-response nonce or fixed build-time +hash and a corresponding CSP change. Validated report strings enter the +document through `textContent` or equivalent DOM properties, never `innerHTML`. + +The JS asset uses `application/javascript; charset=utf-8`; the CSS asset uses +`text/css; charset=utf-8`. Both send `X-Content-Type-Options: nosniff`, +`Cache-Control: public, max-age=31536000, immutable`, and a strong ETag derived +from their build bytes. They accept no dynamic input. `script-src 'self'` is an +origin-level CSP permission, not a path restriction; same-origin script +interference remains inside the stated trust limitation. ### 12.4 Shared templates and ESI @@ -527,20 +867,34 @@ publicly cacheable. - Disabled route: local privacy-safe `404`. - Unsupported method: local `405`; never publisher fallback. +- Rejected activation/end POST: local `403` with no state mutation. - Optional platform fact unavailable: omit the field and continue. -- Cookie parser failure: report `present_invalid` without the value. +- Bounded cookie inspection failure: report the contract-defined invalid or + unavailable state without a value or parser message. - Diagnostics context serialization failure: omit the context, log a bounded server error, and preserve publisher delivery. - TS Console capture failure: fail open for advertising and show incomplete coverage in diagnostics. -- Storage unavailable, quota exceeded, or serialization oversized: remain on - the publisher page, announce the error, and offer direct download. +- Storage unavailable or quota exceeded after a valid bounded report exists: + remain on the publisher page, announce the error, and offer direct download + of that same combined `TraceReportV1`. +- Invalid projection or a report that remains oversized after deterministic + truncation: remain on the publisher page, show a bounded capture-failure + category, and do not claim that a combined trace report exists. - Missing snapshot on endpoint: show setup state, not an empty successful report. - Expired, malformed, or unknown report schema: clear it and explain that the user must reproduce again. - Clipboard or Web Share unavailable: keep JSON download available. - Export failure: retain the on-screen report and show an accessible error. +- End POST failure: report that tracing may remain active and offer an + idempotent server retry; do not undo or block the independent local-deletion + attempt. +- State verification failure or mismatch: use `unconfirmed` wording and offer + an idempotent server retry independently of local report state. +- Local deletion failure: report separately that saved browser data could not be + removed and retain the always-available deletion retry, regardless of the + server end result. Diagnostic failures must never suppress, delay, add, remove, or reorder GPT requests, auctions, targeting, or creative rendering. @@ -551,55 +905,96 @@ requests, auctions, targeting, or creative rendering. - Configuration defaults off and rejects trace-page enablement without GPT diagnostics. -- Exact route, query, method, encoded-path, and fallback behavior. -- Session cookie set/clear attributes and duplicate-directive fail-closed - behavior. +- Exact reserved-route classification, canonical-path, query, method, encoded + path, and fallback behavior. +- Exact versioned asset routes are local and contain no dynamic data; lookalike + asset paths never reach the publisher origin. +- Same-origin POST validation, cross-site/missing signal rejection, cookie + set/clear attributes, fixed 30-minute endpoint activation without request + refresh, and idempotent enable/end behavior. +- Enable/end success requires a separate state request to observe the resulting + cookie; failed and mismatched verification never displays confirmed state. +- Empty-body enforcement rejects positive/invalid lengths, transfer encoding, + the first unexpected body byte, and the two-second deadline without an + unbounded read. - Endpoint skips EC generation/finalization, EID ingestion, auction, telemetry, - and origin fetch. -- Cookie-health parser covers absent, valid, malformed, duplicate, non-UTF-8, - and oversized inputs without retaining values. -- Request-context serializer masks IPv4/IPv6 and omits query, raw headers, IDs, - and unsupported fields. + configured filters, ordinary event context, and origin fetch. +- Cookie-health scanner covers multiple header fields; zero, one, and duplicate + occurrences; mixed valid/invalid duplicates; non-UTF-8; malformed pairs; and + per-value and total-header limits without retaining values. +- Request-context serializer masks IPv4/IPv6; enforces every string bound; and + omits page paths, fingerprints, query, raw headers, IDs, and unsupported + fields. - Active responses remain terminally private/no-store under hostile late header overrides. - Dynamic HTML/JSON values cannot close elements or create executable script. ### 14.2 Adapter parity tests -- Fastly route registration and optional field mapping. +- Fastly early-route ordering and optional field mapping from documented + sources. - Axum, Cloudflare, and Spin return the common route/schema with unavailable fields omitted. -- Named route failures never fall through to publisher origin. -- HEAD and unsupported methods behave identically across adapters. -- Fastly fingerprint fields respect the existing fingerprint-debug gate. +- Trace-route failures never fall through to publisher origin. +- GET, HEAD, state-changing POST, and unsupported methods obey the same + lifecycle contract across adapters. +- Every adapter omits JA4/H2 and rejects control characters or overlong platform + strings. ### 14.3 JavaScript unit tests - Explicit snapshot only; no continuous `sessionStorage` writes. -- Size limit, schema validation, expiry, replacement, clearing, and storage +- Compact UTF-8 size measurement; exact outer/nested schema validation; unknown + fields; per-string/array/numeric/depth caps; hostile mutation; expiry; + future-clock skew; wall-clock rollback; replacement; clearing; and storage exceptions. +- Omission counters use checked arithmetic and reject overflow. +- Trace projection replaces the nested GPT pathname with `/[redacted]`, rejects + any other stored value, emits `TraceGptDiagnosticsV1`, applies deterministic + ordering/truncation, and records exact omission counts in a worst-case 512 KiB + fixture. - Same-tab navigation occurs only after a successful write. - Viewer handles absent optional network facts and every cookie-health state. - Forbidden fields never enter storage or export fixtures. - Download filename and MIME type are deterministic. -- Copy and Web Share success, rejection, absence, and fallback behavior. +- Formatted-JSON copy and JSON-file Web Share success, rejection, absence, and + download/copy fallback behavior. - 320-pixel layout, keyboard navigation, focus handling, and accessible status announcements. ### 14.4 Browser integration tests -- First endpoint visit sets the session and shows setup state. +- First endpoint GET is read-only and shows setup state; a user-initiated, + same-origin enable POST sets the session. +- Successful in-page activation adds no history entry, so Back can return to + the article when it was the prior same-tab page. +- Cross-site top-level GET, form POST, and fetch attempts cannot enable or end + tracing. - A real fixture reload activates diagnostics and captures multiple slots. - `View trace results` navigates in the same tab and renders the captured request context and slot evidence. - Empty, filled, ambiguous, no-candidate, and unattributed slot states remain distinct. - Reloading the trace page retains an unexpired same-tab report. -- A new tab cannot access the original tab's report. -- Disabling clears both cookie and report. +- Opener-cloned tabs and browser session restore never bypass validation or + application expiry; the UI does not promise tab isolation. +- Ending tracing covers successful clearing, offline POST failure, idempotent + retry, verification mismatch, successful local deletion while offline, and + local-storage deletion failure without false success messaging. - Back-forward-cache restoration is not described as a fresh traced request; the setup page tells the user to reload. - Export JSON matches the displayed versioned model. +- A storage-failure direct export matches the combined displayed model rather + than the GPT-only export. +- Hostname changes between apex, `www`, or another subdomain show the recovery + guidance rather than claiming the session followed the user. +- A fixture service worker interception is recognized as a same-origin trust + limitation, and the server endpoint remains correct when the request reaches + it. +- The delivered CSP blocks inline injection, framing, third-party connections, + and report-derived executable HTML. +- Immutable asset fixtures prove published v1 bytes never change; changed bytes + require a new URL referenced by the shell. - Inactive publisher traffic has no trace assets, storage access, listeners, or cache-policy change. @@ -620,58 +1015,84 @@ fixture: - Enable first in a controlled staging publisher configuration. - Validate response cache headers and CDN behavior before production use. - Validate with redacted fixtures before real publisher traffic. -- Log only route outcome, schema version, report-present boolean, and bounded - error category. Never log report contents or cookie/network values. +- Log only server-observable route outcome, served shell/schema version, and + bounded error category. The server cannot know whether a browser-local report + exists and must not add an upload or beacon merely to learn that fact. Never + log report contents or cookie/network values. - Roll back by disabling `trace_page_enabled`; existing `?ts_console=1` diagnostics remain independently configurable. ## 16. Acceptance criteria -1. With the feature disabled, exact trace routes return local `404` and ordinary - traffic is unchanged. -2. A mobile user can enable tracing by opening only `/_ts/trace`; no target URL, - credentials, or trace ID is required. +1. With the feature disabled, trace-route origin requests return local `404` + and ordinary traffic is unchanged. Previously cached inert versioned assets + may remain until cache eviction, but cannot activate tracing or load a shell. +2. A mobile user can enable tracing by opening only `/_ts/trace` and selecting + one prominent action; no target URL, credentials, or trace ID is required, + and a cross-site GET cannot activate tracing. 3. The setup page accurately explains that the problem must be reproduced after activation. 4. A subsequent real publisher-page reload captures redacted request context and existing TS Console evidence without altering ad behavior. -5. `View trace results` transfers one bounded snapshot in the same tab and opens - the report page without server-side storage. +5. `View trace results` transfers one bounded, runtime-validated snapshot in + the supported same-tab journey and opens the report page without server-side + storage or claims of browser-storage isolation. 6. The report separates network, cookie health, auction/render evidence, and coverage/unknowns. 7. JSON export contains the same versioned allowlisted information shown on the page. -8. No raw cookies, user IDs, full IPs, consent strings, query strings, internal - auction IDs, targeting, or creative payloads appear in HTML, browser storage, - logs, or export. +8. No raw cookies, user IDs, full IPs, consent strings, exact page paths, query + strings, fingerprints, internal auction IDs, targeting, or creative payloads + appear in trace HTML, browser storage, logs, or export. 9. Trace HTML and active publisher pages remain terminally private/no-store. 10. Missing platform fields, incomplete auction correlation, storage failure, and unavailable share APIs degrade honestly without affecting advertising. 11. The full report is usable at 320 CSS pixels and with keyboard/screen-reader navigation. -12. Auction fields owned by #1081 are consumed through its versioned public - contract rather than duplicated in #1050. +12. Version one accepts exactly `TraceReportV1` with + `TraceGptDiagnosticsV1`, sourced only from `GptDiagnosticsExportV1`; #1081 + and #1074/#1076 are optional additive follow-ups rather than release gates. +13. Every rendered and exported report is identified as browser-observed and + unverified, and hostile storage content cannot create executable HTML or + unbounded DOM output. +14. Enable/end operations expose partial failure honestly and are safe to + retry; the UI does not claim server-observed cookie state without the + follow-up state request or claim that local data was cleared when deletion + fails. ## 17. Implementation sequencing -This design is one product flow but should be implemented in dependency order: +This design is one product flow, but its implementation is split into three +independently reviewable plans and preferably three PRs: + +1. **Reserved route and privacy foundation:** configuration, shared early-route + classification, same-origin enable/end lifecycle, bounded cookie-health + inspection, base request-context schema, projection of already populated + `ClientInfo`/`GeoInfo` fields, response hardening, and adapter parity. Do not + add speculative new platform fields in this change. +2. **Browser handoff and viewer:** integrate current + `GptDiagnosticsExportV1`, project `TraceGptDiagnosticsV1`, construct and + strictly validate `TraceReportV1`, implement the same-tab workflow, combined + direct/storage exports, mobile viewer, copy/share, expiry, clearing, and + browser/accessibility tests. +3. **Optional network enrichment and future schemas:** add HTTP-version, POP, + ASN, or other fields only from SDK-verified platform sources with explicit + bounds. Adopt #1081 or #1074/#1076 later through a separately reviewed + versioned compatibility change. + +Each plan must include its own adapter, privacy, cache, and failure tests. The +implementation must not claim completion of #1081 or the timing work as part of +#1050. Version one ships with current observed auction/render evidence and +labels unavailable fields honestly. -1. Core request-context schema, cookie-health classification, configuration, - and endpoint shell. -2. Adapter route parity and Fastly optional network enrichment. -3. TS Console request-context envelope and explicit same-tab snapshot handoff. -4. Mobile viewer, export/copy/share, expiry, and clearing. -5. Integration with the current TS Console schema. -6. Additive adoption of #1081 and #1074/#1076 fields after their contracts - merge. -7. Browser, privacy, cache, and real-device acceptance. +## 18. Rejected alternatives -The implementation plan must not claim completion of #1081 or the open timing -PRs as part of #1050. If those dependencies are unavailable, the report ships -only with current observed auction/render evidence and labels unavailable fields -honestly. +### Automatic activation on `GET /_ts/trace` -## 18. Rejected alternatives +Rejected because a cross-site top-level navigation can trigger a public GET and +`SameSite=Lax` does not make that activation intentional. A same-origin fetch +POST after one explicit button press preserves the simple mobile journey and +the useful Back history entry without requiring server-side session storage. ### `/_ts/admin/trace?target=/article` @@ -715,9 +1136,16 @@ length, history, logging, referrer, and accidental-sharing risks. ## 19. Known limitations - The user must reproduce the problem after enabling tracing. -- Same-tab storage prevents cross-device and cross-tab sharing; JSON export is - the handoff artifact. -- Publisher-origin scripts can read the stored public-safe report. +- Same-tab navigation is the supported workflow, but opener-created tabs and + browser session restore may copy or retain session storage. JSON export is + the intentional support handoff artifact. +- Publisher-origin scripts and service workers can read or forge the stored + public-safe report. A same-origin service worker can also intercept or fake + the shell, enable/end/state requests, assets, and report navigation. The + experience is diagnostic evidence, not an authenticity boundary. +- The activation cookie is host-only and session storage is origin-scoped, so + the workflow does not follow the user across apex, `www`, or other + subdomains. - Browser privacy settings may disable storage, clipboard, download, or share capabilities. - Current server/browser correlation does not cover every no-bid, skipped, From 534a691e2e64852c28728be80fb90e44167c22ce Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 1 Sep 2026 16:04:45 +0530 Subject: [PATCH 3/8] Add server auction evidence to mobile trace design --- ...-mobile-ad-render-trace-endpoint-design.md | 726 +++++++++++++++--- 1 file changed, 616 insertions(+), 110 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md index 6ed9f8c48..092466a6d 100644 --- a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -22,12 +22,12 @@ intentionally enables or ends tracing with a same-origin POST action. The endpoint is both a setup page and a report viewer. On the first visit it offers a large `Enable tracing` action and explains how to reproduce the problem. The user then returns to the real publisher page and reloads it. -Trusted Server supplies redacted request context, while the existing TS Console -records GPT, auction, and render evidence. A `View trace results` action creates -one bounded, allowlisted snapshot in same-tab `sessionStorage` and navigates to -`/_ts/trace`. The endpoint reads that untrusted snapshot, validates it, presents -a mobile-first HTML report, and offers JSON export, copy, and progressive Web -Share actions. +Trusted Server supplies redacted request context and a minimal live summary of +each server auction, while the existing TS Console records GPT and render +evidence. A `View trace results` action creates one bounded, allowlisted +snapshot in same-tab `sessionStorage` and navigates to `/_ts/trace`. The +endpoint reads that untrusted snapshot, validates it, presents a mobile-first +HTML report, and offers JSON export, copy, and progressive Web Share actions. The design introduces no report database, server-side trace store, report ID, target URL parameter, telemetry query, or publisher-origin change. It cannot @@ -57,7 +57,10 @@ experience: - TS Console owns observation of the real publisher page. The browser-local handoff joins them without introducing a backend report -service. +service. The report keeps three evidence layers separate: what the server +auction observed, what GPT observed in the page, and what the creative bridge +observed during rendering. Correlation is displayed only when an ephemeral +diagnostic token connects those layers. ## 3. Goals @@ -68,6 +71,9 @@ service. - Display a Fastly-inspired network summary for the traced publisher request. - Report health for an explicit allowlist of Trusted Server cookies without exposing their values. +- Report each observed server auction as initial-navigation SSAT, SPA page-bids, + or the Trusted Server `/auction` API, including bounded auction-local timing, + provider-call outcomes, and per-slot candidate outcomes. - Present the versioned, allowlisted TS Console evidence for every retained GPT slot and request cycle that fits the public report bounds, with explicit omission counts when deterministic size truncation is required. @@ -97,10 +103,15 @@ service. cache URLs, or stack traces. - Reimplementing TS Console auction and creative observability requested by #1081. +- Naming bidders, exposing winning price, assigning final creative numbers, or + asserting which demand path ultimately won when the available evidence does + not prove it; those remain #1081 concerns. - Querying Tinybird to build an interactive report. - Adding exact provider-by-slot no-bid explanations before the auction model can observe those dispositions. -- Direct `POST /auction` browser diagnostics in the first release. +- Introspection into third-party client-side auction internals. Version one can + report that a publisher/Prebid refresh was observed, but cannot identify its + participants or winner unless Trusted Server itself handled that auction. ## 5. Decisions @@ -137,6 +148,13 @@ lifetime bounds accidental private/no-store operation if a user forgets to end tracing; the existing technical query flow keeps its existing session-cookie semantics. +The new trace capture is active only when both `trace_page_enabled = true` and +the request carries exactly one valid diagnostics cookie. The shared cookie by +itself continues to activate the existing TS Console but does not authorize +trace tokens, server-auction projection, trace response extensions, or +correlation sidecars on a deployment whose trace page is disabled. This makes +the configuration flag the disclosure and rollback boundary. + ### 5.3 Browser-local, explicit handoff TS Console remains memory-only during observation. It writes a report to @@ -177,15 +195,25 @@ inconsistent behavior. ### 5.5 Separate issue ownership #1050 defines the report shell, request context, mobile flow, browser-local -handoff, and export. #1081 remains the owner of creative numbering, auction -classification, bidder/price policy, terminology, and normalized auction/render -timing. - -Version one consumes `GptDiagnosticsExportV1` through TS Console's public export -contract and projects it into a distinct redacted `TraceGptDiagnosticsV1`. It -does not read TS Console internals or create an alternate slot correlation -engine. #1081 and #1074/#1076 are additive follow-up work and are not release -gates for this version. +handoff, export, and the minimum server-auction facts needed to answer whether +SSAT or the Trusted Server auction API ran. Core produces a new redacted +`TraceAuctionEvidenceV1`; TS Console consumes it as immutable evidence rather +than reconstructing server behavior from GPT callbacks. + +#1081 remains the owner of bidder/price disclosure policy, creative numbering, +final user-facing terminology, and any richer cross-demand winner +classification. #1074/#1076 remain the owners of request-phase and +request-relative auction milestones. Version one may ship without those +follow-ups because it exposes already-available auction-local total and provider +durations and labels each clock explicitly. + +Version one also consumes `GptDiagnosticsExportV1` through TS Console's public +export contract and projects it into a distinct redacted +`TraceGptDiagnosticsV1`. It does not read TS Console internals or create a +second GPT attribution engine. The existing recorder emits an exact-token +`TraceSlotCorrelationV1` sidecar when it binds an opportunity to a request +cycle. The server-auction, correlation, and GPT projections are separate sibling +contracts in `TraceReportV1`; none is treated as a substitute for another. ## 6. User experience @@ -234,18 +262,43 @@ When a valid snapshot exists, `/_ts/trace` renders: 1. Report summary and capture time. 2. Network and request section for the traced publisher document. 3. Trusted Server cookie-health section. -4. Auction and rendering section grouped by numbered slot. -5. Coverage and ambiguity section. -6. Export actions. -7. `Clear report and end tracing` action. +4. Server-auction section grouped by auction and numbered slot. +5. GPT delivery and creative-rendering section grouped by numbered slot. +6. Coverage and ambiguity section. +7. Export actions. +8. `Clear report and end tracing` action. The setup request's facts are not merged into or substituted for missing traced page facts. Missing fields display `Unavailable`; missing evidence displays `Not observed` or `Unknown`, following TS Console terminology. -The report begins with `Browser-observed, unverified diagnostic data`. It does -not claim that the snapshot is authentic or suitable as forensic or security -evidence. +The viewer presents an evidence chain rather than one overloaded status: + +```text +Server auction -> GPT request/response -> creative render/load/viewability +``` + +For each step it shows the source, observed outcome, and correlation state. +`Client-side refresh observed` describes browser intent only; it must never be +rendered as `client-side auction won`. Likewise, a filled GPT slot does not +prove that the Trusted Server candidate rendered. + +Internal enums are exported for machines, but the page uses plain labels: + +| Evidence enum | Mobile label | +| ------------------------------------------ | ------------------------------------------------- | +| `initial_navigation_ssat` | `Initial-page server auction (SSAT)` | +| `spa_page_bids` | `Trusted Server page-refresh auction` | +| `auction_api` | `Trusted Server auction API` | +| GPT `prebid_refresh`/`publisher_refresh` | `Browser refresh observed; winner not determined` | +| GPT `competing`/`unattributed` | `Multiple or unknown delivery paths` | +| matched non-empty creative-bridge evidence | `Trusted Server creative rendered` | + +The report begins with `Browser-carried, unverified diagnostic data`. Server +auction entries are labeled `Produced by Trusted Server; copied through an +untrusted browser snapshot`, while GPT and creative entries are labeled +`Browser observed`. The viewer does not claim that the stored snapshot is +authentic or suitable as forensic or security evidence. `Copy` copies formatted JSON. `Share` supplies the same JSON file to the native Web Share sheet only after an explicit tap and tells the user that the selected @@ -286,7 +339,8 @@ Real publisher document reload |-- adapter supplies optional network facts |-- core computes allowlisted cookie health |-- core injects redacted TraceRequestContextV1 - |-- existing TS Console observes GPT and TS delivery + |-- core observes live server auctions and emits TraceAuctionEvidenceV1 + |-- existing TS Console observes GPT and creative delivery v User selects "View trace results" | @@ -307,6 +361,10 @@ Report GET /_ts/trace - Provide a shared exact-path reserved-route classifier that runs before event context, filters, auctions, named routes, or publisher fallback. - Define the platform-neutral request-context and report-envelope schemas. +- Build `TraceAuctionEvidenceV1` directly from the live auction observation and + orchestration result; never query telemetry or serialize telemetry rows. +- Mint and thread public diagnostic auction/slot tokens independently of + internal request IDs, including zero-bid and failed auctions. - Build cookie-health facts through read-only parsing. - Convert `ClientInfo` and available geo data into the public network allowlist. - Inject request context only into an active private diagnostics document. @@ -327,9 +385,14 @@ Report GET /_ts/trace ### 7.3 JavaScript responsibilities - Accept the immutable redacted request context at initialization. +- Accept immutable server-auction evidence delivered through the supported + initial-navigation, page-bids, and `/auction` transports. +- Emit the bounded `TraceSlotCorrelationV1` sidecar at the existing GPT + recorder's opportunity-to-cycle binding point. - Preserve the existing bounded TS Console observation store. -- Build and validate `TraceReportV1` with a redacted - `TraceGptDiagnosticsV1` projection on explicit user action. +- Build and validate `TraceReportV1` with redacted + `TraceAuctionEvidenceV1` and `TraceGptDiagnosticsV1` projections on explicit + user action. - Store only one supported report for the same-tab workflow in `sessionStorage`, while treating its contents as untrusted. - Render the report shell from the validated model. @@ -356,8 +419,9 @@ Rules: the allowlisted presence/validity of four HttpOnly Trusted Server cookies visible to same-origin JavaScript whenever the feature is enabled. It also states that masked IP prefixes and coarse geo remain potentially personal or - pseudonymous network data. Enabling the option is the deployment's explicit - acceptance of those bounded disclosures. + pseudonymous network data, and that opaque server-auction outcomes become + visible to same-origin JavaScript. Enabling the option is the deployment's + explicit acceptance of those bounded disclosures. - `GET /_ts/trace` returns the setup/report shell without changing cookies or browser storage at the HTTP layer. After load, the explicitly included viewer script may remove a rejected or expired local entry. Unrelated query @@ -571,16 +635,28 @@ TraceReportV1 schema_version: 1 captured_at: RFC 3339 UTC timestamp request_context: TraceRequestContextV1 + server_auctions: TraceAuctionEvidenceV1[] + slot_correlations: TraceSlotCorrelationV1[] gpt_diagnostics: TraceGptDiagnosticsV1 + auction_coverage: + capture_status: complete | partial | unavailable | not_observed + issues: + evidence_projection_failed | evidence_transport_failed + | evidence_validation_failed | record_evicted + | correlation_unavailable | external_client_side_unobservable truncation: + omitted_server_auctions: u16 + omitted_slot_correlations: u16 omitted_request_cycles: u16 omitted_callback_issues: u16 omitted_attribution_issues: u16 omitted_nested_values: u16 ``` -`TraceGptDiagnosticsV1` is a trace-owned projection sourced only from -`GptDiagnosticsExportV1`. It contains: +`TraceAuctionEvidenceV1` is a server-produced public model defined in section +9.4. `TraceSlotCorrelationV1` is the browser-produced exact-token sidecar +defined in section 9.4.1. `TraceGptDiagnosticsV1` is a separate trace-owned +projection sourced only from `GptDiagnosticsExportV1`. It contains: - `schema_version: 1` and `source_schema_version: 1`; - the source `capturedAt` value; @@ -593,11 +669,24 @@ It is deliberately not named or represented as `GptDiagnosticsExportV1`, because the fixed pathname and trace-level bounds change the source field semantics. TS Console continues to own the source schema; the trace envelope owns its public projection and transport. The initial compatibility matrix is -exactly `TraceReportV1` plus `TraceGptDiagnosticsV1`, sourced from -`GptDiagnosticsExportV1`. The viewer rejects every unknown outer, trace-auction, -or source version with an actionable message rather than guessing. A future TS -Console successor requires an additive source compatibility change and, if the -public projection changes, a new trace-envelope version. +exactly `TraceReportV1`, `TraceAuctionEvidenceV1`, `TraceSlotCorrelationV1`, and +`TraceGptDiagnosticsV1`, with the GPT projection sourced from +`GptDiagnosticsExportV1`. The viewer rejects every unknown outer, +server-auction, correlation, GPT-projection, or GPT-source version with an +actionable message rather than guessing. A future TS Console successor requires +an additive source compatibility change and, if the public projection changes, +a new trace-envelope version. + +`auction_coverage.capture_status` describes only what reached the browser +collector: `not_observed` means no valid server-auction record arrived, not that +no server auction ran. `partial` requires at least one retained record plus a +projection, transport, validation, or eviction issue; `unavailable` requires no +retained records plus a known projection, transport, or validation issue; and +`complete` requires at least one retained record without those capture issues. +`correlation_unavailable` and `external_client_side_unobservable` describe +interpretation limits and do not change an otherwise complete capture status. +The issue array is deduplicated, sorted in enum order, bounded to 16 values, and +contains no error text. Origin validation requires a parseable HTTP(S) origin whose canonical serialization exactly equals `window.location.origin`; credentials, paths, @@ -606,7 +695,268 @@ strict RFC 3339 UTC strings and must be within 60 seconds of `stored_at_ms`. `TraceRequestContextV1.captured_at` requires strict RFC 3339 UTC but may be older because it represents the publisher document request rather than snapshot time. -### 9.4 Storage limits and expiry +### 9.4 Server-auction evidence + +Core creates one `TraceAuctionEvidenceV1` at the live auction boundary. It is +not derived from the browser, reconstructed from winning-bid targeting, or +loaded from auction telemetry: + +```text +TraceAuctionEvidenceV1 + schema_version: 1 + diagnostic_auction_id: string + source: + initial_navigation_ssat | spa_page_bids | auction_api + terminal_status: + completed | execution_failed | dispatch_failed | abandoned | skipped + terminal_reason?: + policy_skipped | no_eligible_slots | no_provider_launched + | provider_execution_failed | collection_failed | unknown + total_time_ms?: u32 + provider_calls: + - provider_number: u16 + role: bidder | mediator | unknown + status: success | no_bid | error | pending | abandoned | unknown + response_time_ms?: u32 + returned_bid_count: u16 + slots: + - slot_number: u16 + slot_ref: string + requested_sizes: [u32, u32][] + returned_bid_count: u16 + candidate: + selected | no_candidate | selected_unrenderable | unknown + selected_creative_size?: [u32, u32] + truncation: + omitted_provider_calls: u16 + omitted_slots: u16 + omitted_nested_values: u16 + coverage: + provider_to_slot_no_bid: unavailable +``` + +The model has deliberately lower cardinality and sensitivity than the existing +telemetry and OpenRTB objects: + +- `diagnostic_auction_id` is a fresh opaque `ts-auc-...` correlation token. When + trace capture is active under the two-part gate in section 5.2, it is minted + once when an eligible auction is observed, before dispatch, and is retained + for zero-bid, skipped, dispatch-failed, execution-failed, and abandoned + outcomes. It is never `AuctionRequest.id`, the telemetry UUID, a provider + request ID, or an identifier joinable to user-bearing logs. +- Auction and slot tokens are the fixed prefixes `ts-auc-` and `ts-slot-` + followed by a canonical lowercase hyphenated UUID v4. Validators reject every + other shape; tokens are not silently shortened or normalized. +- `slot_number` is a one-based ordinal over the exact post-conversion + `AuctionRequest.slots` sequence observed by orchestration. It is display-only + and is never used to map a response back to pre-conversion client input. + `slot_ref` is a fresh auction-local opaque token carried with that slot. Core + creates it for initial-navigation and SPA auctions. For a TSJS `/auction` + request, TSJS creates it only after `buildAdRequest` has finished grouping and + deduplicating the final `adUnits` array, attaches it to that exact outgoing + unit as `adUnits[].ext.trusted_server.trace_slot_ref`, and retains the + request-scoped token-to-unit mapping. Core accepts that member only under the + two-part trace gate, validates and echoes the token for accepted converted + slots, and strips it before every provider or mediator request. A missing or + invalid client token causes core to mint a server token with no browser + correlation; it never changes ordinary auction acceptance. TSJS uses + `crypto.randomUUID()` and, if unavailable, omits the client token rather than + using weak randomness. Raw publisher slot IDs, ad-unit paths, and internal + impression IDs are not copied into this model. Neither the ordinal nor token + is a creative number. +- `source` is assigned by the server call site: initial document auction is + `initial_navigation_ssat`, `/_ts/page-bids` is `spa_page_bids`, and + `POST /auction` is `auction_api`. Browser `requestPath` does not determine or + override this value. +- `provider_number` is assigned deterministically in provider dispatch order + and is stable only within one auction. Provider names, bidder/seat names, and + provider metadata are omitted. `returned_bid_count` is a count, not a bid + payload. +- `terminal_reason` is mapped to the allowlisted category at the observation + boundary. Raw error messages, parser errors, URLs, and provider text never + enter the model. +- Slot candidate state is computed from the requested slots, returned bids, + winner selection, and final response-conversion disposition. A winner that + cannot safely enter the bid map/OpenRTB response is + `selected_unrenderable`, not `selected` and not `no_candidate`. +- Provider calls are auction-wide. The model does not claim that a provider was + called, timed out, or returned no bid for a particular slot. Only returned + bids can contribute to a slot's `returned_bid_count`; the fixed coverage + value makes the missing provider-to-slot no-bid relation explicit. +- `total_time_ms` and `response_time_ms` use the server's auction-local monotonic + durations. They are not request-relative milestones and are never + arithmetically combined with browser timestamps. +- Core applies the provider, slot, size, string, and numeric limits before the + model crosses into HTML or JSON. It retains request/dispatch order and records + every discarded nested entry in the auction-local `truncation` object. An + omission-counter overflow rejects that auction evidence rather than wrapping + or saturating. A duration that cannot convert to its optional public integer + type is omitted and counted; a required count or ordinal conversion failure + rejects that auction evidence. Values are never clamped to a plausible value. + +The server evidence and GPT evidence retain separate meanings: + +| Question | Authoritative v1 source | +| --------------------------------------------------- | ------------------------------------------------------------- | +| Did initial-navigation SSAT run? | `server_auctions[].source = initial_navigation_ssat` | +| Did the Trusted Server auction API run? | `server_auctions[].source = auction_api` | +| Was a publisher/Prebid refresh observed? | GPT `requestPath`, labeled as browser intent | +| Did a server auction select a slot candidate? | matching server-auction slot `candidate` | +| Did GPT request, fill, and render the slot? | `TraceGptDiagnosticsV1` request-cycle evidence | +| Did the Trusted Server creative bridge participate? | `TraceGptDiagnosticsV1` creative-delivery evidence | +| Which path ultimately won? | only when existing correlation proves it; otherwise `Unknown` | + +Absence is not converted into a negative assertion. If transport or +correlation failed, the viewer shows `Server auction evidence unavailable` or +`Correlation unknown`, not `SSAT did not run`. + +#### 9.4.1 Live transport and correlation + +Evidence is transported only while `trace_page_enabled` is true and the +diagnostics cookie is valid. Every response carrying it is terminally +`private, no-store`: + +```text +TraceAuctionTransportV1 + schema_version: 1 + evidence?: TraceAuctionEvidenceV1 + unavailable_reason?: evidence_projection_failed +``` + +Exactly one of `evidence` and `unavailable_reason` is present. This small +transport envelope lets core report a safe projection failure without exposing +the raw error. A malformed envelope is rejected as a whole. A network failure +before an envelope arrives is recorded separately by the browser as +`evidence_transport_failed`. + +Initial-navigation and SPA slot definitions carry their token on the exact slot +object TSJS already consumes: + +```text +AuctionSlot.ext.trusted_server.trace_slot_ref: string +``` + +Core adds that optional nested member only under the two-part trace gate. It +assigns the token while constructing the request-scoped slot definitions and +threads the same token into the corresponding `AuctionRequest` observation, so +neither side needs to recover the relationship from an ordinal or raw slot ID. +The ordinary `AuctionSlot.id`, `gam_unit_path`, `div_id`, formats, targeting, +ordering, and bid-map keys remain unchanged. The extension is absent when the +gate is false and is never copied into `TraceAuctionEvidenceV1` except as its +already-allowlisted opaque `slot_ref`. + +TSJS accepts a slot extension only when its canonical token occurs exactly once +in both the delivered slot list and the matching auction evidence. A missing, +malformed, duplicate, or conflicting token prevents only that sidecar join, +adds `evidence_validation_failed` and `correlation_unavailable`, and does not +drop, reorder, or mutate the ordinary slot or bid. TSJS reads no other extension +property. This validation occurs before the slot is handed to the existing GPT +initialization path. + +The three transport call shapes are exact v1 contracts: + +```text +Initial seam: + scheduleInitialAdInit(bids, slots?, traceAuctionTransport?) + +SPA JSON: + { slots, bids, trace_auction?: TraceAuctionTransportV1 } + +/auction request unit: + adUnits[].ext.trusted_server.trace_slot_ref?: string + +/auction OpenRTB response: + ext.trusted_server.trace_auction?: TraceAuctionTransportV1 +``` + +The initial scheduler validates/records the optional third argument before it +runs `adInit`; cached older bundles may ignore the extra argument without +affecting ads, in which case evidence remains `not_observed`. The SPA parser +accepts only the exact optional top-level member and preserves its existing +`slots` and `bids` behavior. These trace members never become required for a +successful advertising response. + +When the existing GPT recorder consumes the matching Trusted Server opportunity +for a concrete request cycle, it emits this trace-owned sidecar: + +```text +TraceSlotCorrelationV1 + schema_version: 1 + diagnostic_auction_id: string + slot_ref: string + runtime_slot_number: safe positive integer + request_number: safe positive integer +``` + +The recorder already decides which pending opportunity belongs to which GPT +slot/request cycle. The sidecar records that exact decision; it does not rerun +attribution or read a private store during export. It is emitted only when both +opaque server tokens and the concrete GPT cycle are present. The current +`GptDiagnosticsExportV1` remains unchanged and the sidecar contains no slot +element ID or ad-unit path. + +- **Initial-navigation SSAT:** core builds the evidence when the split auction + is collected at the held body tail. It injects the script-safe public model + beside the winning-bid map before initial ad initialization; the corresponding + request-scoped slot definitions already carry + `ext.trusted_server.trace_slot_ref`. Failed, abandoned, skipped, and zero-bid + outcomes still inject their bounded evidence when the publisher document can + be delivered. +- **SPA page-bids:** `/_ts/page-bids` adds an optional, namespaced + `trace_auction` transport envelope beside its existing bid result. TSJS + validates and records it and consumes each returned slot's + `ext.trusted_server.trace_slot_ref` before triggering ad initialization. Both + the envelope and slot extensions are absent when the trace gate is inactive. +- **Trusted Server `/auction` API:** the existing OpenRTB response adds a + namespaced `ext.trusted_server.trace_auction` transport envelope only for an + active diagnostics request. After producing the final grouped `AdRequest`, + both TSJS callers assign one fresh token to each outgoing unit and retain that + exact request-scoped mapping. They validate the echoed evidence and record it + before parsing bids. A converted or skipped unit therefore cannot shift + another slot's correlation. The response member does not replace or expose + the existing orchestrator extension, and the trace projection must not copy + that extension's provider names, bidder names, metadata, price, creative IDs, + domains, or markup. HTTP/transport failures with no readable response are + browser-observed failures only; no successful server evidence is + manufactured. + +The direct TSJS caller records `evidence_transport_failed` from its existing +non-OK, unreadable-JSON, and rejected-`fetch` paths. The Prebid adapter +creates one bounded pending transport record after `buildRequests`, keyed by the +request's original bid IDs and its normalized unit tokens. `interpretResponse` +consumes it on a readable response; the pinned Prebid `onTimeout` and +`onBidderError` bidder-spec hooks consume it and record +`evidence_transport_failed` otherwise. Repeated hooks are idempotent. Pending +records are capped at 128 and expire after the configured bid timeout plus five +seconds. Expiry without any supported success/error/timeout hook proves no +transport outcome, so it removes the marker and leaves evidence `not_observed` +rather than inventing a failure. These hooks collect only bounded categories and +opaque tokens, never XHR error text or response bodies. + +The diagnostic auction token is also attached to the existing GPT opportunity +marker, and the opaque slot token is carried through the corresponding +winning-bid/slot initialization path. The numeric ordinal is never a +correlation key. The viewer joins a server slot to a GPT cycle only when one validated +`TraceSlotCorrelationV1` exactly matches both tokens and the exported +runtime/request numbers. It displays unmatched, duplicate, or conflicting +records independently, preserves competing paths, and never joins by +timestamps, implicit array position, ad-unit path, or a best-effort heuristic. + +TSJS retains at most the newest 16 validated server-auction records and 128 +correlation sidecars in memory. It increments checked eviction counters for +older records; the snapshot adds those counts to the matching truncation fields +and emits `record_evicted` with `partial`. It performs no storage write until +the explicit snapshot action. +Requests for which either side of the trace-capture gate is false do not mint +trace tokens, build trace evidence, add response members, emit sidecars, or +install auction-evidence listeners. + +No database, server-side report store, Tinybird query, beacon, or follow-up +network request is required. Evidence already available at the live auction +boundary is projected and carried forward in the response that the browser is +already receiving. + +### 9.5 Storage limits and expiry - Storage key: a namespaced, versioned constant owned by the diagnostics module. @@ -624,9 +974,10 @@ because it represents the publisher document request rather than snapshot time. - One supported report for the current browsing context; a new explicit snapshot replaces the old report. Browser opener cloning and session restore may copy or retain it. -- The runtime validator accepts only the exact outer and trace-auction v1 - schemas, rejects unknown fields, applies the limits below, and checks compact - UTF-8 size before rendering. +- The runtime validator accepts only the exact outer, server-auction, + slot-correlation, auction-coverage, and GPT-projection v1 schemas, rejects + unknown fields, applies the limits below, and checks compact UTF-8 size before + rendering. - Invalid, oversized, expired, unsupported, or hostile reports are removed when possible and otherwise ignored. Rendering uses DOM properties and `textContent`, never report-derived HTML. @@ -645,6 +996,11 @@ Runtime limits are part of the v1 contract: | Value | Limit | | --------------------------------------------- | ------------------------------------------------------------ | | Container nesting | 8 levels | +| Server auctions | 16 | +| Slot correlations | 128 | +| Provider calls | 16 per server auction | +| Auction slots | 64 per server auction | +| Auction coverage issues | 16 | | Slots | 64 | | Request cycles | 10 per slot before total-size truncation | | Callback issues | 128 | @@ -656,6 +1012,7 @@ Runtime limits are part of the v1 contract: | GPT pathname in trace projection | Exact literal `/[redacted]` | | Slot element ID and ad-unit path | 512 UTF-8 bytes each | | Trusted Server auction ID and callback reason | 256 UTF-8 bytes each | +| Diagnostic auction ID and opaque slot ref | 128 UTF-8 bytes each | | Any other string | 128 UTF-8 bytes | | Enum | Exact documented value only | | Identifier, sequence, or counter | Finite safe integer from 0 through `Number.MAX_SAFE_INTEGER` | @@ -668,15 +1025,24 @@ characters or bidirectional override/isolate controls. This applies to browser source fields as well as platform fields and precedes rendering or export. The snapshot builder creates a new field-by-field projection and rejects an -invalid source value rather than stringifying it. It retains only the first -documented number of requested sizes, yield-group IDs, company IDs, and creative -failure enums, recording discarded entries in `omitted_nested_values`; strings -are never silently shortened. It then measures the complete compact UTF-8 -storage wrapper. If it exceeds 512 KiB, it removes the globally oldest request -cycles first while retaining the newest cycle for each slot, then the oldest -callback issues, then the oldest attribution issues, and finally the oldest -remaining request cycles until the report fits. It records every removal in -`truncation`. +invalid source value rather than stringifying it. Server auctions are already +bounded by core; the browser rejects an invalid inner model rather than +truncating it. The builder retains the newest 16 server auctions in observation +order, the newest 128 correlations in recorder emission order, and only the +first documented number of GPT requested sizes, yield-group IDs, company IDs, +and creative failure enums. It records each discard in +`omitted_server_auctions`, `omitted_slot_correlations`, or +`omitted_nested_values`; strings are never silently shortened. It then measures +the complete compact UTF-8 storage wrapper. If it exceeds 512 KiB, it removes +the globally oldest GPT +request cycles first while retaining the newest cycle for each GPT slot, then +the oldest callback issues, then the oldest attribution issues, then the oldest +uncorrelated server auctions, and finally the oldest remaining GPT cycles and +server auctions until the report fits. It records every removal in +`truncation`. A correlated auction and the newest GPT cycle that references it +are retained or removed together once the algorithm reaches correlated server +auctions; every sidecar referencing a removed auction or cycle is removed and +counted. The report must not retain a dangling token while claiming a join. A report that still cannot fit after this bounded procedure fails snapshot creation. The implementation must include a worst-case fixture proving the result is bounded. @@ -691,48 +1057,61 @@ primitives do not. No accepted report value may enter a ninth container level. The storage wrapper is validated separately as the exact two-field object `{ stored_at_ms, report }`. -For deterministic ordering, a request cycle with no `requestedAtMs` sorts -before a cycle with a timestamp; otherwise cycles sort by `requestedAtMs`, then -`runtimeSlotNumber`, then `requestNumber`. Callback and attribution issues sort -by `timestampMs`, then their original array index. The builder preserves the -relative order of all retained records. +For deterministic ordering, retained server auctions remain in observation +order, provider calls and auction slots retain their server-assigned numeric +order, and correlations retain recorder emission order. A request cycle with no +`requestedAtMs` sorts before a cycle with a timestamp; otherwise cycles sort by +`requestedAtMs`, then `runtimeSlotNumber`, then `requestNumber`. Callback and +attribution issues sort by `timestampMs`, then their original array index. The +builder preserves the relative order of all retained records. ## 10. Auction and rendering evidence -The report uses TS Console's evidence model. It must preserve the distinction -between: +The report combines, but never conflates, the server model from section 9.4 and +TS Console's browser model. It must preserve the distinction between: -- A Trusted Server opportunity. -- A provider response. -- A selected Trusted Server candidate. +- A server-observed auction and its source. +- An auction-wide provider call and response. +- A server-selected candidate for one opaque slot reference. +- A browser-observed Trusted Server opportunity. - A GPT request and response. - A non-empty GPT render. - Trusted Server creative-bridge evidence. - Creative load and viewability. - A publisher or client-side refresh. -The viewer must not infer that Trusted Server rendered an ad merely because GPT -reported a filled slot. Ambiguous and unattributed cycles remain explicit. - -Current diagnostics tokens exist only on delivered winning bids. No-bid, -failed, skipped, hidden, unresolved, and direct `/auction` paths can lack server -correlation. The report displays the available observed facts and `Unknown` -rather than manufacturing a correlation. +The diagnostic auction token is created before dispatch rather than only on a +delivered winner, so zero-bid and terminal failure states have an identity when +a response can carry evidence. The viewer still must not infer that Trusted +Server rendered an ad merely because the server selected a candidate or GPT +reported a filled slot. Ambiguous, competing, unmatched, and unattributed +cycles remain explicit. Provider-call telemetry is auction-wide, while bid rows exist only for returned bids. Exact `provider X was asked for slot Y` and exact no-bid causality require a future provider-impression disposition model. That instrumentation is not silently assumed by this design. -Timing fields introduced by #1074/#1076 are outside the v1 compatibility -matrix. They may be consumed in a later version only after they merge and are -propagated through the public live-diagnostics contract. The report never -queries Tinybird, and it does not combine browser `performance.now()` values -with server-relative timing as though they were one clock. +The UI groups timing into three labeled clocks: + +1. **Server auction-local:** v1 `total_time_ms` and provider + `response_time_ms`, measured from the live orchestration result. +2. **Request-relative server milestones:** dispatched, resolved, and committed + milestones from #1076, unavailable in this schema and adoptable later. +3. **Browser/GPT:** TS Console request, response, render, load, and viewability + timings. -Bidder and winning price are not added by version one. A later version may -consume them only if #1081 approves them in the public TS Console export -contract. #1050 does not independently weaken the existing privacy policy. +The report never queries Tinybird and never subtracts or combines values from +different clocks. #1074/#1076 may add request-relative fields only through a +separately reviewed compatibility change. + +Bidder identity, provider identity, winning price, currency, creative numbering, +and a final `SSAT/TS/client-side winner` label are not added by version one. A +later version may consume them only if #1081 approves their meaning and public +disclosure policy. #1050 does not independently weaken the existing privacy +policy. Version one can nevertheless answer the narrower, evidence-based +questions: which Trusted Server entry point ran, what bounded server outcome it +reported, what GPT did afterward, and where correlation is missing. ## 11. Network scope @@ -798,13 +1177,15 @@ the report. Validation errors expose only bounded categories. Forbidden data includes: - Raw `Cookie` and `Set-Cookie` headers. -- EC IDs, EIDs, bidder user IDs, and consent strings. +- EC IDs, EIDs, bidder user IDs, provider/bidder/seat names, and consent + strings. - Unmasked client IP. - Query strings and fragments. - Fastly or internal request identifiers that can join to user-bearing logs. - Internal `AuctionRequest.id`. -- Bid requests/responses, losing-bid payloads, targeting, creative markup, - cache URLs, and stack traces. +- Bid requests/responses, bid prices/currency, losing-bid payloads, provider + metadata, targeting, creative IDs/domains/markup, cache URLs, and stack + traces. ### 12.2 Same-origin script visibility @@ -815,15 +1196,17 @@ may intercept navigation. Therefore the stored model must be safe even if read or forged by any same-origin code. A random storage key, closed shadow root, or public endpoint does not change this requirement. -Every report view and exported artifact is labeled `Browser-observed, -unverified diagnostic data`. Support documentation says that it helps -troubleshoot rendering but is not proof of a server event, user identity, or -security incident. +Every report view and exported artifact is labeled `Browser-carried, +unverified diagnostic data`. Individual server entries retain their +server-produced provenance, but support documentation says the browser-carried +copy helps troubleshoot rendering and is not cryptographic proof of a server +event, user identity, or security incident. ### 12.3 Response hardening -The HTML shell, enable/end responses, and every active diagnostic publisher -response are terminally `private, no-store`. The fixed versioned JS/CSS assets +The HTML shell, enable/end responses, every active diagnostic publisher +response, and every dynamic page-bids or `/auction` response carrying trace +evidence are terminally `private, no-store`. The fixed versioned JS/CSS assets are the sole exception and may be publicly cached because they contain no request or report data. HTML and JSON endpoint responses also send: @@ -859,6 +1242,9 @@ interference remains inside the stated trust limitation. ### 12.4 Shared templates and ESI Per-request trace context must never enter a shared template or ESI fragment. +This includes diagnostic auction/slot tokens and the optional `AuctionSlot` +extension; active responses add them only in request-scoped injection or the +request-scoped body seam. The existing diagnostics private/no-store decision remains a load-bearing gate. Tests must prove that late response-header handlers cannot make traced content publicly cacheable. @@ -873,6 +1259,15 @@ publicly cacheable. unavailable state without a value or parser message. - Diagnostics context serialization failure: omit the context, log a bounded server error, and preserve publisher delivery. +- Server-auction evidence construction or serialization failure: omit only the + affected evidence, retain a bounded `unavailable` coverage marker when safe, + log no report content, and preserve the normal auction/result path. +- Initial-navigation evidence cannot be injected because the body tail is not + reached: preserve publisher delivery. Because the browser received no safe + marker, display `not_observed` rather than claiming a known server failure. +- Page-bids or `/auction` evidence is absent or rejected by its strict client + validator: parse the ordinary bid response exactly as before, discard the + diagnostic member, and show an unmatched/invalid-evidence coverage category. - TS Console capture failure: fail open for advertising and show incomplete coverage in diagnostics. - Storage unavailable or quota exceeded after a valid bounded report exists: @@ -896,8 +1291,10 @@ publicly cacheable. removed and retain the always-available deletion retry, regardless of the server end result. -Diagnostic failures must never suppress, delay, add, remove, or reorder GPT -requests, auctions, targeting, or creative rendering. +Diagnostic failures must never suppress, delay, add, remove, reorder, or change +the success status of GPT requests, auctions, bid responses, targeting, or +creative rendering. The evidence projection is a side effect of already-known +results, never a prerequisite for returning them. ## 14. Testing strategy @@ -905,6 +1302,10 @@ requests, auctions, targeting, or creative rendering. - Configuration defaults off and rejects trace-page enablement without GPT diagnostics. +- With GPT diagnostics enabled but `trace_page_enabled = false`, a valid + console cookie still enables the existing console but never mints trace + tokens, builds auction evidence, adds response extensions, or emits + correlation sidecars. - Exact reserved-route classification, canonical-path, query, method, encoded path, and fallback behavior. - Exact versioned asset routes are local and contain no dynamic data; lookalike @@ -925,6 +1326,38 @@ requests, auctions, targeting, or creative rendering. - Request-context serializer masks IPv4/IPv6; enforces every string bound; and omits page paths, fingerprints, query, raw headers, IDs, and unsupported fields. +- Server-auction projection maps initial navigation, SPA page-bids, and auction + API call sites to the exact public source enums without using a browser hint. +- The diagnostic auction token is minted before dispatch and remains identical + across completed, zero-bid, skipped, failed, and abandoned evidence and the + corresponding browser opportunity marker. It never equals or contains the + internal auction ID or telemetry UUID. +- Server-auction projection covers every terminal status/reason mapping, + auction-local total duration, provider role/status/duration/count, per-slot + requested sizes/bid count/candidate disposition, and checked numeric + conversion. +- Provider numbering and opaque slot references are deterministic and bounded; + provider names, bidder/seat names, prices, currency, publisher slot IDs, + creative identifiers/domains/markup, metadata, raw errors, and internal IDs + are absent from serialized fixtures. +- Auction-wide provider no-bid evidence is never projected as a per-slot + disposition, and `provider_to_slot_no_bid` remains `unavailable`. +- Evidence projection/serialization failure leaves the ordinary bid map or + OpenRTB response unchanged apart from the bounded unavailable transport + envelope. +- `/auction` request tokens survive the exact AdRequest-to-AuctionRequest slot + conversion, are stripped before provider dispatch, and remain correctly + associated across grouped multi-bidder units, duplicate codes, skipped + non-banner units, and mixed accepted/filtered inputs. Numeric ordinals are + never used for client correlation. +- Token tests cover canonical UUID-v4 shape, missing Web Crypto, malformed or + duplicate request extensions, the disabled trace gate, and proof that invalid + tokens neither fail nor otherwise alter the ordinary auction. +- Initial and SPA slot JSON attaches the exact + `ext.trusted_server.trace_slot_ref` token that appears in server evidence; + inactive responses omit it. Missing, duplicate, conflicting, malformed, and + evidence-mismatched slot tokens suppress only correlation and produce the + specified coverage issues without changing slot/bid order or contents. - Active responses remain terminally private/no-store under hostile late header overrides. - Dynamic HTML/JSON values cannot close elements or create executable script. @@ -949,6 +1382,31 @@ requests, auctions, targeting, or creative rendering. future-clock skew; wall-clock rollback; replacement; clearing; and storage exceptions. - Omission counters use checked arithmetic and reject overflow. +- Strict validation of every server-auction enum, token, numeric bound, array + bound, nesting level, and unknown property; invalid evidence is discarded + without changing ordinary bid parsing. +- Transport envelopes require exactly one of evidence/unavailable reason; + projection, transport, validation, eviction, correlation, and external-client + coverage states produce the specified complete/partial/unavailable/not-observed + result without treating absence as proof that no auction ran. +- Direct fetch failures and Prebid `interpretResponse`, `onTimeout`, and + `onBidderError` paths consume their pending transport record exactly once; + capped/expired records and absent hooks follow the specified `not_observed` + behavior without retaining error text or bodies. +- Exact-token correlation joins matching server auctions, GPT opportunities, + and slot references; unmatched, duplicated, conflicting, missing, and + forged tokens stay separate and produce explicit coverage states. No + timestamp, index, or ad-unit-path heuristic is used. +- `TraceSlotCorrelationV1` is emitted only at the existing recorder's exact + opportunity-to-cycle binding, is capped and evicted deterministically, + contains only opaque tokens plus runtime/request numbers, and does not alter + `GptDiagnosticsExportV1`. +- Source presentation distinguishes server-owned SSAT/page-bids/auction API + facts from browser-observed publisher refresh, Prebid refresh, competing, and + unattributed request paths. No fixture turns intent or a GPT fill into a + winner assertion. +- Server auction-local, request-relative unavailable, and browser/GPT timings + render in separate labeled groups and are never combined arithmetically. - Trace projection replaces the nested GPT pathname with `/[redacted]`, rejects any other stored value, emits `TraceGptDiagnosticsV1`, applies deterministic ordering/truncation, and records exact omission counts in a worst-case 512 KiB @@ -971,8 +1429,20 @@ requests, auctions, targeting, or creative rendering. - Cross-site top-level GET, form POST, and fetch attempts cannot enable or end tracing. - A real fixture reload activates diagnostics and captures multiple slots. +- Initial-navigation SSAT fixtures cover selected, no-candidate, + selected-unrenderable, skipped, dispatch-failed, execution-failed, and + abandoned outcomes without requiring a winning bid. +- SPA page-bids and both TSJS `/auction` callers consume the optional + `trace_auction` member before ad initialization/bid parsing; inactive + responses have no member, and malformed members do not affect bids. +- Initial seams pass the optional transport as the scheduler's third argument, + SPA JSON uses the exact optional top-level member, and old-scheduler/absent + transport fixtures preserve ad initialization while reporting `not_observed`. - `View trace results` navigates in the same tab and renders the captured request context and slot evidence. +- A correlated fixture renders the chain `server auction -> GPT -> creative`, + while unmatched server, client-side refresh, competing, and transport-failure + fixtures show honest independent evidence and `Unknown` where appropriate. - Empty, filled, ambiguous, no-candidate, and unattributed slot states remain distinct. - Reloading the trace page retains an unexpired same-tab report. @@ -996,7 +1466,10 @@ requests, auctions, targeting, or creative rendering. - Immutable asset fixtures prove published v1 bytes never change; changed bytes require a new URL referenced by the shell. - Inactive publisher traffic has no trace assets, storage access, listeners, or - cache-policy change. + cache-policy change, diagnostic token generation, or trace-auction response + extension. +- Disabling `trace_page_enabled` removes every new capture behavior even when a + technical `?ts_console=1` session leaves a valid diagnostics cookie present. ### 14.5 Manual acceptance @@ -1007,6 +1480,9 @@ fixture: - Touch targets, scrolling, zoom, safe areas, download, copy, and native share behavior are usable. - The user can distinguish setup information from captured-page information. +- The user can distinguish `SSAT/Trusted Server auction ran`, `browser refresh +observed`, `GPT filled/rendered`, and `Unknown` without understanding internal + request-path names. - A failed share or download does not lose the visible report. ## 15. Rollout and observability @@ -1032,58 +1508,74 @@ fixture: and a cross-site GET cannot activate tracing. 3. The setup page accurately explains that the problem must be reproduced after activation. -4. A subsequent real publisher-page reload captures redacted request context - and existing TS Console evidence without altering ad behavior. +4. A subsequent real publisher-page reload captures redacted request context, + live server-auction evidence, and existing TS Console evidence without + altering ad behavior. 5. `View trace results` transfers one bounded, runtime-validated snapshot in the supported same-tab journey and opens the report page without server-side storage or claims of browser-storage isolation. -6. The report separates network, cookie health, auction/render evidence, and - coverage/unknowns. +6. The report separates network, cookie health, server auction, GPT delivery, + creative rendering, and coverage/unknowns. 7. JSON export contains the same versioned allowlisted information shown on the page. 8. No raw cookies, user IDs, full IPs, consent strings, exact page paths, query - strings, fingerprints, internal auction IDs, targeting, or creative payloads - appear in trace HTML, browser storage, logs, or export. + strings, fingerprints, internal auction IDs, provider/bidder/seat names, + prices, targeting, or creative identifiers/payloads appear in trace HTML, + browser storage, trace-specific logs, or export. 9. Trace HTML and active publisher pages remain terminally private/no-store. -10. Missing platform fields, incomplete auction correlation, storage failure, - and unavailable share APIs degrade honestly without affecting advertising. +10. Missing platform fields, failed evidence projection/transport, incomplete + auction correlation, storage failure, and unavailable share APIs degrade + honestly without affecting advertising. 11. The full report is usable at 320 CSS pixels and with keyboard/screen-reader navigation. -12. Version one accepts exactly `TraceReportV1` with - `TraceGptDiagnosticsV1`, sourced only from `GptDiagnosticsExportV1`; #1081 - and #1074/#1076 are optional additive follow-ups rather than release gates. -13. Every rendered and exported report is identified as browser-observed and - unverified, and hostile storage content cannot create executable HTML or +12. Version one accepts exactly `TraceReportV1` with server-produced + `TraceAuctionEvidenceV1`, browser-produced `TraceSlotCorrelationV1`, and + `TraceGptDiagnosticsV1`; only the latter is sourced from + `GptDiagnosticsExportV1`. #1081 and #1074/#1076 are optional additive + follow-ups rather than release gates. +13. Every rendered and exported report is identified as browser-carried and + unverified; server-produced and browser-observed entries retain distinct + provenance, and hostile storage content cannot create executable HTML or unbounded DOM output. 14. Enable/end operations expose partial failure honestly and are safe to retry; the UI does not claim server-observed cookie state without the follow-up state request or claim that local data was cleared when deletion fails. +15. A report can show that initial-navigation SSAT, SPA page-bids, or the + Trusted Server auction API ran; show its bounded provider and per-slot + outcome; show subsequent GPT/creative evidence; and display `Unknown` + rather than inventing a client-side winner or an unsupported correlation. ## 17. Implementation sequencing -This design is one product flow, but its implementation is split into three -independently reviewable plans and preferably three PRs: +This design is one product flow, but its implementation is split into four +independently reviewable plans and preferably four PRs: 1. **Reserved route and privacy foundation:** configuration, shared early-route classification, same-origin enable/end lifecycle, bounded cookie-health inspection, base request-context schema, projection of already populated `ClientInfo`/`GeoInfo` fields, response hardening, and adapter parity. Do not add speculative new platform fields in this change. -2. **Browser handoff and viewer:** integrate current +2. **Live server-auction evidence:** introduce the public diagnostic auction and + slot tokens, project `TraceAuctionEvidenceV1` at the live observation + boundary, transport it through initial navigation, page-bids, and both + TSJS `/auction` callers, emit `TraceSlotCorrelationV1` from the existing GPT + recorder binding, and prove ordinary bid behavior is unchanged on + absent/invalid evidence. +3. **Browser handoff and viewer:** integrate current `GptDiagnosticsExportV1`, project `TraceGptDiagnosticsV1`, construct and strictly validate `TraceReportV1`, implement the same-tab workflow, combined - direct/storage exports, mobile viewer, copy/share, expiry, clearing, and - browser/accessibility tests. -3. **Optional network enrichment and future schemas:** add HTTP-version, POP, + server/GPT/creative correlation, direct/storage exports, mobile viewer, + copy/share, expiry, clearing, and browser/accessibility tests. +4. **Optional network enrichment and future schemas:** add HTTP-version, POP, ASN, or other fields only from SDK-verified platform sources with explicit bounds. Adopt #1081 or #1074/#1076 later through a separately reviewed versioned compatibility change. Each plan must include its own adapter, privacy, cache, and failure tests. The -implementation must not claim completion of #1081 or the timing work as part of -#1050. Version one ships with current observed auction/render evidence and -labels unavailable fields honestly. +implementation must not claim completion of #1081 or request-relative timing as +part of #1050. Version one ships with minimal live server-auction evidence plus +current GPT/render evidence and labels unavailable fields honestly. ## 18. Rejected alternatives @@ -1148,8 +1640,22 @@ length, history, logging, referrer, and accidental-sharing risks. subdomains. - Browser privacy settings may disable storage, clipboard, download, or share capabilities. -- Current server/browser correlation does not cover every no-bid, skipped, - failed, hidden, unresolved, or direct-auction path. +- A server can report an auction failure only when it can still deliver a + response containing evidence. Network termination or an unreadable `/auction` + response remains a browser-observed transport failure with no server outcome. +- Provider calls are auction-wide. Version one cannot attribute a provider + no-bid, timeout, or error to a specific slot. +- Third-party client-side auction participants, bids, and winners are not + observable. Publisher/Prebid refresh is browser intent, not proof that a + client-side bidder won. +- `/auction` evidence requires the TSJS request to reach the same Trusted Server + host with the active diagnostics cookie. A custom cross-origin auction + endpoint does not inherit this trace session and is shown as unavailable. +- Exact-token correlation can remain unavailable for hidden, unresolved, + competing, or independently initiated GPT cycles. The report preserves both + sides instead of guessing. +- Version one has auction-local server durations and browser/GPT timings, but + request-relative dispatched/resolved/committed milestones await #1076. - Fastly-only transport details do not exist on every adapter. - The current 30-pixel TS Console controls are not sufficient for this mobile report; the endpoint uses independent 44-pixel touch targets. From e3f371f8cb102d4b7f713d375f8587a34c6f01fd Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Sat, 19 Sep 2026 11:34:08 +0530 Subject: [PATCH 4/8] Reconcile mobile trace design contracts --- ...-mobile-ad-render-trace-endpoint-design.md | 419 +++++++++++++----- 1 file changed, 302 insertions(+), 117 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md index 092466a6d..5d86b9092 100644 --- a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -14,7 +14,8 @@ ## 1. Summary -Add a deployment-controlled, public, privacy-safe `GET /_ts/trace` page for a +Add a deployment-controlled, privacy-safe `GET /_ts/trace` page, public subject +to operator authentication rules, for a mobile end user who needs to reproduce an ad-rendering problem and give support an exportable diagnostic report. Visiting the page is read-only. The user intentionally enables or ends tracing with a same-origin POST action. @@ -44,9 +45,10 @@ Issue #1050 names three required data groups: 3. End-user cookie information. The title additionally establishes two product constraints: the experience is -for a mobile user, and it is reached through an endpoint. A mobile user should -not need browser developer tools, Basic Authentication, a copied trace ID, or a -second copy of the affected page URL. +for a mobile user, and it is reached through an endpoint. On a deployment whose +authentication rules leave trace routes public, a mobile +user should not need browser developer tools, credentials, a copied trace ID, +or a second copy of the affected page URL. A standalone request cannot know what occurred in a previous document. Exact render evidence exists only while the publisher page is running in the browser. @@ -117,10 +119,13 @@ diagnostic token connects those layers. ### 5.1 Public, redacted endpoint with intentional activation -`/_ts/trace` is public when explicitly enabled by deployment configuration. It -is not placed under `/_ts/admin`, because the intended user is a layperson on a -phone and the existing Basic Authentication flow is unsuitable for that -journey. +`/_ts/trace` is available when explicitly enabled by deployment configuration +and is public only when no operator authentication rule covers it. It is not +placed under `/_ts/admin`, because the intended user is a layperson on a phone. +Existing Basic Authentication rules still apply to every trace path, including +assets and actions; the early dispatcher must not carve out an exemption. +Operators offering the credential-free journey must scope authentication to +the paths they intend to protect, for example `^/_ts/admin` (see section 12.5). Public access is safe only because both the page and export use a strict allowlist. The activation cookie is a feature toggle, not authentication. No @@ -141,19 +146,51 @@ executing on the publisher origin. The endpoint reuses `__Host-ts-console` and the existing GPT diagnostics activation semantics rather than creating a second `ts-trace` session. The cookie remains host-only, `Secure`, `HttpOnly`, and `SameSite=Lax`. -`POST /_ts/trace/enable` sets it with a fixed 30-minute `Max-Age` and does not -refresh that lifetime on publisher requests; `POST /_ts/trace/end` clears it. -Neither action accepts state-changing query parameters. The shorter endpoint -lifetime bounds accidental private/no-store operation if a user forgets to end -tracing; the existing technical query flow keeps its existing session-cookie -semantics. - -The new trace capture is active only when both `trace_page_enabled = true` and -the request carries exactly one valid diagnostics cookie. The shared cookie by +Both `POST /_ts/trace/enable` and the existing `?ts_console=1` writer must +set `Max-Age=1800` through one shared cookie policy, including when +`trace_page_enabled` is false. This explicitly changes the technical query flow +from a browser-session cookie to a 30-minute cookie. An explicit activation +restarts the 30-minute lifetime; ordinary publisher requests do not refresh it. +`POST /_ts/trace/end` and `?ts_console=0` clear the same cookie. Neither POST +action accepts state-changing query parameters. A later query activation must +never replace the bounded cookie with a session cookie. + +This bounds browser-managed persistence from the most recent activation by an +updated writer, not total session duration or server-enforced authorization. +Pre-existing session cookies carry no expiry in the request and cannot be +retroactively aged; users with those cookies must end or re-enable diagnostics +to adopt the new lifetime. Cookie expiry also does not unload a running page +or clear an existing report; report expiry is defined separately in section 9.5. + +The base trace gate requires both `trace_page_enabled = true` and exactly one +valid incoming diagnostics cookie, inspected before cookie sanitation. +Publisher-document tracing additionally requires the existing effective +`GptDiagnosticsRequestDecision.active` decision. Query disable or invalid +directives, prefetches, bots, and other ineligible navigations therefore +suppress document trace context, auction evidence, and browser activation even +when an incoming cookie is valid. A query activation without an incoming valid +cookie enables the existing console; trace capture starts on the next eligible +reload carrying that cookie. Page-bids and `/auction` requests use the base +configuration-plus-cookie gate without requiring a document-navigation +decision. References below to the trace gate include the additional effective +diagnostics decision for publisher documents. The shared cookie by itself continues to activate the existing TS Console but does not authorize trace tokens, server-auction projection, trace response extensions, or correlation sidecars on a deployment whose trace page is disabled. This makes -the configuration flag the disclosure and rollback boundary. +the configuration flag the disclosure and rollback boundary for new requests. + +Core exposes the evaluated document trace gate as the literal boolean +`window.__tsjs_trace_active` before TSJS initializes on the publisher document; +it is true only when the base gate and effective document diagnostics decision +are both active. TSJS requires +`window.__tsjs_trace_active === true` before minting slot tokens, retaining +request mappings, installing trace listeners, collecting transport evidence, +emitting sidecars, or offering the trace handoff action. Missing or non-boolean +values mean inactive; `window.__tsjs_gpt_diagnostics_active` alone is +insufficient. Core independently rechecks the gate on each request and never +trusts the browser flag as authorization. Configuration changes or cookie expiry +cannot revoke code already loaded in a document; rollback takes effect on its +next reload, and subsequent server requests stop returning evidence immediately. ### 5.3 Browser-local, explicit handoff @@ -324,7 +361,7 @@ the report. ```text GET /_ts/trace | - |-- early reserved-route classifier terminates locally + |-- early reserved-route classifier applies configured auth, then terminates locally |-- HTML explains forward reproduction; no state mutation v POST /_ts/trace/enable after explicit user action @@ -411,7 +448,7 @@ enabled = true trace_page_enabled = false ``` -Rules: +Rules (route responses below apply after configured authentication): - `trace_page_enabled = true` requires `enabled = true`; invalid combinations fail configuration validation. @@ -481,9 +518,15 @@ Every adapter implements the following order: 1. Parse the method, canonical host/origin, path, query, and bounded headers required for route safety. 2. Classify an exact Trusted Server reserved path. -3. For a trace path, terminate locally after only trace-specific validation and - bounded request-context inspection, including an optional read-only platform - geo lookup used solely for the displayed setup request. +3. For a trace path, enforce the existing configured Basic Authentication + rules before any trace response, cookie mutation, body inspection, or setup + context projection. A matching rule challenges missing/invalid credentials + locally with the ordinary `401` and `WWW-Authenticate` behavior. After + authentication succeeds or no rule matches, apply the trace-specific + validation and bounded request-context inspection, including an optional + read-only platform geo lookup, and terminate locally. This applies to the + entire reserved namespace, assets, unsupported methods, and disabled routes; + the route statuses below authentication are never an auth exemption. 4. For all other paths, continue through the adapter's ordinary event context, authentication, request filters, geo enrichment, EC/EID processing, named routes, auction handling, telemetry, and publisher fallback. @@ -496,8 +539,9 @@ ordinary named-route registration alone does not satisfy this contract. After an enable or end POST succeeds, the client performs a no-store state GET. It claims `Tracing is on — cookie observed by server` only when that separate -request reports active, and `Tracing is off — cookie absent on server request` -only when it reports inactive. A mismatch or failed verification is +request reports active, and `Tracing is off — no valid diagnostics session +observed` only when it reports inactive. An inactive result covers absent, +invalid, duplicate, or uninspectable cookies; it does not prove cookie absence. A mismatch or failed verification is `Activation unconfirmed` or `Deactivation unconfirmed` and offers an idempotent retry. These are server-observation statements, not proof that browser state is authentic: same-origin service workers can forge or suppress the whole exchange. @@ -508,7 +552,8 @@ the uncached shell, state, and action routes being disabled; inert cached assets alone cannot activate tracing or access a report page. The current `?ts_console=1` and `?ts_console=0` activation flow remains -supported for technical users. Both activation surfaces drive the same cookie +supported for technical users with the shared 30-minute cookie policy in +section 5.2. Both activation surfaces drive the same cookie and runtime; they must not create two concurrent diagnostic modes. That pre-existing query flow has its existing top-level-navigation activation risk; #1050 neither expands it to the new trace GET nor claims to remediate it. @@ -517,8 +562,8 @@ pre-existing query flow has its existing top-level-navigation activation risk; ### 9.1 Request context -The server injects one immutable `TraceRequestContextV1` into active diagnostic -documents: +The server injects one immutable `TraceRequestContextV1` only into documents +that satisfy the applicable trace gate in section 5.2: ```text TraceRequestContextV1 @@ -546,8 +591,12 @@ The request-context envelope intentionally contains no page URL, path, referrer, query, or fragment. During the field-by-field trace projection, `GptDiagnosticsExportV1.page.origin` is retained after validation and its `pathname` is replaced with the literal `/[redacted]`. The trace viewer accepts -only that literal. Version one therefore does not store or export an exact page -path. Any future route-template policy requires a new schema and privacy review +only that literal. The projection also omits `slotElementId` and `adUnitPath` +from slots and `slotElementId` from callback and attribution issues, because +these values can embed the same page path. They are not hashed or truncated. +Numbered slots and request cycles retain grouping and exact-token correlation. +Version one therefore does not intentionally retain an exact page path in any +of these source fields. Any future route-template policy requires a new schema and privacy review because paths can contain accounts, emails, preview tokens, and other secrets. `masked_client_ip` uses a deterministic display-only mask for the current @@ -661,13 +710,26 @@ projection sourced only from `GptDiagnosticsExportV1`. It contains: - `schema_version: 1` and `source_schema_version: 1`; - the source `capturedAt` value; - `page.origin` after validation and `page.pathname` fixed to `/[redacted]`; -- field-for-field allowlisted copies of the current v1 slots, requests, - callback issues, attribution issues, coverage, and metadata, subject to the - bounds and truncation below. +- explicit field-by-field projections of current v1 slots, requests, callback + issues, attribution issues, coverage, and metadata, subject to the following + exclusions and the bounds/truncation below. + +The trace schema omits the entire request-cycle `adManager` object, including +`lineItemId`, `creativeId`, `campaignId`, `advertiserId`, +`sourceAgnosticLineItemId`, `sourceAgnosticCreativeId`, `yieldGroupIds`, and +`companyIds`, and omits `previousCreativeId`. Derived `responseClass` and +`creativeChanged` facts remain eligible without their underlying identifiers. +It also omits slot `slotElementId` and `adUnitPath` and callback/attribution +issue `slotElementId`. These properties are forbidden in the trace schema, +not optional passthrough fields: the builder never copies them and the viewer +rejects a stored report that supplies them. The remaining current v1 fields +retain their source meaning and require explicit allowlisting; future source +fields are not inherited automatically. `trustedServerAuctionId`, when +present, must satisfy the diagnostic auction-token contract in section 9.4. It is deliberately not named or represented as `GptDiagnosticsExportV1`, -because the fixed pathname and trace-level bounds change the source field -semantics. TS Console continues to own the source schema; the trace envelope +because redaction, excluded identifiers, and trace-level bounds change the +source field semantics. TS Console continues to own the source schema; the trace envelope owns its public projection and transport. The initial compatibility matrix is exactly `TraceReportV1`, `TraceAuctionEvidenceV1`, `TraceSlotCorrelationV1`, and `TraceGptDiagnosticsV1`, with the GPT projection sourced from @@ -678,13 +740,17 @@ an additive source compatibility change and, if the public projection changes, a new trace-envelope version. `auction_coverage.capture_status` describes only what reached the browser -collector: `not_observed` means no valid server-auction record arrived, not that -no server auction ran. `partial` requires at least one retained record plus a -projection, transport, validation, or eviction issue; `unavailable` requires no -retained records plus a known projection, transport, or validation issue; and +collector: `not_observed` means no valid server-auction record arrived and no +capture issue is known, not that no server auction ran. `partial` requires at +least one retained record plus a projection, transport, validation, or eviction +issue; `unavailable` requires no retained records plus a known projection, +transport, validation, or eviction issue; and `complete` requires at least one retained record without those capture issues. `correlation_unavailable` and `external_client_side_unobservable` describe interpretation limits and do not change an otherwise complete capture status. +Evicting all received records therefore yields `unavailable` with +`record_evicted`, never `not_observed`. Recompute status after both in-memory +eviction and snapshot size truncation. The issue array is deduplicated, sorted in enum order, bounded to 16 values, and contains no error text. @@ -739,24 +805,29 @@ The model has deliberately lower cardinality and sensitivity than the existing telemetry and OpenRTB objects: - `diagnostic_auction_id` is a fresh opaque `ts-auc-...` correlation token. When - trace capture is active under the two-part gate in section 5.2, it is minted + trace capture is active under the applicable trace gate in section 5.2, it is minted once when an eligible auction is observed, before dispatch, and is retained for zero-bid, skipped, dispatch-failed, execution-failed, and abandoned outcomes. It is never `AuctionRequest.id`, the telemetry UUID, a provider request ID, or an identifier joinable to user-bearing logs. -- Auction and slot tokens are the fixed prefixes `ts-auc-` and `ts-slot-` - followed by a canonical lowercase hyphenated UUID v4. Validators reject every - other shape; tokens are not silently shortened or normalized. +- Auction tokens are `ts-auc-` followed by a lowercase UUID v4 in the existing + producer's 32-hex-digit unhyphenated form (`Uuid::new_v4().simple()`). Slot + tokens are `ts-slot-` followed by a canonical lowercase hyphenated UUID v4, + matching `crypto.randomUUID()`. Both validators enforce UUID version 4 and + the RFC variant and reject every other shape. Tokens are compared verbatim, + never normalized during validation or correlation; no producer format change + is required for the existing GPT auction opportunity marker. - `slot_number` is a one-based ordinal over the exact post-conversion `AuctionRequest.slots` sequence observed by orchestration. It is display-only and is never used to map a response back to pre-conversion client input. `slot_ref` is a fresh auction-local opaque token carried with that slot. Core creates it for initial-navigation and SPA auctions. For a TSJS `/auction` - request, TSJS creates it only after `buildAdRequest` has finished grouping and + request, TSJS creates it only when `window.__tsjs_trace_active === true` and + only after `buildAdRequest` has finished grouping and deduplicating the final `adUnits` array, attaches it to that exact outgoing unit as `adUnits[].ext.trusted_server.trace_slot_ref`, and retains the request-scoped token-to-unit mapping. Core accepts that member only under the - two-part trace gate, validates and echoes the token for accepted converted + applicable trace gate, validates and echoes the token for accepted converted slots, and strips it before every provider or mediator request. A missing or invalid client token causes core to mint a server token with no browser correlation; it never changes ordinary auction acceptance. TSJS uses @@ -812,8 +883,8 @@ correlation failed, the viewer shows `Server auction evidence unavailable` or #### 9.4.1 Live transport and correlation -Evidence is transported only while `trace_page_enabled` is true and the -diagnostics cookie is valid. Every response carrying it is terminally +Evidence is transported only when the applicable trace gate in section 5.2 +is active, including the effective diagnostics decision for publisher documents. Every response carrying it is terminally `private, no-store`: ```text @@ -836,7 +907,7 @@ object TSJS already consumes: AuctionSlot.ext.trusted_server.trace_slot_ref: string ``` -Core adds that optional nested member only under the two-part trace gate. It +Core adds that optional nested member only under the applicable trace gate. It assigns the token while constructing the request-scoped slot definitions and threads the same token into the corresponding `AuctionRequest` observation, so neither side needs to recover the relationship from an ordinal or raw slot ID. @@ -845,13 +916,15 @@ ordering, and bid-map keys remain unchanged. The extension is absent when the gate is false and is never copied into `TraceAuctionEvidenceV1` except as its already-allowlisted opaque `slot_ref`. -TSJS accepts a slot extension only when its canonical token occurs exactly once +When valid auction evidence is supplied, TSJS accepts a slot extension only +when its canonical token occurs exactly once in both the delivered slot list and the matching auction evidence. A missing, malformed, duplicate, or conflicting token prevents only that sidecar join, adds `evidence_validation_failed` and `correlation_unavailable`, and does not drop, reorder, or mutate the ordinary slot or bid. TSJS reads no other extension property. This validation occurs before the slot is handed to the existing GPT -initialization path. +initialization path. An absent optional transport member does not trigger +missing-token validation; no sidecar is emitted and no capture issue is added. The three transport call shapes are exact v1 contracts: @@ -876,8 +949,9 @@ accepts only the exact optional top-level member and preserves its existing `slots` and `bids` behavior. These trace members never become required for a successful advertising response. -When the existing GPT recorder consumes the matching Trusted Server opportunity -for a concrete request cycle, it emits this trace-owned sidecar: +For initial-navigation SSAT and SPA page-bids only, when the existing GPT +recorder consumes the matching Trusted Server opportunity for a concrete request +cycle, it emits this trace-owned sidecar: ```text TraceSlotCorrelationV1 @@ -895,6 +969,19 @@ opaque server tokens and the concrete GPT cycle are present. The current `GptDiagnosticsExportV1` remains unchanged and the sidecar contains no slot element ID or ad-unit path. +Version one does not correlate either `/auction` caller to a GPT cycle. Prebid +refresh records only `prebid_refresh` intent and has no existing token-bearing +opportunity binding; passing its tokens through `recordTrustedServerOpportunity` +would incorrectly introduce `trusted_server_direct` attribution. The direct +`requestAds` caller renders outside GPT and supplies no GPT request cycle. Both +callers retain server evidence and the request-unit mapping, but emit no +`TraceSlotCorrelationV1`. The viewer displays API evidence independently with +`correlation_unavailable`; this interpretation limit does not downgrade an +otherwise complete server capture. A sidecar referencing an `auction_api` +record is invalid in v1. A future Prebid join requires a separately designed +association between request-unit tokens and exact GPT slot/request identities +that preserves existing request-path attribution. + - **Initial-navigation SSAT:** core builds the evidence when the split auction is collected at the held body tail. It injects the script-safe public model beside the winning-bid map before initial ad initialization; the corresponding @@ -908,12 +995,17 @@ element ID or ad-unit path. `ext.trusted_server.trace_slot_ref` before triggering ad initialization. Both the envelope and slot extensions are absent when the trace gate is inactive. - **Trusted Server `/auction` API:** the existing OpenRTB response adds a - namespaced `ext.trusted_server.trace_auction` transport envelope only for an - active diagnostics request. After producing the final grouped `AdRequest`, - both TSJS callers assign one fresh token to each outgoing unit and retain that - exact request-scoped mapping. They validate the echoed evidence and record it + namespaced `ext.trusted_server.trace_auction` transport envelope only for a + request satisfying the base trace gate in section 5.2. After producing the + final grouped `AdRequest`, + both TSJS callers check `window.__tsjs_trace_active === true` before + assigning one fresh token to each outgoing unit and retaining that exact + request-scoped mapping. An inactive caller creates neither tokens nor pending + transport records and installs no trace-specific timeout/error hooks. Active + callers validate the echoed evidence and record it before parsing bids. A converted or skipped unit therefore cannot shift - another slot's correlation. The response member does not replace or expose + another request unit's server-evidence association. This mapping does not + imply a GPT-cycle join. The response member does not replace or expose the existing orchestrator extension, and the trace projection must not copy that extension's provider names, bidder names, metadata, price, creative IDs, domains, or markup. HTTP/transport failures with no readable response are @@ -933,8 +1025,9 @@ transport outcome, so it removes the marker and leaves evidence `not_observed` rather than inventing a failure. These hooks collect only bounded categories and opaque tokens, never XHR error text or response bodies. -The diagnostic auction token is also attached to the existing GPT opportunity -marker, and the opaque slot token is carried through the corresponding +For SSAT and SPA page-bids, the diagnostic auction token is also attached to +the existing GPT opportunity marker, and the opaque slot token is carried +through the corresponding winning-bid/slot initialization path. The numeric ordinal is never a correlation key. The viewer joins a server slot to a GPT cycle only when one validated `TraceSlotCorrelationV1` exactly matches both tokens and the exported @@ -945,9 +1038,10 @@ timestamps, implicit array position, ad-unit path, or a best-effort heuristic. TSJS retains at most the newest 16 validated server-auction records and 128 correlation sidecars in memory. It increments checked eviction counters for older records; the snapshot adds those counts to the matching truncation fields -and emits `record_evicted` with `partial`. It performs no storage write until +and emits `record_evicted`, with `partial` if server records remain or +`unavailable` if none remain after snapshot truncation. It performs no storage write until the explicit snapshot action. -Requests for which either side of the trace-capture gate is false do not mint +Requests that fail the applicable trace gate do not mint trace tokens, build trace evidence, add response members, emit sidecars, or install auction-evidence listeners. @@ -993,32 +1087,40 @@ failure is handled even when the report is below the application limit. Runtime limits are part of the v1 contract: -| Value | Limit | -| --------------------------------------------- | ------------------------------------------------------------ | -| Container nesting | 8 levels | -| Server auctions | 16 | -| Slot correlations | 128 | -| Provider calls | 16 per server auction | -| Auction slots | 64 per server auction | -| Auction coverage issues | 16 | -| Slots | 64 | -| Request cycles | 10 per slot before total-size truncation | -| Callback issues | 128 | -| Attribution issues | 128 | -| Requested slot sizes | 16 per cycle | -| Ad Manager yield-group or company IDs | 8 of each per cycle | -| Creative-failure enums | 16 per cycle | -| Origin | 255 UTF-8 bytes | -| GPT pathname in trace projection | Exact literal `/[redacted]` | -| Slot element ID and ad-unit path | 512 UTF-8 bytes each | -| Trusted Server auction ID and callback reason | 256 UTF-8 bytes each | -| Diagnostic auction ID and opaque slot ref | 128 UTF-8 bytes each | -| Any other string | 128 UTF-8 bytes | -| Enum | Exact documented value only | -| Identifier, sequence, or counter | Finite safe integer from 0 through `Number.MAX_SAFE_INTEGER` | -| Browser-relative timestamp or duration | Finite number from 0 through `Number.MAX_SAFE_INTEGER` | -| Visibility percentage | Finite number from 0 through 100 | -| Slot dimension | Finite integer from 1 through 100,000 | +| Value | Limit | +| ----------------------------------------------- | ------------------------------------------------------------ | +| Container nesting | 8 levels | +| Server auctions | 16 | +| Slot correlations | 128 | +| Provider calls | 16 per server auction | +| Auction slots | 64 per server auction | +| Auction coverage issues | 16 | +| Slots | 64 | +| Request cycles | 10 per slot before total-size truncation | +| Callback issues | 128 | +| Attribution issues | 128 | +| Requested slot sizes | 16 per cycle | +| Creative-failure enums | 16 per cycle | +| Origin | 255 UTF-8 bytes | +| GPT pathname in trace projection | Exact literal `/[redacted]` | +| Slot element ID and ad-unit path | Forbidden, including callback/attribution issue copies | +| Trusted Server auction ID | Exact diagnostic auction-token shape | +| Callback reason | Exact documented value only | +| Diagnostic auction ID and opaque slot ref | 128 UTF-8 bytes each | +| Any other string | 128 UTF-8 bytes | +| Enum | Exact documented value only | +| Identifier, sequence, or counter | Finite safe integer from 0 through `Number.MAX_SAFE_INTEGER` | +| Browser-relative timestamp or duration | Finite number from 0 through `Number.MAX_SAFE_INTEGER` | +| Visibility percentage | Finite number from 0 through 100 | +| Requested or selected creative dimension | Finite integer from 1 through 100,000 | +| GPT-reported fill dimension (`size`) | Finite integer from 1 through 100,000 | +| Observed CSS box dimension (`observedSlotSize`) | Finite integer from 0 through 100,000 | + +`observedSlotSize` preserves zero dimensions, including `[0, 0]` for a +hidden or collapsed element after a filled GPT render. Zero is a measured box +size, not missing data or evidence of an empty GPT response; do not omit it or +change `isEmpty`. Positive dimensions remain required for requested and +selected creative sizes and GPT-reported fill sizes. Every accepted string must be valid Unicode and must not contain C0/C1 control characters or bidirectional override/isolate controls. This applies to browser @@ -1029,8 +1131,7 @@ invalid source value rather than stringifying it. Server auctions are already bounded by core; the browser rejects an invalid inner model rather than truncating it. The builder retains the newest 16 server auctions in observation order, the newest 128 correlations in recorder emission order, and only the -first documented number of GPT requested sizes, yield-group IDs, company IDs, -and creative failure enums. It records each discard in +first documented number of GPT requested sizes and creative failure enums. It records each discard in `omitted_server_auctions`, `omitted_slot_correlations`, or `omitted_nested_values`; strings are never silently shortened. It then measures the complete compact UTF-8 storage wrapper. If it exceeds 512 KiB, it removes @@ -1180,7 +1281,9 @@ Forbidden data includes: - EC IDs, EIDs, bidder user IDs, provider/bidder/seat names, and consent strings. - Unmasked client IP. -- Query strings and fragments. +- Exact page paths, slot element IDs, ad-unit paths (including issue copies), + query strings, and fragments. +- The entire GPT `adManager` identity object and `previousCreativeId`. - Fastly or internal request identifiers that can join to user-bearing logs. - Internal `AuctionRequest.id`. - Bid requests/responses, bid prices/currency, losing-bid payloads, provider @@ -1233,9 +1336,11 @@ hash and a corresponding CSP change. Validated report strings enter the document through `textContent` or equivalent DOM properties, never `innerHTML`. The JS asset uses `application/javascript; charset=utf-8`; the CSS asset uses -`text/css; charset=utf-8`. Both send `X-Content-Type-Options: nosniff`, -`Cache-Control: public, max-age=31536000, immutable`, and a strong ETag derived -from their build bytes. They accept no dynamic input. `script-src 'self'` is an +`text/css; charset=utf-8`. Both send `X-Content-Type-Options: nosniff` and a strong ETag derived +from their build bytes. They use +`Cache-Control: public, max-age=31536000, immutable` only when no configured +authentication rule covers the asset; otherwise they use `private, no-store` +as specified in section 12.5. They accept no dynamic input. `script-src 'self'` is an origin-level CSP permission, not a path restriction; same-origin script interference remains inside the stated trust limitation. @@ -1249,9 +1354,29 @@ The existing diagnostics private/no-store decision remains a load-bearing gate. Tests must prove that late response-header handlers cannot make traced content publicly cacheable. +### 12.5 Operator authentication policy + +Trace routing preserves the existing `auth.rs` namespace contract: every +matching operator Basic Authentication rule is enforced, including `^/_ts` or +`^/`. Classification may run early, but does not authorize a request. Only +authentication is factored ahead of trace handling; ordinary event, identity, +filter, and auction processing stays outside trace routes. A challenge exposes +no setup context, changes no cookie, and is terminally private/no-store. + +The credential-free mobile journey requires deployment rules that leave the +trace namespace public. Document this beside `trace_page_enabled`; do not +silently narrow an operator rule. On origin requests, authentication precedes +disabled-route `404` and unsupported-method `405` responses as well. Asset +responses covered by an authentication rule must use `private, no-store` +instead of public immutable caching. Previously public cached inert assets may +remain available, but contain no report data and cannot bypass the uncached +shell or actions. + ## 13. Failure handling -- Disabled route: local privacy-safe `404`. +- Configured authentication failure: local private/no-store `401` challenge + before trace handling, including on disabled routes. +- Disabled route after authentication: local privacy-safe `404`. - Unsupported method: local `405`; never publisher fallback. - Rejected activation/end POST: local `403` with no state mutation. - Optional platform fact unavailable: omit the field and continue. @@ -1265,9 +1390,14 @@ publicly cacheable. - Initial-navigation evidence cannot be injected because the body tail is not reached: preserve publisher delivery. Because the browser received no safe marker, display `not_observed` rather than claiming a known server failure. -- Page-bids or `/auction` evidence is absent or rejected by its strict client - validator: parse the ordinary bid response exactly as before, discard the - diagnostic member, and show an unmatched/invalid-evidence coverage category. +- A successful page-bids or `/auction` response omits the optional evidence + member: parse ordinary bids unchanged and add no capture issue. With no other + records or known capture issues, coverage is `not_observed`; absence does not + reset previously collected evidence or failures. +- A supplied evidence member fails strict validation: discard that member, + parse ordinary bids unchanged, and add `evidence_validation_failed`. Coverage + is `partial` when valid records remain or `unavailable` when none remain, + following section 9.3. - TS Console capture failure: fail open for advertising and show incomplete coverage in diagnostics. - Storage unavailable or quota exceeded after a valid bounded report exists: @@ -1306,15 +1436,24 @@ results, never a prerequisite for returning them. console cookie still enables the existing console but never mints trace tokens, builds auction evidence, adds response extensions, or emits correlation sidecars. +- With a valid incoming cookie, `?ts_console=0`, invalid/duplicate directives, + prefetches, bots, and other ineligible navigations suppress all document trace + capture through the effective diagnostics decision. Query enable without a + cookie activates only the console until the next eligible cookie-bearing + reload. Page-bids and `/auction` use the base gate without navigation checks. - Exact reserved-route classification, canonical-path, query, method, encoded path, and fallback behavior. - Exact versioned asset routes are local and contain no dynamic data; lookalike asset paths never reach the publisher origin. - Same-origin POST validation, cross-site/missing signal rejection, cookie - set/clear attributes, fixed 30-minute endpoint activation without request - refresh, and idempotent enable/end behavior. + set/clear attributes, shared `Max-Age=1800` for endpoint and query activation + without ordinary-request refresh, and idempotent enable/end behavior. Cover + enable then query activation, query then enable, repeated explicit activation, + clearing through either surface, and the pre-existing session-cookie caveat. - Enable/end success requires a separate state request to observe the resulting cookie; failed and mismatched verification never displays confirmed state. + Absent, invalid, duplicate, and uninspectable cookies all report inactive, + with no claim that inactive proves the cookie is absent. - Empty-body enforcement rejects positive/invalid lengths, transfer encoding, the first unexpected body byte, and the two-second deadline without an unbounded read. @@ -1330,7 +1469,8 @@ results, never a prerequisite for returning them. API call sites to the exact public source enums without using a browser hint. - The diagnostic auction token is minted before dispatch and remains identical across completed, zero-bid, skipped, failed, and abandoned evidence and the - corresponding browser opportunity marker. It never equals or contains the + corresponding SSAT/SPA browser opportunity marker. API tokens remain + server-evidence identifiers without GPT sidecars. No token equals or contains the internal auction ID or telemetry UUID. - Server-auction projection covers every terminal status/reason mapping, auction-local total duration, provider role/status/duration/count, per-slot @@ -1350,13 +1490,15 @@ results, never a prerequisite for returning them. associated across grouped multi-bidder units, duplicate codes, skipped non-banner units, and mixed accepted/filtered inputs. Numeric ordinals are never used for client correlation. -- Token tests cover canonical UUID-v4 shape, missing Web Crypto, malformed or +- Token tests cover the existing simple auction UUID-v4 shape and hyphenated + slot UUID-v4 shape, exact producer-to-sidecar joins, missing Web Crypto, malformed or duplicate request extensions, the disabled trace gate, and proof that invalid tokens neither fail nor otherwise alter the ordinary auction. - Initial and SPA slot JSON attaches the exact `ext.trusted_server.trace_slot_ref` token that appears in server evidence; - inactive responses omit it. Missing, duplicate, conflicting, malformed, and - evidence-mismatched slot tokens suppress only correlation and produce the + inactive responses omit it. With valid matching auction evidence supplied, + missing, duplicate, conflicting, malformed, and evidence-mismatched slot + tokens suppress only correlation and produce the specified coverage issues without changing slot/bid order or contents. - Active responses remain terminally private/no-store under hostile late header overrides. @@ -1369,6 +1511,11 @@ results, never a prerequisite for returning them. - Axum, Cloudflare, and Spin return the common route/schema with unavailable fields omitted. - Trace-route failures never fall through to publisher origin. +- Broad `^/_ts` and `^/` authentication rules challenge every trace path + (shell/state/actions/assets, disabled routes, and unsupported methods); valid + credentials proceed to trace handling, while `^/_ts/admin` leaves trace + routes public. Challenges expose no context or cookie mutation, and + protected assets remain private/no-store. - GET, HEAD, state-changing POST, and unsupported methods obey the same lifecycle contract across adapters. - Every adapter omits JA4/H2 and rejects control characters or overlong platform @@ -1377,10 +1524,19 @@ results, never a prerequisite for returning them. ### 14.3 JavaScript unit tests - Explicit snapshot only; no continuous `sessionStorage` writes. +- Both TSJS auction callers require the literal `__tsjs_trace_active === true` + before token generation or trace collection. Cover missing/false/non-boolean + flags with GPT diagnostics active: no tokens, mappings, listeners, pending + transport records, sidecars, or handoff action are created. - Compact UTF-8 size measurement; exact outer/nested schema validation; unknown fields; per-string/array/numeric/depth caps; hostile mutation; expiry; future-clock skew; wall-clock rollback; replacement; clearing; and storage exceptions. +- A filled GPT cycle with `observedSlotSize: [0, 0]`, `[0, 250]`, or `[300, 0]` + survives projection, storage validation, rendering, and export unchanged. + Negative, fractional, non-finite, and over-limit observed dimensions fail + validation; requested/selected creative and GPT fill dimensions retain their + positive bounds. - Omission counters use checked arithmetic and reject overflow. - Strict validation of every server-auction enum, token, numeric bound, array bound, nesting level, and unknown property; invalid evidence is discarded @@ -1388,12 +1544,18 @@ results, never a prerequisite for returning them. - Transport envelopes require exactly one of evidence/unavailable reason; projection, transport, validation, eviction, correlation, and external-client coverage states produce the specified complete/partial/unavailable/not-observed - result without treating absence as proof that no auction ran. + result without treating absence as proof that no auction ran. A fixture that + receives valid evidence and then evicts every server record during size + truncation must yield `unavailable` with `record_evicted` and exact omission + counts; partial eviction remains `partial`. +- Successful responses without an optional transport member add no validation + issue; malformed supplied members add `evidence_validation_failed`. Cover + both an empty collector and one with retained evidence or earlier failures. - Direct fetch failures and Prebid `interpretResponse`, `onTimeout`, and `onBidderError` paths consume their pending transport record exactly once; capped/expired records and absent hooks follow the specified `not_observed` behavior without retaining error text or bodies. -- Exact-token correlation joins matching server auctions, GPT opportunities, +- Exact-token correlation joins matching SSAT/SPA server auctions, GPT opportunities, and slot references; unmatched, duplicated, conflicting, missing, and forged tokens stay separate and produce explicit coverage states. No timestamp, index, or ad-unit-path heuristic is used. @@ -1401,6 +1563,11 @@ results, never a prerequisite for returning them. opportunity-to-cycle binding, is capped and evicted deterministically, contains only opaque tokens plus runtime/request numbers, and does not alter `GptDiagnosticsExportV1`. +- Prebid `/auction` evidence remains independent of its `prebid_refresh` GPT + cycle, and direct `requestAds` evidence remains independent of GPT. Neither + path emits a sidecar or introduces `trusted_server_direct`/`competing` + attribution. Both show `correlation_unavailable` without downgrading complete + server capture; supplied API sidecars are rejected by the viewer. - Source presentation distinguishes server-owned SSAT/page-bids/auction API facts from browser-observed publisher refresh, Prebid refresh, competing, and unattributed request paths. No fixture turns intent or a GPT fill into a @@ -1413,7 +1580,12 @@ results, never a prerequisite for returning them. fixture. - Same-tab navigation occurs only after a successful write. - Viewer handles absent optional network facts and every cookie-health state. -- Forbidden fields never enter storage or export fixtures. +- Populate every excluded `adManager` field, `previousCreativeId`, slot + `slotElementId`/`adUnitPath`, and callback/attribution issue `slotElementId` + with distinct sentinel values, including a synthetic secret-bearing path. + Verify none enter trace HTML, storage, copy/share, or direct export; hostile + stored copies containing any excluded property are rejected. Numbered + slot/cycle correlation still joins after redaction. - Download filename and MIME type are deterministic. - Formatted-JSON copy and JSON-file Web Share success, rejection, absence, and download/copy fallback behavior. @@ -1440,7 +1612,7 @@ results, never a prerequisite for returning them. transport fixtures preserve ad initialization while reporting `not_observed`. - `View trace results` navigates in the same tab and renders the captured request context and slot evidence. -- A correlated fixture renders the chain `server auction -> GPT -> creative`, +- A correlated SSAT/SPA fixture renders the chain `server auction -> GPT -> creative`, while unmatched server, client-side refresh, competing, and transport-failure fixtures show honest independent evidence and `Unknown` where appropriate. - Empty, filled, ambiguous, no-candidate, and unattributed slot states remain @@ -1468,8 +1640,10 @@ results, never a prerequisite for returning them. - Inactive publisher traffic has no trace assets, storage access, listeners, or cache-policy change, diagnostic token generation, or trace-auction response extension. -- Disabling `trace_page_enabled` removes every new capture behavior even when a - technical `?ts_console=1` session leaves a valid diagnostics cookie present. +- After disabling `trace_page_enabled`, a fresh publisher load removes every + new capture behavior even when a technical `?ts_console=1` session leaves a + valid diagnostics cookie present. Already-loaded documents cannot be remotely + deactivated, but subsequent server responses contain no trace evidence. ### 14.5 Manual acceptance @@ -1500,12 +1674,17 @@ observed`, `GPT filled/rendered`, and `Unknown` without understanding internal ## 16. Acceptance criteria -1. With the feature disabled, trace-route origin requests return local `404` - and ordinary traffic is unchanged. Previously cached inert versioned assets +1. With the feature disabled, trace-route origin requests that pass configured + authentication return local `404` + and ordinary requests without explicit diagnostics activation are unchanged. + Query activation adopts the shared cookie lifetime in section 5.2. + Previously cached inert versioned assets may remain until cache eviction, but cannot activate tracing or load a shell. -2. A mobile user can enable tracing by opening only `/_ts/trace` and selecting +2. On a deployment whose authentication rules leave trace routes public, a + mobile user can enable tracing by opening only `/_ts/trace` and selecting one prominent action; no target URL, credentials, or trace ID is required, - and a cross-site GET cannot activate tracing. + and a cross-site GET cannot activate tracing. Matching operator auth rules + remain enforced on all trace paths. 3. The setup page accurately explains that the problem must be reproduced after activation. 4. A subsequent real publisher-page reload captures redacted request context, @@ -1552,7 +1731,8 @@ This design is one product flow, but its implementation is split into four independently reviewable plans and preferably four PRs: 1. **Reserved route and privacy foundation:** configuration, shared early-route - classification, same-origin enable/end lifecycle, bounded cookie-health + classification with operator authentication, same-origin enable/end lifecycle, + shared 30-minute endpoint/query cookie policy, bounded cookie-health inspection, base request-context schema, projection of already populated `ClientInfo`/`GeoInfo` fields, response hardening, and adapter parity. Do not add speculative new platform fields in this change. @@ -1593,9 +1773,11 @@ target validation and open-redirect risk, and is unsuitable for a layperson. ### Basic Authentication -Rejected for the mobile end-user workflow. Authentication also would not make -it safe to inject raw secrets into a publisher page containing third-party -JavaScript. +Rejected as a new mandatory prerequisite for the mobile end-user workflow. +Existing operator authentication rules remain enforced; deployment configuration +must leave trace routes public to offer the credential-free journey. +Authentication also would not make it safe to inject raw secrets into a +publisher page containing third-party JavaScript. ### Server-managed trace sessions @@ -1651,6 +1833,9 @@ length, history, logging, referrer, and accidental-sharing risks. - `/auction` evidence requires the TSJS request to reach the same Trusted Server host with the active diagnostics cookie. A custom cross-origin auction endpoint does not inherit this trace session and is shown as unavailable. +- Version one has no GPT correlation for `/auction`: the Prebid caller lacks + a token-bearing GPT binding, and direct `requestAds` renders outside GPT. + Their server evidence is displayed independently with correlation unavailable. - Exact-token correlation can remain unavailable for hidden, unresolved, competing, or independently initiated GPT cycles. The report preserves both sides instead of guessing. From 14c274cbae3e33b338fe6119e98ec4470d4b075d Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 21 Sep 2026 10:50:53 +0530 Subject: [PATCH 5/8] Clarify mobile trace design contracts --- ...-mobile-ad-render-trace-endpoint-design.md | 190 +++++++++++++++--- 1 file changed, 160 insertions(+), 30 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md index 5d86b9092..e3aaca4c6 100644 --- a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -451,7 +451,17 @@ trace_page_enabled = false Rules (route responses below apply after configured authentication): - `trace_page_enabled = true` requires `enabled = true`; invalid combinations - fail configuration validation. + fail configuration validation, including when `enabled` is omitted (defaults + to false). Add a raw-config validation hook in both deploy and runtime + validation, following `validate_js_asset_proxy_config` in + `crates/trusted-server-core/src/config.rs`. It must deserialize and validate + this combination outside the enabled gate: `IntegrationSettings::get_typed` + returns `Ok(None)` before `validate()` for disabled integrations, so a schema + validator alone cannot enforce this rule. Preserve unknown-field rejection + on disabled configurations. This spec explicitly requires invalid enabled + configuration to fail rather than be logged and disabled; the prior rationale + is the HIGH-severity finding in + `2026-03-11-production-readiness-report-design.md`, not a contributor-doc rule. - Operator documentation beside this option states that the public page makes the allowlisted presence/validity of four HttpOnly Trusted Server cookies visible to same-origin JavaScript whenever the feature is enabled. It also @@ -465,7 +475,12 @@ Rules (route responses below apply after configured authentication): parameters do not activate or deactivate tracing and are not reflected into the page or export. - `HEAD /_ts/trace` returns the GET status and headers without a body or state - mutation. + mutation. Explicitly handle `HEAD` on every trace path in all four adapters, + returning a bodyless local 405 on POST-only paths. Fastly's + `publisher_fallback_methods()` includes `HEAD`; a GET registration alone + sends HEAD to the publisher fallback, as pinned by + `dispatch_head_on_named_get_route_falls_through_to_publisher_fallback` in + `crates/trusted-server-adapter-fastly/src/app.rs`. - `GET /_ts/trace/state` returns private/no-store JSON containing only `observed_active: true|false`, determined from whether that request carried exactly one valid diagnostics cookie. `HEAD` returns the same status and @@ -485,13 +500,20 @@ Rules (route responses below apply after configured authentication): cookie using `X-TS-Trace-Action: end`, and returns a small local JSON result. After explicit user confirmation, client JavaScript independently attempts local report deletion and the end POST. Neither result gates the other. -- For both POST paths, absent or exactly-zero `Content-Length` is accepted, - `Transfer-Encoding` is rejected, and the adapter reads at most one byte when - it must verify an absent length. Any body byte or positive/invalid length - returns local `413 Payload Too Large` without draining or processing an - unbounded body. The one-byte read inherits a maximum two-second adapter - request-body deadline; timeout returns local `408 Request Timeout` with no - mutation. +- For both POST paths, accept absent or exactly-zero `Content-Length` only + when the body is empty; reject `Transfer-Encoding`, positive/invalid lengths, + and any actual body bytes with local `413 Payload Too Large` and no mutation. + Precheck headers, then use `Body::into_bytes_bounded(0)` to check emptiness, + following the header-precheck/body-size-check pattern in + `crates/trusted-server-core/src/auction/endpoints.rs`. This is an application + acceptance limit, not a transport read or allocation limit. Pinned EdgeZero + v0.0.8 buffers the Fastly body with blocking `read_to_end` and the Cloudflare + body with `req.bytes().await` before core handling. Spin also buffers the + body; Axum buffers JSON bodies but can stream other content types. `Body` + exposes no read deadline, and Fastly uses `futures::executor::block_on` without a timer. + Therefore v1 promises neither a one-byte transport read, a two-second timeout, + nor a local 408. Transport-level size/deadline protection requires a separate + adapter/upstream change; document the pre-buffering limitation at deployment. - State-changing POSTs require an `Origin` exactly matching the canonical request origin and `Sec-Fetch-Site: same-origin`. Missing, conflicting, malformed, cross-site, or duplicate control values return local `403` without @@ -503,7 +525,16 @@ Rules (route responses below apply after configured authentication): default ports removed before exact comparison. Invalid or multi-valued host, authority, scheme, or origin input fails closed. - Unsupported methods on a shell or state-changing path return a local 405 - Method Not Allowed response with the path-specific `Allow` header. + Method Not Allowed response with the path-specific `Allow` header: + `GET, HEAD` for shell/state/assets and `POST` for enable/end. Classification + must intercept unsupported methods before router dispatch. The router's + `MethodNotAllowed` response has no `Allow` header and bypasses + `FinalizeResponseMiddleware`, as pinned by + `dispatch_unregistered_method_returns_405_at_router_level` in the Fastly + adapter. The trace responder itself supplies `Allow` and all section 12.3 + error-response hardening; it must not rely on router-generated errors. + Fastly entry-point finalization can add ordinary headers later, but does not + establish this trace-specific contract on behalf of the router. - Disabled deployments return a local `404` for the complete trace route set, including assets, and never fall through to the publisher origin. - The `/_ts/trace` namespace is reserved. A trailing slash, extra path segment, @@ -513,6 +544,13 @@ Rules (route responses below apply after configured authentication): The adapter classifies from its canonical parsed path while retaining enough raw-path information to reject ambiguous encodings consistently. +Reuse the bounded percent-decode-to-fixed-point classification pattern from +`deny_admin_diagnostic_fallback` in `crates/trusted-server-core/src/ec/admin.rs` +(`MAX_PERCENT_DECODE_ROUNDS = 4`), moving trace classification before dispatch +rather than relying on fallback. Register and intercept trace paths on Fastly, +Axum, Cloudflare, and Spin from the first implementation PR; existing Fastly-only +`/_ts/*` routes are not a parity precedent. + Every adapter implements the following order: 1. Parse the method, canonical host/origin, path, query, and bounded headers @@ -650,7 +688,11 @@ The classifier uses this deterministic contract: precedence is therefore diagnostic rather than first- or last-value selection. - Per-value limits are 512 bytes for `ts-ec`, 8 KiB for `ts-eids`, and 16 bytes - each for `ts-tester` and `__Host-ts-console`. A single value beyond its limit + each for `ts-tester` and `__Host-ts-console`. Only the 8 KiB EID limit is + inherited (`MAX_EIDS_COOKIE_BYTES` in `ec/prebid_eids.rs`); the other limits + are new trace-inspection bounds. The 512-byte EC limit is an outer guard, + above the exact 71-character format accepted by `is_valid_ec_id` in + `ec/generation.rs`, and does not broaden EC validity. A single value beyond its limit is `present_invalid/oversized`; it does not change the other three states. - One `ts-ec` occurrence is valid only when the canonical EC cookie validator accepts its complete value. @@ -727,6 +769,40 @@ retain their source meaning and require explicit allowlisting; future source fields are not inherited automatically. `trustedServerAuctionId`, when present, must satisfy the diagnostic auction-token contract in section 9.4. +The following is the exhaustive v1 property allowlist derived from the current +interfaces in `crates/trusted-server-js/lib/src/core/types.ts`, after the above +exclusions. Preserve source optionality and validate each source enum against its +explicit current members; do not spread source objects or dynamically inherit +later fields. Section 9.5 supplies numeric, string, array, and depth bounds. + +| Object | Allowed properties | +| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| GPT projection | `schema_version`, `source_schema_version`, `capturedAt`, `page`, `slots`, `callbackIssues`, `attributionIssues`, `coverage`, `metadata` | +| `page` | `origin`, `pathname` (fixed redacted literal) | +| Slot | `runtimeSlotNumber`, `binding`, `currentVisibilityPercentage`, `maximumVisibilityPercentage`, `requests` | +| `binding` | `status`, `reason` (current `GptDiagnosticsBindingReason` enum) | +| Request cycle | `requestNumber`, `requestedAtMs`, `responseAtMs`, `renderAtMs`, `loadAtMs`, `viewableAtMs`, `durations`, `isEmpty`, `requestedSlotSizes`, `size`, `observedSlotSize`, `isBackfill`, `slotContentChanged`, `incompleteSequence`, `responseClass`, `requestPath`, `requestIntentId`, `trustedServerAuctionId`, `opportunityToRequestMs`, `replacedRequestNumber`, `previousRenderToRequestMs`, `creativeChanged`, `loadObservedBeforeRender`, `trustedServerOpportunity`, `trustedServerCreativeRequestAtMs`, `trustedServerCreativeResponseAtMs`, `trustedServerCreativeFailures`, `delivery` | +| `durations` | `requestToResponseMs`, `responseToRenderMs`, `requestToRenderMs`, `renderToLoadMs`, `renderToViewableMs` | +| Callback issue | `kind`, `runtimeSlotNumber`, `timestampMs`, `disposition`, `reason` (only the documented values below) | +| Attribution issue | `reason` (current `GptDiagnosticsAttributionIssueReason` enum), `timestampMs`, `runtimeSlotNumber` | +| `coverage` | Exactly the six current callback-kind keys: `slotRequested`, `slotResponseReceived`, `slotRenderEnded`, `slotOnload`, `impressionViewable`, `slotVisibilityChanged` | +| Each coverage counter | `observed`, `matched`, `unmatched`, `ambiguous` | +| `metadata` | `droppedCallbacks`, `droppedAttributionIssues`, `evictedSlots`, `evictedRequestCycles` | + +Callback `reason` accepts only the current store's six emitted literals: +`invalid_event_order`, `missing_response_before_render`, +`invalid_visibility_percentage`, `evicted_slot`, `no_compatible_request_cycle`, +and `overlapping_request_cycles`. +Although the source interface types it as `string`, the trace validator rejects +all other values rather than copying arbitrary text. + +`requestPath` is the bounded `GptDiagnosticsRequestPath` enum, not a URL or +pathname; include it as browser intent only. `requestIntentId` is a local numeric +sequence, not an external request identifier. Source `version` becomes +`source_schema_version`; no other source properties pass through. Tests must +classify every current source member as copied, transformed, or excluded, and +reject extra keys at every object boundary. + It is deliberately not named or represented as `GptDiagnosticsExportV1`, because redaction, excluded identifiers, and trace-level bounds change the source field semantics. TS Console continues to own the source schema; the trace envelope @@ -812,11 +888,16 @@ telemetry and OpenRTB objects: request ID, or an identifier joinable to user-bearing logs. - Auction tokens are `ts-auc-` followed by a lowercase UUID v4 in the existing producer's 32-hex-digit unhyphenated form (`Uuid::new_v4().simple()`). Slot - tokens are `ts-slot-` followed by a canonical lowercase hyphenated UUID v4, + tokens are new: neither a `ts-slot-` producer nor a TSJS `crypto.randomUUID()` + call exists today. This design introduces tokens that are `ts-slot-` followed by a canonical lowercase hyphenated UUID v4, matching `crypto.randomUUID()`. Both validators enforce UUID version 4 and the RFC variant and reject every other shape. Tokens are compared verbatim, never normalized during validation or correlation; no producer format change - is required for the existing GPT auction opportunity marker. + is required for the existing GPT auction opportunity marker. For the GPT + projection, validation and comparison start from the marker returned by + `normalizedAuctionId` in `gpt_diagnostics/store.ts`, which already trims + whitespace and caps the stored value. Trace validation performs no further + normalization; invalid shapes are not repaired into valid tokens. - `slot_number` is a one-based ordinal over the exact post-conversion `AuctionRequest.slots` sequence observed by orchestration. It is display-only and is never used to map a response back to pre-conversion client input. @@ -838,7 +919,10 @@ telemetry and OpenRTB objects: - `source` is assigned by the server call site: initial document auction is `initial_navigation_ssat`, `/_ts/page-bids` is `spa_page_bids`, and `POST /auction` is `auction_api`. Browser `requestPath` does not determine or - override this value. + override this value. These are intentionally trace-owned public names: + existing `AuctionSource` in `auction/telemetry.rs` uses `initial_navigation`, + `spa_navigation`, and `auction_api`, respectively. Map these explicitly; + do not change telemetry vocabulary or serialize it directly into the report. - `provider_number` is assigned deterministically in provider dispatch order and is stable only within one auction. Provider names, bidder/seat names, and provider metadata are omitted. `returned_bid_count` is a count, not a bid @@ -901,7 +985,10 @@ before an envelope arrives is recorded separately by the browser as `evidence_transport_failed`. Initial-navigation and SPA slot definitions carry their token on the exact slot -object TSJS already consumes: +object TSJS already consumes. This is a two-sided addition: add optional `ext?` +to the TypeScript `AuctionSlot` interface in `core/types.ts` and the matching +nested key to Rust's `build_slot_json` in `publisher.rs`. Neither has this member +today; Rust builds free-form JSON, and TSJS currently retains the slot objects: ```text AuctionSlot.ext.trusted_server.trace_slot_ref: string @@ -989,7 +1076,8 @@ that preserves existing request-path attribution. `ext.trusted_server.trace_slot_ref`. Failed, abandoned, skipped, and zero-bid outcomes still inject their bounded evidence when the publisher document can be delivered. -- **SPA page-bids:** `/_ts/page-bids` adds an optional, namespaced +- **SPA page-bids:** `/_ts/page-bids` and its still-live deprecated alias + `/__ts/page-bids` both add an optional, namespaced `trace_auction` transport envelope beside its existing bid result. TSJS validates and records it and consumes each returned slot's `ext.trusted_server.trace_slot_ref` before triggering ad initialization. Both @@ -1089,7 +1177,7 @@ Runtime limits are part of the v1 contract: | Value | Limit | | ----------------------------------------------- | ------------------------------------------------------------ | -| Container nesting | 8 levels | +| Container nesting | 10 levels | | Server auctions | 16 | | Slot correlations | 128 | | Provider calls | 16 per server auction | @@ -1116,6 +1204,12 @@ Runtime limits are part of the v1 contract: | GPT-reported fill dimension (`size`) | Finite integer from 1 through 100,000 | | Observed CSS box dimension (`observedSlotSize`) | Finite integer from 0 through 100,000 | +The depth cap includes two levels of headroom above the current deepest valid +GPT path: report (1), GPT projection (2), slots array (3), slot (4), requests +array (5), cycle (6), `requestedSlotSizes` array (7), size tuple (8). Headroom +does not permit unknown fields; future schema additions still require explicit +compatibility review. Test the complete current projection and over-depth input. + `observedSlotSize` preserves zero dimensions, including `[0, 0]` for a hidden or collapsed element after a filled GPT render. Zero is a measured box size, not missing data or evidence of an empty GPT response; do not omit it or @@ -1154,7 +1248,7 @@ wrapping or saturating the count. For depth accounting, the `TraceReportV1` object—not its storage wrapper—is level 1; entering either an object or an array increments the level by one; -primitives do not. No accepted report value may enter a ninth container level. +primitives do not. No accepted report value may enter an eleventh container level. The storage wrapper is validated separately as the exact two-field object `{ stored_at_ms, report }`. @@ -1311,7 +1405,10 @@ The HTML shell, enable/end responses, every active diagnostic publisher response, and every dynamic page-bids or `/auction` response carrying trace evidence are terminally `private, no-store`. The fixed versioned JS/CSS assets are the sole exception and may be publicly cached because they contain no -request or report data. HTML and JSON endpoint responses also send: +request or report data. HTML and JSON endpoint responses, including state +results and all local authentication, routing, validation, and disabled-route errors, also send the +following headers. Every such error is `private, no-store`; only successful +fixed-asset responses qualify for the cache exception: - Path-appropriate `Content-Type`: `text/html; charset=utf-8` for the shell and `application/json; charset=utf-8` for enable, end, and state results. @@ -1323,9 +1420,19 @@ usb=()` ```text default-src 'none'; script-src 'self'; style-src 'self'; base-uri 'none'; -object-src 'none'; frame-ancestors 'none'; form-action 'none'; connect-src 'self' +object-src 'none'; frame-ancestors 'none'; form-action 'none'; connect-src 'self'; +img-src data: ``` +The shell supplies a fixed data-URL favicon; `img-src data:` allows it without +an automatic publisher `/favicon.ico` fetch. All styles live in the fixed CSS +asset: toggle classes or the `hidden` attribute, with no inline style attributes, +style blocks, or JavaScript style-property writes. JSON download uses a Blob +object URL assigned directly to an `` followed by a click, then +revokes the URL after the download has started. Do not fetch the blob URL or +embed it in a frame. This fixes the download mechanism without widening +`connect-src` or enabling frames; browser tests exercise it under this exact CSP. + The endpoint makes no third-party requests. Its script and stylesheet are fixed same-origin static assets. Setup-request values are server-rendered as escaped text nodes, and the viewer obtains the report only from browser storage. Active @@ -1352,13 +1459,26 @@ extension; active responses add them only in request-scoped injection or the request-scoped body seam. The existing diagnostics private/no-store decision remains a load-bearing gate. Tests must prove that late response-header handlers cannot make traced content -publicly cacheable. +publicly cacheable. Reuse +`apply_response_headers_with_cache_privacy` in `response_privacy.rs`, which +skips operator cache-header overrides on already uncacheable responses. Fastly +also re-runs privacy guards in `apply_terminal_response_effects` after late EC +and filter effects; retain its +`late_filter_effects_cannot_make_an_assembled_response_public` regression. +`apply_finalize_headers` is terminal on Axum, Cloudflare, and Spin, but not on +Fastly; trace handling must preserve the appropriate terminal protection. ### 12.5 Operator authentication policy -Trace routing preserves the existing `auth.rs` namespace contract: every -matching operator Basic Authentication rule is enforced, including `^/_ts` or -`^/`. Classification may run early, but does not authorize a request. Only +Trace routing preserves existing first-match-wins operator authentication: +`Settings::handler_for_path` selects one handler, and trace handling enforces +that handler's Basic Authentication policy. Rules do not compose; a preceding +narrow rule can shadow `^/_ts` or `^/`. With no matching handler, trace remains +public: the fail-closed unmatched-path backstop in `enforce_basic_auth` applies +only to `/_ts/admin`, not trace. Call the synchronous `enforce_basic_auth` with +settings and the request before serving any trace response. Do not copy the +Fastly `/_ts/debug/ja4` early return, which bypasses `AuthMiddleware`. +Classification may run early, but does not authorize a request. Only authentication is factored ahead of trace handling; ordinary event, identity, filter, and auction processing stays outside trace routes. A challenge exposes no setup context, changes no cookie, and is terminally private/no-store. @@ -1431,7 +1551,8 @@ results, never a prerequisite for returning them. ### 14.1 Core unit tests - Configuration defaults off and rejects trace-page enablement without GPT - diagnostics. + diagnostics, with `enabled` both explicitly false and omitted, on both + deploy and runtime validation paths. - With GPT diagnostics enabled but `trace_page_enabled = false`, a valid console cookie still enables the existing console but never mints trace tokens, builds auction evidence, adds response extensions, or emits @@ -1455,8 +1576,9 @@ results, never a prerequisite for returning them. Absent, invalid, duplicate, and uninspectable cookies all report inactive, with no claim that inactive proves the cookie is absent. - Empty-body enforcement rejects positive/invalid lengths, transfer encoding, - the first unexpected body byte, and the two-second deadline without an - unbounded read. + nonempty bodies even with absent/zero lengths, and verifies no mutation on + rejection. Tests must not claim a transport bound or timeout that the pinned + adapters cannot enforce. - Endpoint skips EC generation/finalization, EID ingestion, auction, telemetry, configured filters, ordinary event context, and origin fetch. - Cookie-health scanner covers multiple header fields; zero, one, and duplicate @@ -1467,6 +1589,8 @@ results, never a prerequisite for returning them. fields. - Server-auction projection maps initial navigation, SPA page-bids, and auction API call sites to the exact public source enums without using a browser hint. + Canonical and legacy page-bids routes produce equivalent gated evidence and + slot extensions, including the TSJS retry against the legacy alias. - The diagnostic auction token is minted before dispatch and remains identical across completed, zero-bid, skipped, failed, and abandoned evidence and the corresponding SSAT/SPA browser opportunity marker. API tokens remain @@ -1510,12 +1634,16 @@ results, never a prerequisite for returning them. sources. - Axum, Cloudflare, and Spin return the common route/schema with unavailable fields omitted. -- Trace-route failures never fall through to publisher origin. +- Trace-route failures never fall through to publisher origin, including HEAD + on each exact path, malformed reserved paths, and arbitrary unsupported + methods intercepted before router dispatch. Assert path-specific `Allow`, + bodyless HEAD errors, and hardening headers on local errors. - Broad `^/_ts` and `^/` authentication rules challenge every trace path (shell/state/actions/assets, disabled routes, and unsupported methods); valid credentials proceed to trace handling, while `^/_ts/admin` leaves trace routes public. Challenges expose no context or cookie mutation, and - protected assets remain private/no-store. + protected assets remain private/no-store. Also test an earlier narrow handler + shadowing a broad rule to pin first-match-wins behavior. - GET, HEAD, state-changing POST, and unsupported methods obey the same lifecycle contract across adapters. - Every adapter omits JA4/H2 and rejects control characters or overlong platform @@ -1735,7 +1863,9 @@ independently reviewable plans and preferably four PRs: shared 30-minute endpoint/query cookie policy, bounded cookie-health inspection, base request-context schema, projection of already populated `ClientInfo`/`GeoInfo` fields, response hardening, and adapter parity. Do not - add speculative new platform fields in this change. + add speculative new platform fields in this change. Include the raw-config + validation hook and method-independent dispatch on all four adapters in + this first PR; Fastly-only route registration does not satisfy the contract. 2. **Live server-auction evidence:** introduce the public diagnostic auction and slot tokens, project `TraceAuctionEvidenceV1` at the live observation boundary, transport it through initial navigation, page-bids, and both From 155d3b09ffb0fe637cf59f6972cf327be5ce7cc9 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Thu, 24 Sep 2026 10:34:10 +0530 Subject: [PATCH 6/8] Clarify trace body validation and browser cleanup --- ...-mobile-ad-render-trace-endpoint-design.md | 45 +++++++++++++++---- 1 file changed, 36 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md index e3aaca4c6..e12d79760 100644 --- a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -503,9 +503,20 @@ Rules (route responses below apply after configured authentication): - For both POST paths, accept absent or exactly-zero `Content-Length` only when the body is empty; reject `Transfer-Encoding`, positive/invalid lengths, and any actual body bytes with local `413 Payload Too Large` and no mutation. - Precheck headers, then use `Body::into_bytes_bounded(0)` to check emptiness, - following the header-precheck/body-size-check pattern in - `crates/trusted-server-core/src/auction/endpoints.rs`. This is an application + Precheck headers, then explicitly match both body variants: accept + `Body::Once` only when its bytes are empty; for `Body::Stream`, consume chunks + until EOF, skipping empty chunks and rejecting the first non-empty chunk + with a local `413`. A stream read error returns local `400 Bad Request` + without mutation; only a clean EOF proves a streamed body empty. Build the + `StatusCode::PAYLOAD_TOO_LARGE` response explicitly, as in the + header-precheck/body-size-check pattern in + `crates/trusted-server-core/src/auction/endpoints.rs`. Do not copy that + handler's `into_bytes().unwrap_or_default()`: `Body::into_bytes` returns + `None` for streams, which would incorrectly treat a non-empty streamed body + as empty. Nor should `Body::into_bytes_bounded(0)` errors be propagated as + the response: overflow is `EdgeError::bad_request`, which maps to `400`, + and `EdgeError` has no `413` variant. All local body-validation errors use + the section 12.3 response hardening. This is an application acceptance limit, not a transport read or allocation limit. Pinned EdgeZero v0.0.8 buffers the Fastly body with blocking `read_to_end` and the Cloudflare body with `req.bytes().await` before core handling. Spin also buffers the @@ -1424,12 +1435,16 @@ object-src 'none'; frame-ancestors 'none'; form-action 'none'; connect-src 'self img-src data: ``` -The shell supplies a fixed data-URL favicon; `img-src data:` allows it without -an automatic publisher `/favicon.ico` fetch. All styles live in the fixed CSS -asset: toggle classes or the `hidden` attribute, with no inline style attributes, +The shell declares a fixed data-URL favicon with `` to suppress +the implicit publisher `/favicon.ico` fallback; `img-src data:` permits that +declared data URL to load despite `default-src 'none'`. All styles live in the +fixed CSS asset: toggle classes or the `hidden` attribute, with no inline style attributes, style blocks, or JavaScript style-property writes. JSON download uses a Blob object URL assigned directly to an `` followed by a click, then -revokes the URL after the download has started. Do not fetch the blob URL or +schedules `URL.revokeObjectURL` with `setTimeout` for 1000 ms after the click. +Never revoke synchronously after `click()`; the delay gives the browser time +to acquire the Blob and is not a download-completion signal. Each download +schedules cleanup of its own object URL. Do not fetch the blob URL or embed it in a frame. This fixes the download mechanism without widening `connect-src` or enabling frames; browser tests exercise it under this exact CSP. @@ -1577,8 +1592,11 @@ results, never a prerequisite for returning them. with no claim that inactive proves the cookie is absent. - Empty-body enforcement rejects positive/invalid lengths, transfer encoding, nonempty bodies even with absent/zero lengths, and verifies no mutation on - rejection. Tests must not claim a transport bound or timeout that the pinned - adapters cannot enforce. + rejection. Cover both `Body::Once` and `Body::Stream`, including clean EOF, + empty chunks before EOF or a non-empty chunk, and stream read errors. Assert + local `413` for body bytes and local `400` for read errors, with section 12.3 + hardening and no cookie mutation. Tests must not claim a transport bound or + timeout that the pinned adapters cannot enforce. - Endpoint skips EC generation/finalization, EID ingestion, auction, telemetry, configured filters, ordinary event context, and origin fetch. - Cookie-health scanner covers multiple header fields; zero, one, and duplicate @@ -1646,6 +1664,8 @@ results, never a prerequisite for returning them. shadowing a broad rule to pin first-match-wins behavior. - GET, HEAD, state-changing POST, and unsupported methods obey the same lifecycle contract across adapters. +- Enable/end POSTs without `Content-Type` accept empty bodies and reject actual + body bytes without cookie mutation, including Axum's streaming path. - Every adapter omits JA4/H2 and rejects control characters or overlong platform strings. @@ -1715,6 +1735,9 @@ results, never a prerequisite for returning them. stored copies containing any excluded property are rejected. Numbered slot/cycle correlation still joins after redaction. - Download filename and MIME type are deterministic. +- Fake timers verify that each download clicks its Blob-backed anchor before + scheduling cleanup, never revokes synchronously, and revokes its own URL + when the 1000 ms timer fires, including repeated downloads. - Formatted-JSON copy and JSON-file Web Share success, rejection, absence, and download/copy fallback behavior. - 320-pixel layout, keyboard navigation, focus handling, and accessible status @@ -1722,6 +1745,10 @@ results, never a prerequisite for returning them. ### 14.4 Browser integration tests +- Under the exact section 12.3 CSP, JSON downloads contain the complete report + with deferred URL cleanup, including the storage-failure recovery path and + clipboard/Web Share fallback. The declared data-URL favicon loads without + an implicit `/favicon.ico` request. - First endpoint GET is read-only and shows setup state; a user-initiated, same-origin enable POST sets the session. - Successful in-page activation adds no history entry, so Back can return to From 4811d98edd316294f7e87cfd909119cf4434c87f Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Thu, 1 Oct 2026 09:47:25 +0530 Subject: [PATCH 7/8] Clarify trace failure and export cleanup contracts --- ...9-01-mobile-ad-render-trace-endpoint-design.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md index e12d79760..bc6387914 100644 --- a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -252,6 +252,15 @@ second GPT attribution engine. The existing recorder emits an exact-token cycle. The server-auction, correlation, and GPT projections are separate sibling contracts in `TraceReportV1`; none is treated as a substitute for another. +The deferred object-URL cleanup in section 12.3 deliberately diverges from TS +Console's existing export, which revokes synchronously in a `finally` block +immediately after `anchor.click()` +(`crates/trusted-server-js/lib/src/integrations/gpt_diagnostics/api.ts`, +reachable through the console's `export()` action). That older path keeps the +truncation risk this design avoids. Version one does not change it; correcting +it belongs to TS Console under #1081, and until then the two download paths +intentionally differ. + ## 6. User experience ### 6.1 First visit: no captured report @@ -1513,6 +1522,12 @@ shell or actions. before trace handling, including on disabled routes. - Disabled route after authentication: local privacy-safe `404`. - Unsupported method: local `405`; never publisher fallback. +- Reserved-namespace path that is a trailing slash, extra segment, unsupported + asset name, repeated separator, or lookalike: local `404`; an encoded + separator or ambiguous dot segment: local `400`. Never publisher fallback. +- Non-empty or unreadable activation/end body: local `413` for any body bytes, + positive/invalid `Content-Length`, or `Transfer-Encoding`, and local `400` + for a stream read error, both with no cookie mutation. - Rejected activation/end POST: local `403` with no state mutation. - Optional platform fact unavailable: omit the field and continue. - Bounded cookie inspection failure: report the contract-defined invalid or From 2028bc6e1e523861e01ac5105d93279eb7ca9114 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Fri, 2 Oct 2026 10:34:03 +0530 Subject: [PATCH 8/8] Clarify mobile trace capture and truncation rules --- ...-mobile-ad-render-trace-endpoint-design.md | 121 +++++++++++++----- 1 file changed, 92 insertions(+), 29 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md index bc6387914..7b220df24 100644 --- a/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md +++ b/docs/superpowers/specs/2026-09-01-mobile-ad-render-trace-endpoint-design.md @@ -287,6 +287,9 @@ is on — cookie observed by server`. The page must not imply that setup-request network facts or an empty auction section describe the affected page. +The setup-request projection reuses the section 9.1 `network` block and section +9.2 `CookieHealth` contracts, including their bounds, without requiring a valid +diagnostics cookie or constructing a `TraceReportV1` envelope. ### 6.2 Active publisher page @@ -838,13 +841,14 @@ a new trace-envelope version. `auction_coverage.capture_status` describes only what reached the browser collector: `not_observed` means no valid server-auction record arrived and no capture issue is known, not that no server auction ran. `partial` requires at -least one retained record plus a projection, transport, validation, or eviction -issue; `unavailable` requires no retained records plus a known projection, -transport, validation, or eviction issue; and -`complete` requires at least one retained record without those capture issues. +least one retained server-auction record plus a projection, transport, +validation, or eviction issue; `unavailable` requires no retained server-auction +records plus a known projection, transport, validation, or eviction issue; and +`complete` requires at least one retained server-auction record without those +capture issues. `correlation_unavailable` and `external_client_side_unobservable` describe interpretation limits and do not change an otherwise complete capture status. -Evicting all received records therefore yields `unavailable` with +Evicting all received server-auction records therefore yields `unavailable` with `record_evicted`, never `not_observed`. Recompute status after both in-memory eviction and snapshot size truncation. The issue array is deduplicated, sorted in enum order, bounded to 16 values, and @@ -1027,8 +1031,12 @@ When valid auction evidence is supplied, TSJS accepts a slot extension only when its canonical token occurs exactly once in both the delivered slot list and the matching auction evidence. A missing, malformed, duplicate, or conflicting token prevents only that sidecar join, -adds `evidence_validation_failed` and `correlation_unavailable`, and does not -drop, reorder, or mutate the ordinary slot or bid. TSJS reads no other extension +adds `correlation_unavailable`, and does not +drop, reorder, or mutate the ordinary slot or bid. It does not add +`evidence_validation_failed`: the evidence member itself validated, so this is +a correlation-layer limit that leaves `capture_status` unchanged per section +9.3. `evidence_validation_failed` is reserved for a supplied evidence member +that fails strict validation. TSJS reads no other extension property. This validation occurs before the slot is handed to the existing GPT initialization path. An absent optional transport member does not trigger missing-token validation; no sidecar is emitted and no capture issue is added. @@ -1124,13 +1132,32 @@ The direct TSJS caller records `evidence_transport_failed` from its existing non-OK, unreadable-JSON, and rejected-`fetch` paths. The Prebid adapter creates one bounded pending transport record after `buildRequests`, keyed by the request's original bid IDs and its normalized unit tokens. `interpretResponse` -consumes it on a readable response; the pinned Prebid `onTimeout` and -`onBidderError` bidder-spec hooks consume it and record +consumes it on a readable response. Add `onTimeout` and `onBidderError` to the +existing bidder-spec object passed to +`pbjs.registerBidAdapter(undefined, ADAPTER_CODE, spec)`; neither hook exists in +the TSJS adapter today. Confirm against the pinned Prebid build that this +registration wraps the spec through `newBidder` and routes both callbacks. +The new hooks consume the pending record and record `evidence_transport_failed` otherwise. Repeated hooks are idempotent. Pending -records are capped at 128 and expire after the configured bid timeout plus five -seconds. Expiry without any supported success/error/timeout hook proves no -transport outcome, so it removes the marker and leaves evidence `not_observed` -rather than inventing a failure. These hooks collect only bounded categories and +records are capped at 128. At record creation, read the effective browser +`pbjs.getConfig('bidderTimeout')` value in milliseconds; use it only when it is +a finite integer from 0 through `2^31 - 1 - 5000`, otherwise use the pinned +Prebid 10.26.0 default of 3000 ms (`DEFAULT_BIDDER_TIMEOUT` in its +`src/config.ts`). Each record +expires after that captured timeout plus 5000 ms, so the delay fits the browser +timer's signed 32-bit limit. Compute the expiry with checked safe-integer +addition to the record's finite nonnegative safe-integer creation timestamp. +If the creation clock or the resulting expiry is invalid or unrepresentable, +decline the trace-only pending record without adding a capture issue or changing +previously collected evidence; that attempt remains `not_observed` and ordinary +bidding continues unchanged. +The TSJS integration's +`merged.timeout`, including the injected `[integrations.prebid].timeout_ms`, +sets this browser `bidderTimeout` when supplied; neither `[auction].timeout_ms` +nor `[auction].auction_timeout_ms` is its source. Expiry without any supported +success/error/timeout hook proves no transport outcome, so it removes the +marker and leaves evidence `not_observed` rather than inventing a failure. +These hooks collect only bounded categories and opaque tokens, never XHR error text or response bodies. For SSAT and SPA page-bids, the diagnostic auction token is also attached to @@ -1145,10 +1172,15 @@ timestamps, implicit array position, ad-unit path, or a best-effort heuristic. TSJS retains at most the newest 16 validated server-auction records and 128 correlation sidecars in memory. It increments checked eviction counters for -older records; the snapshot adds those counts to the matching truncation fields -and emits `record_evicted`, with `partial` if server records remain or -`unavailable` if none remain after snapshot truncation. It performs no storage write until -the explicit snapshot action. +older records; the snapshot adds those counts to the matching truncation fields. +Evicting a server-auction record emits `record_evicted`, with `partial` if +server records remain or +`unavailable` if none remain after snapshot truncation. Evicting a correlation +sidecar increments `omitted_slot_correlations` and adds +`correlation_unavailable` only; it never emits `record_evicted` and never +changes `capture_status`, because server capture is unaffected. This distinction +also applies to sidecars removed during snapshot size truncation. It performs +no storage write until the explicit snapshot action. Requests that fail the applicable trace gate do not mint trace tokens, build trace evidence, add response members, emit sidecars, or install auction-evidence listeners. @@ -1252,15 +1284,22 @@ the complete compact UTF-8 storage wrapper. If it exceeds 512 KiB, it removes the globally oldest GPT request cycles first while retaining the newest cycle for each GPT slot, then the oldest callback issues, then the oldest attribution issues, then the oldest -uncorrelated server auctions, and finally the oldest remaining GPT cycles and -server auctions until the report fits. It records every removal in -`truncation`. A correlated auction and the newest GPT cycle that references it -are retained or removed together once the algorithm reaches correlated server -auctions; every sidecar referencing a removed auction or cycle is removed and -counted. The report must not retain a dangling token while claiming a join. +uncorrelated server auctions, and finally the oldest eligible correlated +server auctions until the report fits. The newest cycle for each GPT slot is a +floor for the whole procedure, not only its first stage: no stage removes a +slot's last remaining cycle. When removing a correlated auction in the final +stage, remove every retained GPT cycle that references it together with the +auction; an auction referenced by any floor cycle is therefore protected from +removal. A correlated auction is eligible for removal only when removing all +its referencing cycles preserves every slot's floor. It records every removal +in `truncation`; every sidecar referencing a removed auction or cycle is removed +and counted. The report must not retain a dangling token while claiming a join. A report that still cannot fit after this bounded procedure fails snapshot -creation. The implementation must include a worst-case fixture proving the -result is bounded. +creation, following the oversized case in section 13. The protected floor and +its correlated auctions can exceed the budget on their own. The implementation +must include a worst-case fixture proving successful reports are bounded, and +one proving the floor-exceeds-budget case fails snapshot creation rather than +emptying the GPT section. All omission counters use checked addition. If any source collection would make a counter exceed `u16::MAX`, projection rejects the source instead of @@ -1548,6 +1587,11 @@ shell or actions. parse ordinary bids unchanged, and add `evidence_validation_failed`. Coverage is `partial` when valid records remain or `unavailable` when none remain, following section 9.3. +- Valid evidence cannot join a missing, malformed, duplicate, or conflicting + slot token, or a correlation sidecar is evicted: add only + `correlation_unavailable`, preserve ordinary slots and bids, and leave server + `capture_status` unchanged. Count evicted sidecars in + `omitted_slot_correlations`. - TS Console capture failure: fail open for advertising and show incomplete coverage in diagnostics. - Storage unavailable or quota exceeded after a valid bounded report exists: @@ -1711,13 +1755,26 @@ results, never a prerequisite for returning them. receives valid evidence and then evicts every server record during size truncation must yield `unavailable` with `record_evicted` and exact omission counts; partial eviction remains `partial`. +- Missing, malformed, duplicate, and conflicting slot extensions with valid + evidence add only `correlation_unavailable` and preserve complete server + capture. Evicting correlation sidecars at the 128-record memory cap or during + snapshot truncation increments `omitted_slot_correlations` exactly and never + adds `record_evicted` or changes server `capture_status`. - Successful responses without an optional transport member add no validation issue; malformed supplied members add `evidence_validation_failed`. Cover both an empty collector and one with retained evidence or earlier failures. - Direct fetch failures and Prebid `interpretResponse`, `onTimeout`, and `onBidderError` paths consume their pending transport record exactly once; capped/expired records and absent hooks follow the specified `not_observed` - behavior without retaining error text or bodies. + behavior without retaining error text or bodies. Exercise the newly added + hooks through the pinned Prebid `registerBidAdapter`/`newBidder` registration + path, not only by calling the spec functions directly. Verify expiry uses the + captured browser `bidderTimeout` plus 5000 ms, including configured values, + the 3000 ms default, missing/invalid-value fallback, the maximum accepted + timeout, and fallback for values that would overflow the browser timer. + Invalid creation clocks and unsafe timestamp addition decline the pending + record without changing bids or inventing a transport failure. Later + configuration changes do not alter an existing record's expiry. - Exact-token correlation joins matching SSAT/SPA server auctions, GPT opportunities, and slot references; unmatched, duplicated, conflicting, missing, and forged tokens stay separate and produce explicit coverage states. No @@ -1740,7 +1797,9 @@ results, never a prerequisite for returning them. - Trace projection replaces the nested GPT pathname with `/[redacted]`, rejects any other stored value, emits `TraceGptDiagnosticsV1`, applies deterministic ordering/truncation, and records exact omission counts in a worst-case 512 KiB - fixture. + fixture. An oversized newest-cycle-per-slot floor, including its protected + correlated auctions, fails snapshot creation without dropping a slot's last + cycle or offering a combined-report export. - Same-tab navigation occurs only after a successful write. - Viewer handles absent optional network facts and every cookie-health state. - Populate every excluded `adManager` field, `previousCreativeId`, slot @@ -1805,8 +1864,12 @@ results, never a prerequisite for returning them. it. - The delivered CSP blocks inline injection, framing, third-party connections, and report-derived executable HTML. -- Immutable asset fixtures prove published v1 bytes never change; changed bytes - require a new URL referenced by the shell. +- A committed digest of each published v1 asset is asserted against the bytes + the build produces, so changing those bytes fails the check. Changed bytes + require a new asset-set URL and its + committed digest, preserving the published URL/digest pairs; a fixture also + asserts the shell references the current asset-set URL. This pins the byte + contract at build time rather than claiming a test can observe future releases. - Inactive publisher traffic has no trace assets, storage access, listeners, or cache-policy change, diagnostic token generation, or trace-auction response extension.