Repository navigation
Make the Axum adapter a supported self-hosted deployment target #1236
Description
Context
The Axum adapter is documented as a local development target only (architecture.md, adapter README). This epic tracks what Trusted Server needs to run Axum as a supported self-hosted deployment: a container image we own, feature parity for the Edge Cookie subsystem, and correct behavior behind a reverse proxy.
EdgeZero is building the framework side under IABTechLab/edgezero#391:
| EdgeZero | Scope | Status |
|---|---|---|
| IABTechLab/edgezero#401 (for #398) | Hardened multi-stage Axum Dockerfile + .dockerignore templates, container smoke |
Open PR |
| IABTechLab/edgezero#275 | Request lifecycle, fallible Hooks::configure / App::build, response completion. Breaking |
Open, draft |
| IABTechLab/edgezero#392 | SIGTERM, bounded drain, readiness states | Open |
| IABTechLab/edgezero#396 | cwd-independent config/KV paths, fail-before-ready, redb single-writer model | Open |
| IABTechLab/edgezero#399 | amd64/arm64 image CI, downstream publishing recipe | Open |
| IABTechLab/edgezero#400 | Docker / Kubernetes / managed-cloud deployment guide | Open |
EdgeZero owns generic packaging and lifecycle. Trusted Server owns its image, config, stores, and the parts of its runtime that only work on Fastly today. #897 (ts up) is the installer that sits on top of this work and depends on it.
Spec first
Write a design spec and get it approved before opening any implementation PR or child issue.
- Put it at
docs/superpowers/specs/YYYY-MM-DD-1236-axum-self-hosting-design.mdand follow the existing specs there. Include the issue link, date, status, problem, goals, non-goals, design, test plan and rollout. - Land the spec in its own PR. Each implementation PR links the approved spec and follows it. If the design changes during implementation, update the spec in the same PR.
- The spec maps every gap below to a design decision, an existing owning issue, or an explicit "unsupported on Axum" statement.
- The spec must settle these open decisions:
- Supported topology for the first release: one replica, or several. Most of the decisions below depend on this one.
- Backend for the Edge Cookie identity graph: embedded redb or an external store with compare-and-set. How it meets the
EcKvStorecontract (strongly consistentkey_exists, generation-conditional writes, prefix list/count), and how operators back it up and restore it. - Backing for the
RateLimiterthat batch-sync and pull sync use: in-process counters per replica, or the shared store. - Client IP: which forwarding headers are trusted and how trusted hops are configured. Geo source: trusted CDN headers, a local MMDB file, or both. Behavior with no geo source, decided with An adapter with no geo module fails closed on Edge Cookie creation and on the auction #1134.
- Config and secret delivery: env var envelope, mounted file, or both (Make Axum consume the local config store written by ts config push #1141, Axum: make production configuration and local storage explicit edgezero#396). Key rotation model: rotate with
tsand restart, or a writable signing store. - Image contract: base images, UID, writable mounts, which inputs are runtime-only, and how the Dockerfile keeps up with the Scaffold hardened Axum application Docker recipes edgezero#401 recipe.
- Health, readiness and shutdown: what readiness checks, and the default drain deadline (Define a uniform cross-adapter health and startup-failure contract #1142, Axum: define a production server lifecycle with bounded graceful shutdown edgezero#392).
- Caching: whether a CDN in front is assumed, and which cache headers Trusted Server emits on Axum.
- Which Fastly-only features stay unsupported on Axum (for example the JA4 debug route), and how the docs say so.
- Sequencing against EdgeZero: what ships on v0.0.8 and what waits for the feat: implement portable outbound HTTP and request lifecycle contracts edgezero#275 upgrade.
Gaps at 666953a0d
| # | Gap | Current behavior | Tracked by |
|---|---|---|---|
| 1 | Edge Cookie lifecycle | Axum never runs EC finalize or pull sync. Fastly does both in main.rs#L297-L345. /_ts/api/v1/identify and /_ts/api/v1/batch-sync are not registered, and /_ts/admin/ec* returns 501 (app.rs#L468). Batch-sync and pull sync also need a per-partner RateLimiter, which only Fastly implements (app.rs#L720) |
New |
| 2 | KV | UnavailableKvStore for runtime KV (platform.rs#L562), kv: None for integration proxies (app.rs#L232). No EcKvStore implementation |
New, #1139 |
| 3 | Geo | AxumPlatformGeo always returns None. Jurisdiction stays Unknown, so the auction consent gate fails closed unless the request carries TCF consent |
#1134, new (geo source) |
| 4 | Client IP | Raw TCP peer address (platform.rs#L537). Behind a load balancer every request has the balancer's IP, which feeds EC generation and geo |
New |
| 5 | Bind address | PORT binds 127.0.0.1 (main.rs#L15). Without PORT, AxumDevServerConfig::default() also binds 127.0.0.1:8787. A container cannot receive traffic from outside |
New |
| 6 | Shutdown | edgezero v0.0.8 waits only for Ctrl-C. No SIGTERM handling and no drain deadline | IABTechLab/edgezero#392 |
| 7 | Health / readiness | /health exists only on the healthy router. A startup failure returns the error on every route |
#1142 |
| 8 | Config handoff | App config arrives as an envelope in a TRUSTED_SERVER_CONFIG_* env var. Secrets come from TRUSTED_SERVER_SECRET_* env vars |
#1141, IABTechLab/edgezero#396 |
| 9 | Key rotation | /_ts/admin/keys/* returns 501 because config and secret stores are read-only env vars (app.rs#L452) |
New (decision) |
| 10 | Logging | RUST_LOG is ignored, always logs at trace |
#1129 |
| 11 | Ignored settings | proxy.certificate_check is accepted and ignored, and proxy.asset_routes is dispatched only on Fastly |
#1132, #1125 |
| 12 | Tester routes | /_ts/set-tester and /_ts/clear-tester exist on Fastly only |
New (parity) |
| 13 | Outbound HTTP | A new reqwest client per request, no connection pooling (platform.rs#L567-L571). The comment records that a pooled client previously regressed a Next.js flow |
New (measure first) |
| 14 | Image | No Dockerfile. JS bundles are embedded at compile time, and trusted-server-js/build.rs runs npm ci only when npm is present |
New |
Proposed work
This is the expected shape, not a decision; the spec confirms or replaces it. After the spec is approved, each item becomes a child issue unless an existing issue already covers it.
- Image and bind (first after the spec). Add a Trusted Server Dockerfile adapted from the Scaffold hardened Axum application Docker recipes edgezero#401 recipe. Trusted Server is not generated by
edgezero new, so we copy the recipe rather than consume the template. No EdgeZero version bump is needed for this item.- Pinned Node 24.12.0 stage:
npm ciand a fresh JS build before Cargo. No Node in the final image. cargo build --release --locked -p trusted-server-adapter-axumwith the toolchain fromrust-toolchain.toml, nothing else.- Debian-slim runtime, numeric non-root UID, read-only root filesystem, exec-form entrypoint, CA certificates.
- No config, secrets, or keys baked into the image.
- Make the bind host configurable (honor
EDGEZERO__ADAPTER__HOST) so the container can listen on0.0.0.0while the dev default stays on loopback. - CI job that builds the image and runs the
scripts/smoke-axum.shassertions against the container.
- Pinned Node 24.12.0 stage:
- Edge Cookie identity graph on Axum. Implement
EcKvStore(kv_backend.rs#L73) for Axum, andRateLimiterfor Axum, then wire EC finalize, pull sync, identify, batch-sync, and admin EC lookup the way the Fastly entry point does. The trait requires a strongly consistentkey_exists, generation-conditional inserts, and prefix list/count.- Embedded redb (EdgeZero's Axum KV) meets this for one replica only.
- Running more than one replica needs an external store with compare-and-set (for example Redis or Postgres).
- The spec picks which ships first. Test against the real backend, since the in-memory double cannot catch consistency bugs.
- Client IP and geo behind a proxy. Add trusted-proxy configuration for
X-Forwarded-For/Forwarded, and choose a geo source: headers from a trusted upstream CDN, a local MMDB file, or both. Coordinate the no-geo behavior with An adapter with no geo module fails closed on Edge Cookie creation and on the auction #1134. - Lifecycle. Upgrade off v0.0.8 once feat: implement portable outbound HTTP and request lifecycle contracts edgezero#275 lands (breaking changes to app assembly and response completion). Adopt the Axum: define a production server lifecycle with bounded graceful shutdown edgezero#392 SIGTERM and drain hooks, and implement Define a uniform cross-adapter health and startup-failure contract #1142 readiness on top of them.
- Config and secrets. Close Make Axum consume the local config store written by ts config push #1141, and adopt read-only config mounts from Axum: make production configuration and local storage explicit edgezero#396. Decide the key rotation model: rotate with
tsand restart, or back the signing stores with something writable. - Parity cleanup. RUST_LOG has no effect on the Axum adapter, which always logs at trace #1129, proxy.certificate_check is accepted and ignored by the Axum adapter #1132, proxy.asset_routes is parsed and validated on every adapter but dispatched only on Fastly #1125, tester routes, and an HTTP client pooling measurement.
- Docs. Replace the "development only" statements in the architecture guide, getting-started guide, and adapter README with a self-hosting guide that lists the remaining differences from Fastly. Note that Axum has no edge cache, so deployments that want caching need a CDN in front. Link Axum: document Docker, Kubernetes, and managed-cloud deployments edgezero#400 for generic Docker and Kubernetes guidance.
Done when
- The design spec is approved and merged under
docs/superpowers/specs/, and every implementation PR links it. - A Trusted Server image built from a committed Dockerfile with
--lockedruns as non-root with a read-only root filesystem, accepts traffic on a configured non-loopback address, and passes the smoke test in CI. - On Axum, Edge Cookie issuance, identity graph persistence across a restart, identify, batch-sync, pull sync, and admin EC lookup work, with tests against the Axum KV backend.
- Behind a reverse proxy, client IP and geo come from configured trusted sources. With no geo source, behavior matches the An adapter with no geo module fails closed on Edge Cookie creation and on the auction #1134 decision.
- SIGTERM drains in-flight requests within a bounded deadline, and readiness follows the Define a uniform cross-adapter health and startup-failure contract #1142 contract.
- Config and secrets come only from runtime inputs, and key rotation has a documented, tested path.
- The docs list Axum as a supported self-hosted target and state its remaining differences from Fastly.
Non-goals
- Publishing images and the
ts upinstaller: Proposed Spec:ts up— turnkey self-hosted Trusted Server installer #897, using the Axum: validate multi-platform containers and provide an application publishing workflow edgezero#399 recipe. - Generic Kubernetes and managed-cloud guidance: Axum: document Docker, Kubernetes, and managed-cloud deployments edgezero#400.
- Multiple writers sharing one embedded redb file.
Affected area
Core (Edge Cookies, GDPR); CI / Tooling; Documentation. Primarily crates/trusted-server-adapter-axum.
Activity