Add a failover gateway over profiles - #54
Closed
jellologic wants to merge 1 commit into
Closed
Conversation
A launcher answers "which provider should this session use?" once, at launch. It cannot answer it again twenty minutes later, when that provider starts returning 529 — which is the failure this addresses. On 2026-09-03 Anthropic returned 529 thirty-three times in about four minutes while a healthy second provider sat configured and idle. `swisscode config proxy --profile work --fallback glm` runs a local gateway that retries, honours retry-after, and then moves to the next profile. It has no configuration of its own. Every route is derived from existing profiles through the same resolution a launch uses (resolveProfileRefs -> buildIntent), so the gateway and the launcher cannot disagree about what a profile means. A second config format would be a second thing to keep true. Failover remaps the model by TIER rather than forwarding the client's string: an opus-tier request reaches the fallback's opus-tier model, because sending claude-opus-5 to z.ai is a 404 in a working request's costume. Properties worth recording: - Requests are forwarded byte for byte. Anthropic signs `thinking` blocks against the bytes it received, so re-encoding a body invalidates them; the gateway re-serializes only when it actually changes the model, on failover. - A credential never crosses routes. A profile authenticated by a login rather than a key forwards the caller's own credential instead of substituting one. Both directions are asserted in tests. - count_tokens is answered locally. Claude Code calls it to decide when to auto-compact and Anthropic rejects it for subscription tokens, so forwarding or 404ing it breaks compaction silently. - A 429 whose body reads as a spend cap is not retried; that one will still be 429 in an hour. - The routing table, the retry policy and the estimator are pure and separately tested. Only server.ts touches a socket — the same split adapters/web makes. Off the launch path by construction: reached solely through the existing dynamic import of config-root, so the closure stays at 42 modules with no node:http and no fetch. test/architecture.test.ts still passes unchanged. The "no proxy, no daemon" claim in README, AGENTS.md and ARCHITECTURE.md is narrowed to what is actually enforced — a LAUNCH leaves nothing running. The gateway is opt-in, foreground-only, and stopped with Ctrl-C. The size budget is raised 150 -> 175 kB. Worth noting in review: the artifact had already drifted to 147.7 kB, 1.5% under the old ceiling, while the comment beside it still claimed ~27% headroom. The gateway adds 4.8 kB packed; the stale headroom is recorded in the script so the next raise starts honest. Signed-off-by: jellologic <31935831+jellologic@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
src/adapters/gateway/— a local gateway that routes across profiles and fails over when a provider is overloaded. Motivated by 2026-09-03, when Anthropic returned 529 thirty-three times in ~4 minutes while a healthy z.ai profile sat idle.Groundwork for #42, #43 and #44 in EPIC #52.
resolveProfileRefs→buildIntent; no second config format.claude-opus-5to a host that has never heard of it.thinkingblocks against the bytes it received, so the body is re-serialized only when the model actually changes.count_tokensanswered locally — Claude Code calls it for auto-compact and Anthropic rejects it for subscription tokens, so forwarding or 404ing it breaks compaction silently.Off the launch path by construction: reached only through the existing dynamic import of
config-root, so the closure stays at 42 modules with nonode:httpand nofetch.test/architecture.test.tspasses unchanged.Note for review, given #35: the
swisscode config proxyentry point added here is scheduled to move to the web UI. The adapter undersrc/adapters/gateway/is the durable part and is what #42/#43 build on.Also raises the size budget 150 → 175 kB. Worth a look: the artifact had already drifted to 147.7 kB — 1.5% under the old ceiling — while the comment beside it still claimed ~27% headroom. The gateway adds 4.8 kB packed. See #36.
844 tests green.