diff --git a/AGENTS.md b/AGENTS.md index 2671fc3..61245b1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,7 +16,12 @@ about it. Read what your agent produced before you submit it. `swisscode` is a **launcher**. It resolves a profile (provider + credential + per-tier models + flags), builds a child environment, and `execve`s the real coding CLI — `claude`, `kilo`, or `opencode` — replacing its own process image. -No proxy, no daemon, nothing left running. +A launch leaves nothing running: no proxy, no daemon, no background process. + +The one exception is opt-in and off the launch path — `swisscode config proxy` +runs a local gateway that fails over between profiles when a provider returns +529. It is foreground-only, reached solely through a dynamic import, and +invariant 1 below still holds: the launch path itself never touches a socket. TypeScript, published as compiled JavaScript. Node >= 22. Four runtime dependencies, all reachable only from the Ink wizard. diff --git a/README.md b/README.md index 894690e..cb7824e 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ CI node current MIT license - no proxy, no daemon + a launch leaves nothing running agent-written PRs welcome

@@ -25,18 +25,24 @@ exactly like `claude`, only pointed at [OpenRouter](https://openrouter.ai), Anthropic-compatible endpoint. Unlike a router/proxy or a desktop GUI, swisscode is a **launcher**: it sets the -right environment and `exec`s the real CLI, so there is **no proxy, no daemon, no -background process** — and it fixes third-party correctness bugs (like the +right environment and `exec`s the real CLI, so **a launch leaves nothing +running** — no proxy, no daemon, no background process between you and your +agent. It also fixes third-party correctness bugs (like the [silent 200K → 1M context downgrade](#extended-context-1m)) that a log-reader or proxy structurally cannot. +There is one thing you can start on purpose: +[`swisscode config proxy`](#gateway) runs a local gateway that fails over +between profiles when a provider is overloaded. It is opt-in, runs in the +foreground, and is not on the launch path. + - **Any provider** — OpenRouter, z.ai/GLM, Kimi, DeepSeek, Qwen, ModelScope, SiliconFlow, or a custom Anthropic-compatible endpoint. - **Local models, no key** — [Ollama](#ollama) speaks the Anthropic Messages API natively, so `swisscode` points Claude Code at `localhost` with no proxy and nothing to sign up for. - **Any agent** — Claude Code (default), [Kilo](https://kilo.ai) or [OpenCode](https://opencode.ai), selectable per profile or per run. - **Named profiles & per-directory bindings** — the right backend per repo, automatically. - **Correctness fixes** — real 1M context (`[1m]`), catalog-driven auto-compaction, gateway compatibility flags. - **A preflight `doctor`** — binary, endpoint, credential, models, real tool-calling probe, and the context window your local server actually loaded. -- **No proxy, no daemon, no GUI** — a single binary that `exec`s the real CLI, so nothing sits between you and your agent. +- **A launch leaves nothing running** — a single binary that `exec`s the real CLI, so nothing sits between you and your agent. The optional [gateway](#gateway) is the one process you start deliberately. It replaces shell aliases like this: @@ -538,6 +544,55 @@ shipped presets are tested against: no `/v1` suffix, no hand-typed `[1m]`, real compatibility flags. Shipped presets stay read-only, and a custom provider cannot shadow one. +## Gateway + +`swisscode config proxy` runs a local gateway that sits in front of several +profiles and fails over between them. It exists for one failure a launcher +structurally cannot fix: a provider that is fine when you start and overloaded +twenty minutes later. + +```sh +swisscode config proxy --profile work --fallback glm +swisscode config proxy --profile work --fallback glm,local --port 8787 + +# then point any agent at it +swisscode --cc-base-url http://127.0.0.1:8787 +``` + +It has **no configuration of its own**. Every route is derived from the profiles +you already keep, through the same resolution a launch uses, so the gateway and +the launcher can never disagree about what "my work profile" means. + +When the primary returns `429` or `529` it retries — honouring `retry-after` — +and then moves to the next profile. Failover **remaps the model by tier**: a +request for your `opus`-tier model reaches the fallback's `opus`-tier model, so +failing over to z.ai asks for `glm-5.2` rather than forwarding `claude-opus-5` +to a host that has never heard of it. + +A few properties worth stating plainly: + +- **Requests are forwarded byte for byte.** Anthropic signs `thinking` blocks + against the exact bytes it received, so a proxy that re-encodes a body + invalidates them. This one only re-serializes when it actually changes the + model, on failover. +- **A credential never crosses routes.** Each route carries its own account's + key, and a profile authenticated by a Claude *login* rather than a key + forwards your own credential untouched instead of substituting one. +- **`/v1/messages/count_tokens` is answered locally.** Claude Code calls it to + decide when to auto-compact, and Anthropic rejects it outright for + subscription tokens, so a gateway that forwards or 404s it breaks compaction + silently. The count is an estimate and is documented as one. +- **Foreground only.** Ctrl-C ends it. There is no daemon and no PID file — the + port bind is the mutex, exactly as it is for the web UI. It binds `127.0.0.1` + only. + +`/health` lists the routes and `/usage` reports per-profile token totals; both +omit credentials. Totals are also printed on exit. + +This is the one part of swisscode that keeps running. It is opt-in, it is not on +the launch path, and a plain `swisscode` launch is unaffected by whether it is +running. + ## Agents The **provider** is which model backend you talk to; the **agent** is which diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 688b09e..dfdc64e 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -22,10 +22,18 @@ argv ─┬─> parse ──> select profile ──> apply overrides ──> bui (process replaced) ``` -There is no proxy, no daemon, no background process, and after `execve` no -swisscode. That single fact drives most of the design: anything that would make -the launch slower, heavier or less auditable is pushed off the launch path or -out of the project. +A launch involves no proxy, no daemon, no background process, and after `execve` +no swisscode. That single fact drives most of the design: anything that would +make the launch slower, heavier or less auditable is pushed off the launch path +or out of the project. + +Note the scope of that claim. It is about the **launch path**, which is the part +`test/architecture.test.ts` actually enforces — a closure rooted at `src/cli.ts` +that may not import `node:http` or call `fetch`, and that is capped at 42 +modules. `swisscode config proxy` starts a long-running local gateway, and is +legal for exactly the reason the web UI is: it is reached only through a dynamic +`import()`, so it never joins that closure. A launch still leaves nothing +running; a gateway is something you start on purpose and stop with Ctrl-C. The tool's job is small; its **failure modes are expensive**. Sending a z.ai token to OpenRouter, silently billing an Anthropic account because a stale @@ -114,6 +122,12 @@ that an adapter meets it. Here the check is real in three places: caught. Gated on the provider id rather than generalised into an "introspect a provider" port method: one example is not enough to know that abstraction's shape, and the second caller is what should define it. +- **`gateway/*`** — `swisscode config proxy`. `server` is the only module that + touches a socket; `table` derives routes from profiles, `dispatch` holds the + retry and failover policy, `tokens` estimates a count locally. The split is + the same one the web feature makes, for the same reason: a policy you can test + without a network is a policy that gets tested. Reached only through a dynamic + import, so it never joins the launch closure. - **`claude-session/*`** — Claude subscription logins, which belong to the agent rather than to us. Split three ways on purpose: `identity` reads *who* an account is (a file read, no credential, no prompt, so listing is free), diff --git a/scripts/size-budget.js b/scripts/size-budget.js index e03ea7e..838921e 100644 --- a/scripts/size-budget.js +++ b/scripts/size-budget.js @@ -19,13 +19,27 @@ import { execFileSync } from 'node:child_process' * which is the same as not having one. Raising it should be a visible line in a * diff with a reason attached. * - * LOWERED from 260 when the artifact fell to ~118 kB — stripping comments from - * the emitted JS, minifying dist/ui.js, and swapping react-dom for preact/compat - * in the browser bundle. A ceiling with more slack beneath it than artifact - * above it is not a budget; it is a number. 150 keeps ~27% headroom, which is - * room for an honest feature and not room for a silent regression. + * LOWERED from 260 to 150 when the artifact fell to ~118 kB — stripping comments + * from the emitted JS, minifying dist/ui.js, and swapping react-dom for + * preact/compat in the browser bundle. A ceiling with more slack beneath it than + * artifact above it is not a budget; it is a number. + * + * RAISED to 175 for `config proxy`, the local gateway (+4.8 kB packed). Two + * things are worth recording about that number, because the 150 it replaced had + * quietly stopped doing its job: + * + * The artifact had drifted from ~118 kB to 147.7 kB — 2.3 kB, or 1.5%, under + * the ceiling — while this comment still claimed ~27% headroom. A budget that + * flush fails on the next honest change, which is exactly the reflexive-raise + * failure the paragraph above warns about; it had become that, unnoticed, + * because nothing re-reads a number that keeps passing. + * + * The gateway is the same trade as the web UI: everyone downloads it, most + * people never start it. 4.8 kB for failover across providers is a trade worth + * making once — and, like the web UI, it is the reason to keep measuring rather + * than an excuse to stop. */ -const BUDGET_KB = 150 +const BUDGET_KB = 175 // `--ignore-scripts` because `prepare` runs the whole build, and this script is // meant to MEASURE the artifact, not rebuild it — in CI the build has already diff --git a/src/adapters/gateway/dispatch.ts b/src/adapters/gateway/dispatch.ts new file mode 100644 index 0000000..4368265 --- /dev/null +++ b/src/adapters/gateway/dispatch.ts @@ -0,0 +1,86 @@ +// Retry and failover policy. Pure arithmetic and predicates — the decisions, +// not the requests. +// +// Split from server.ts on the same principle as the web feature, where routing +// lives in api.ts and only server.ts touches a socket: a policy you can test +// without a network is a policy that gets tested. + +/** + * Statuses worth trying again. + * + * 529 is Anthropic's "overloaded" and is the reason this gateway exists — it + * arrived 33 times in four minutes on 2026-09-03 while a healthy second + * provider sat idle. 429 is included, but see `isTerminalRateLimit`. + */ +const RETRYABLE = Object.freeze([408, 429, 500, 502, 503, 504, 529]) + +export function isRetryable(status: number): boolean { + return RETRYABLE.includes(status) +} + +/** + * A 429 that will still be a 429 in an hour. + * + * A spend cap and a rate limit share a status code and mean opposite things: + * one clears in seconds, the other needs a human with a credit card. Retrying + * the second is pure latency. The signal is weak — no header distinguishes + * them — so this reads the message rather than guessing, and errs toward + * retrying when unsure. + */ +export function isTerminalRateLimit(body: string): boolean { + return /insufficient|balance|quota|credit|recharge|billing|payment/i.test(body) +} + +export type RetryPolicy = { + /** Attempts against one route before moving to the next. */ + attempts: number + baseDelayMs: number + maxDelayMs: number +} + +export const DEFAULT_POLICY: RetryPolicy = Object.freeze({ + attempts: 3, + baseDelayMs: 500, + maxDelayMs: 8000, +}) + +/** + * How long to wait before the next attempt. + * + * A server that tells us when to come back beats any local guess, so + * `retry-after` wins outright — in both its numeric-seconds and HTTP-date + * forms. Otherwise exponential backoff with jitter, so that concurrent + * requests do not all retry on the same tick and rebuild the thundering herd + * the backoff exists to prevent. + */ +export function retryDelay( + attempt: number, + retryAfter: string | null, + policy: RetryPolicy = DEFAULT_POLICY, + random: () => number = Math.random, +): number { + if (retryAfter) { + const seconds = Number(retryAfter) + if (Number.isFinite(seconds) && seconds >= 0) { + return Math.min(seconds * 1000, policy.maxDelayMs) + } + const at = Date.parse(retryAfter) + if (!Number.isNaN(at)) return Math.min(Math.max(at - Date.now(), 0), policy.maxDelayMs) + } + const backoff = policy.baseDelayMs * 2 ** attempt + return Math.min(backoff + random() * policy.baseDelayMs, policy.maxDelayMs) +} + +/** Condense an upstream error body to one line worth logging. */ +export function summarize(body: string): string { + try { + const error = (JSON.parse(body) as { error?: { type?: string; message?: string } }).error + if (error) { + const line = `${error.type ?? 'error'}: ${error.message ?? ''}`.trim() + if (line !== 'error:') return line + } + } catch { + // Not JSON. The raw excerpt below is more useful than a parse complaint. + } + return body.replace(/\s+/g, ' ').trim().slice(0, 300) || '(empty body)' +} diff --git a/src/adapters/gateway/server.ts b/src/adapters/gateway/server.ts new file mode 100644 index 0000000..388e092 --- /dev/null +++ b/src/adapters/gateway/server.ts @@ -0,0 +1,338 @@ +// node:http glue for the gateway. The ONLY module here that knows about +// sockets; the routing table, the retry policy and the token estimate are all +// pure and live beside it. +// +// Off the launch path by construction: test/architecture.test.ts bans node:http +// there by name, so this is reached only through a dynamic import — the same +// treatment the web UI, the wizard and the doctor already get. The launcher +// still leaves nothing running; this is a process the user starts on purpose. + +import { createServer, type IncomingMessage, type ServerResponse } from 'node:http' +import { Readable } from 'node:stream' +import { DEFAULT_POLICY, isRetryable, isTerminalRateLimit, retryDelay, summarize } from './dispatch.ts' +import type { RetryPolicy } from './dispatch.ts' +import { modelFor, tierOf, type Route } from './table.ts' +import { estimateRequestTokens } from './tokens.ts' + +/** + * Hop-by-hop headers, plus the ones the upstream fetch recomputes. Forwarding + * any of these produces a request that describes a connection that no longer + * exists. + */ +const STRIP = new Set([ + 'host', 'content-length', 'connection', 'transfer-encoding', 'keep-alive', + 'upgrade', 'expect', 'proxy-authorization', 'proxy-connection', +]) + +export type GatewayUsage = { + requests: number + inputTokens: number + outputTokens: number + cacheReadTokens: number +} + +export type GatewayServerOptions = { + routes: Route[] + port?: number + policy?: RetryPolicy + out: (line: string) => void + /** Injected so tests do not have to spend real seconds on backoff. */ + sleep?: (ms: number) => Promise +} + +export type RunningGateway = { + url: string + port: number + usage: Map + close: () => Promise +} + +const wait = (ms: number) => new Promise((r) => setTimeout(r, ms)) + +export function startGateway(options: GatewayServerOptions): Promise { + const { routes, out, policy = DEFAULT_POLICY, sleep = wait } = options + const port = options.port ?? 8787 + const usage = new Map() + + const primary = routes[0] + if (!primary) return Promise.reject(new Error('the gateway needs at least one route.')) + + // An arrow bound after the guard rather than a hoisted declaration: a + // function declaration is analysed before `primary` is narrowed, and would + // force a non-null assertion on every use. + const handle = async (req: IncomingMessage, res: ServerResponse): Promise => { + const path = req.url ?? '/' + + if (path === '/health') { + return respondJson(res, 200, { ok: true, routes: routes.map((r) => r.profile) }) + } + if (path === '/usage') { + return respondJson(res, 200, Object.fromEntries(usage)) + } + + // Buffered once: the model decides the route, and every failover attempt + // replays these exact bytes. Byte-exact replay is not an optimisation — + // Anthropic signs `thinking` blocks against the serialization it received, + // so re-encoding a body invalidates them. + const body = await readBody(req) + let parsed: Record | undefined + try { + parsed = JSON.parse(body.toString('utf8') || '{}') as Record + } catch { + // Not JSON. Forwarded as-is to the primary route. + } + const requested = typeof parsed?.model === 'string' ? parsed.model : undefined + const tier = tierOf(primary, requested) + + if (path.startsWith('/v1/messages/count_tokens')) { + const tokens = estimateRequestTokens(parsed) + out(`${(requested ?? '-').padEnd(28)} → local 200 0ms ${path}`) + out(`${' '.repeat(31)}└─ estimated ${tokens} input tokens`) + return respondJson(res, 200, { input_tokens: tokens }) + } + + let last: { status: number; body: string } | undefined + + for (const [index, route] of routes.entries()) { + const model = modelFor(route, tier, requested) + const outcome = await attempt({ route, model, req, path, body, parsed, res, policy, sleep, out, usage }) + + if (outcome.kind === 'sent') return + last = outcome.error + if (index < routes.length - 1) { + out(`${' '.repeat(31)}└─ failing over to ${routes[index + 1]?.profile}`) + } + } + + if (last) return respondRaw(res, last.status, last.body) + respondError(res, 502, 'api_error', 'every route failed.') + } + + const server = createServer((req, res) => { + void handle(req, res).catch((e: unknown) => { + if (res.headersSent) { + res.destroy() + return + } + respondError(res, 502, 'api_error', String(e)) + }) + }) + + return new Promise((resolveServer, rejectServer) => { + server.once('error', rejectServer) + // 127.0.0.1 explicitly, never 0.0.0.0: this process holds credentials for + // every route in the table, and nothing about it should be reachable from + // the network. + server.listen(port, '127.0.0.1', () => { + const bound = (server.address() as { port: number }).port + resolveServer({ + url: `http://127.0.0.1:${bound}`, + port: bound, + usage, + close: () => new Promise((done) => { server.close(() => done()) }), + }) + }) + }) +} + +type Attempt = + | { kind: 'sent' } + | { kind: 'failed'; error: { status: number; body: string } } + +async function attempt(ctx: { + route: Route + model: string | undefined + req: IncomingMessage + path: string + body: Buffer + parsed: Record | undefined + res: ServerResponse + policy: RetryPolicy + sleep: (ms: number) => Promise + out: (line: string) => void + usage: Map +}): Promise { + const { route, model, req, path, res, policy, sleep, out, usage } = ctx + + // Re-serialize only when the model actually changes. On the common path the + // original bytes go out untouched, which is what keeps signed thinking + // blocks valid. + let payload = ctx.body + if (model && ctx.parsed && model !== ctx.parsed.model) { + payload = Buffer.from(JSON.stringify({ ...ctx.parsed, model })) + } + + let last: { status: number; body: string } | undefined + + for (let n = 0; n < policy.attempts; n++) { + const upstream = new AbortController() + // A client that hangs up should not leave the provider generating tokens + // nobody will read. + const abort = () => upstream.abort() + req.once('aborted', abort) + req.once('close', abort) + + const started = Date.now() + let response: Response + try { + response = await fetch(route.baseUrl + path, { + method: req.method ?? 'POST', + headers: upstreamHeaders(req, route), + ...(payload.length > 0 ? { body: new Uint8Array(payload) } : {}), + signal: upstream.signal, + }) + } catch (e) { + req.off('aborted', abort) + req.off('close', abort) + if (upstream.signal.aborted) return { kind: 'sent' } // client left; nothing to send + out(`${(model ?? '-').padEnd(28)} → ${route.profile.padEnd(10)} ERR ${Date.now() - started}ms ${path}`) + out(`${' '.repeat(31)}└─ ${String(e)}`) + last = { status: 502, body: errorBody('api_error', `${route.profile} unreachable: ${String(e)}`) } + if (n < policy.attempts - 1) await sleep(retryDelay(n, null, policy)) + continue + } + + const ms = Date.now() - started + out(`${(model ?? '-').padEnd(28)} → ${route.profile.padEnd(10)} ${String(response.status).padStart(3)} ${ms}ms ${path}`) + + if (response.ok) { + await pipe(response, res, route, usage) + req.off('aborted', abort) + req.off('close', abort) + return { kind: 'sent' } + } + + const text = await response.text() + req.off('aborted', abort) + req.off('close', abort) + out(`${' '.repeat(31)}└─ ${summarize(text)}`) + last = { status: response.status, body: text } + + if (!isRetryable(response.status)) return { kind: 'failed', error: last } + // A spend cap and a rate limit share a status code; only one of them clears + // on its own. Retrying the other just delays the failover. + if (response.status === 429 && isTerminalRateLimit(text)) { + out(`${' '.repeat(31)}└─ not retrying: reads as a balance or quota problem`) + return { kind: 'failed', error: last } + } + if (n < policy.attempts - 1) { + const delay = retryDelay(n, response.headers.get('retry-after'), policy) + out(`${' '.repeat(31)}└─ retrying in ${Math.round(delay)}ms (${n + 2}/${policy.attempts})`) + await sleep(delay) + } + } + + return { kind: 'failed', error: last ?? { status: 502, body: errorBody('api_error', 'no response.') } } +} + +/** + * Build the upstream headers. + * + * A route with no credential is in session mode: the caller holds the login and + * its header is forwarded untouched. Otherwise both credential headers are + * cleared before the route's own is written — a credential for one provider + * must never leave with a request bound for another, which is the same rule the + * launcher enforces at the account level. + */ +function upstreamHeaders(req: IncomingMessage, route: Route): Headers { + const headers = new Headers() + for (const [key, value] of Object.entries(req.headers)) { + if (STRIP.has(key.toLowerCase())) continue + if (typeof value === 'string') headers.set(key, value) + else if (Array.isArray(value)) headers.set(key, value.join(', ')) + } + // fetch decompresses transparently, which leaves a stale content-encoding on + // a body that is already plain; an explicit identity avoids the whole class. + headers.set('accept-encoding', 'identity') + + if (route.credential) { + headers.delete('authorization') + headers.delete('x-api-key') + if (route.header === 'x-api-key') headers.set('x-api-key', route.credential) + else headers.set('authorization', `Bearer ${route.credential}`) + } + return headers +} + +/** Forward a successful response, counting tokens without altering the bytes. */ +async function pipe( + response: Response, + res: ServerResponse, + route: Route, + usage: Map, +): Promise { + const headers: Record = {} + response.headers.forEach((value, key) => { + if (key === 'content-encoding' || key === 'content-length') return + headers[key] = value + }) + res.writeHead(response.status, headers) + + if (!response.body) return void res.end() + + const isStream = (response.headers.get('content-type') ?? '').includes('text/event-stream') + if (!isStream) { + const text = await response.text() + record(usage, route.profile, text) + return void res.end(text) + } + + // Streamed: forward chunks as they arrive and read usage off them in + // passing. Buffering here would defeat streaming, and rewriting the frames + // would risk dropping the `ping` events a client uses to tell a slow + // response from a dead one. + let tail = '' + const source = Readable.fromWeb(response.body as Parameters[0]) + for await (const chunk of source) { + const buf = chunk as Buffer + res.write(buf) + tail = (tail + buf.toString('utf8')).slice(-4096) + } + record(usage, route.profile, tail) + res.end() +} + +/** Pull whatever usage numbers are present, best-effort. */ +function record(usage: Map, profile: string, text: string): void { + const totals = usage.get(profile) ?? { + requests: 0, inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, + } + totals.requests += 1 + for (const match of text.matchAll(/"usage"\s*:\s*(\{[^}]*\})/g)) { + try { + const u = JSON.parse(match[1] ?? '{}') as Record + if (typeof u.input_tokens === 'number') totals.inputTokens += u.input_tokens + if (typeof u.output_tokens === 'number') totals.outputTokens += u.output_tokens + if (typeof u.cache_read_input_tokens === 'number') totals.cacheReadTokens += u.cache_read_input_tokens + } catch { + // Usage accounting must never break a response that is already flowing. + } + } + usage.set(profile, totals) +} + +function readBody(req: IncomingMessage): Promise { + return new Promise((resolveBody, rejectBody) => { + const chunks: Buffer[] = [] + req.on('data', (c: Buffer) => chunks.push(c)) + req.on('end', () => resolveBody(Buffer.concat(chunks))) + req.on('error', rejectBody) + }) +} + +function errorBody(type: string, message: string): string { + return JSON.stringify({ type: 'error', error: { type, message } }) +} + +function respondJson(res: ServerResponse, status: number, value: unknown): void { + respondRaw(res, status, JSON.stringify(value)) +} + +function respondError(res: ServerResponse, status: number, type: string, message: string): void { + respondRaw(res, status, errorBody(type, message)) +} + +function respondRaw(res: ServerResponse, status: number, body: string): void { + res.writeHead(status, { 'content-type': 'application/json' }) + res.end(body) +} diff --git a/src/adapters/gateway/table.ts b/src/adapters/gateway/table.ts new file mode 100644 index 0000000..633ec57 --- /dev/null +++ b/src/adapters/gateway/table.ts @@ -0,0 +1,114 @@ +// Turning profiles into a routing table. +// +// Pure: no I/O, no sockets. The gateway's whole configuration story is that it +// has none — every row here is derived from the profiles the user already +// maintains, through the same resolution the launcher uses. A second config +// format would be a second thing to keep true. + +import { resolveProfileRefs } from '../../core/resolve.ts' +import { buildIntent } from '../../core/intent.ts' +import { TIERS } from '../../core/tiers.ts' +import type { Tier, TierRecord } from '../../ports/provider.ts' +import type { State } from '../../ports/config-store.ts' +import type { ProviderRegistryPort } from '../../ports/provider.ts' +import type { EnvMap } from '../../ports/process.ts' + +/** Where the Anthropic API lives when a descriptor declines to say. */ +const ANTHROPIC_DEFAULT_BASE = 'https://api.anthropic.com' + +export type Route = { + /** Profile name, used in logs and errors. */ + profile: string + provider: string + baseUrl: string + /** + * Empty means SESSION mode: the account authenticates with a login rather + * than a key, so the caller's own credential header is forwarded untouched + * and the gateway never substitutes one. + */ + credential: string + /** Which header carries the credential, when there is one. */ + header: 'x-api-key' | 'authorization' + models: TierRecord +} + +export type TableResult = + | { ok: true; routes: Route[] } + | { ok: false; reason: string } + +/** + * Build the ordered chain: primary first, then each fallback. + * + * `cursor` is deliberately not threaded through. `resolveProfileRefs` advances + * a round-robin cursor as a side effect of resolving, so passing one here would + * rotate every account each time the table is built — once at startup, and + * again on any future rebuild — silently changing which account a launch would + * have used. + */ +export function buildTable( + state: State, + registry: ProviderRegistryPort, + names: string[], + ambientEnv: EnvMap = {}, +): TableResult { + if (names.length === 0) return { ok: false, reason: 'no profile named.' } + + const routes: Route[] = [] + for (const name of names) { + const resolution = resolveProfileRefs(state, name) + if (!resolution.ok) return { ok: false, reason: resolution.reason } + + const descriptor = registry.byId(resolution.resolved.provider) + if (!descriptor) { + return { + ok: false, + reason: `profile "${name}" uses provider "${resolution.resolved.provider}", which is not registered.`, + } + } + + const intent = buildIntent(resolution.resolved, descriptor, ambientEnv) + + // A null baseUrl means "the provider's own default endpoint", which for + // Anthropic is the only case that reaches here — every other descriptor + // states its host. The launcher expresses this by leaving the variable + // unset; a proxy has to name the host it will actually dial. + const baseUrl = intent.baseUrl ?? ANTHROPIC_DEFAULT_BASE + + routes.push({ + profile: name, + provider: resolution.resolved.provider, + credential: intent.credential, + baseUrl: baseUrl.replace(/\/$/, ''), + header: descriptor.credentialEnv === 'ANTHROPIC_API_KEY' ? 'x-api-key' : 'authorization', + models: intent.models, + }) + } + + return { ok: true, routes } +} + +/** + * Which tier an incoming model id belongs to, according to a route's own map. + * + * The gateway needs this to fail over honestly: a request for the primary's + * opus-tier model should reach the fallback's opus-tier model, not the literal + * string the client sent. Sending `claude-opus-5` to z.ai is a 404 wearing a + * working request's clothes. + */ +export function tierOf(route: Route, model: string | undefined): Tier | null { + if (!model) return null + for (const tier of TIERS) { + if (route.models[tier] === model) return tier + } + return null +} + +/** + * The model to request from `route`, given what the client asked the primary + * for. Falls back to the client's own string when no tier matches — an unknown + * model is forwarded verbatim rather than silently rewritten to something else. + */ +export function modelFor(route: Route, tier: Tier | null, requested: string | undefined): string | undefined { + if (tier === null) return requested + return route.models[tier] ?? requested +} diff --git a/src/adapters/gateway/tokens.ts b/src/adapters/gateway/tokens.ts new file mode 100644 index 0000000..4312d76 --- /dev/null +++ b/src/adapters/gateway/tokens.ts @@ -0,0 +1,87 @@ +// Local estimation for /v1/messages/count_tokens. +// +// Claude Code calls this endpoint to decide when to auto-compact. Proxies that +// 404 it break compaction silently — the session simply runs into the context +// limit later and errors. Anthropic also rejects the endpoint outright for +// subscription tokens ("jwt auth is not yet supported on count_tokens"), so +// forwarding is not always an option either. Answering locally is. +// +// This is an ESTIMATE and the numbers should not be presented as anything +// else: Anthropic publishes no client-side tokenizer, `tiktoken` is not valid +// for Claude, and tokenizers differ across model generations. + +/** Characters per token for English prose mixed with code. */ +const CHARS_PER_TOKEN = 3.5 +/** Per-message framing: role markers and delimiters. */ +const PER_MESSAGE_OVERHEAD = 4 +/** + * Flat cost for an image. + * + * Real cost depends on dimensions we cannot read from a base64 blob without + * decoding it. Over-estimating compacts slightly early; under-estimating lets + * a request grow until the provider rejects it, so the bias is deliberate. + */ +const IMAGE_TOKENS = 1600 + +export function estimateTokens(text: string): number { + if (!text) return 0 + return Math.ceil(text.length / CHARS_PER_TOKEN) +} + +type Block = { type?: string; text?: string; thinking?: string; name?: string; input?: unknown; content?: unknown } + +export function estimateRequestTokens(body: unknown): number { + if (!body || typeof body !== 'object') return 0 + const req = body as { system?: unknown; messages?: unknown; tools?: unknown } + let total = 0 + + if (typeof req.system === 'string') total += estimateTokens(req.system) + else if (Array.isArray(req.system)) total += estimateBlocks(req.system) + + if (Array.isArray(req.messages)) { + for (const raw of req.messages) { + total += PER_MESSAGE_OVERHEAD + const message = raw as { content?: unknown } + if (typeof message?.content === 'string') total += estimateTokens(message.content) + else if (Array.isArray(message?.content)) total += estimateBlocks(message.content) + } + } + + if (Array.isArray(req.tools)) { + for (const raw of req.tools) { + const tool = raw as { name?: string; description?: string; input_schema?: unknown } + total += estimateTokens(tool?.name ?? '') + estimateTokens(tool?.description ?? '') + if (tool?.input_schema) total += estimateTokens(JSON.stringify(tool.input_schema)) + } + } + + return total +} + +function estimateBlocks(blocks: unknown[]): number { + let total = 0 + for (const raw of blocks) { + const block = raw as Block + switch (block?.type) { + case 'text': + total += estimateTokens(block.text ?? '') + break + case 'thinking': + total += estimateTokens(block.thinking ?? '') + break + case 'tool_use': + total += estimateTokens(block.name ?? '') + estimateTokens(JSON.stringify(block.input ?? {})) + break + case 'tool_result': + if (typeof block.content === 'string') total += estimateTokens(block.content) + else if (Array.isArray(block.content)) total += estimateBlocks(block.content) + break + case 'image': + total += IMAGE_TOKENS + break + default: + if (typeof block?.text === 'string') total += estimateTokens(block.text) + } + } + return total +} diff --git a/src/composition/config-root.ts b/src/composition/config-root.ts index 70683ca..1d21b4c 100644 --- a/src/composition/config-root.ts +++ b/src/composition/config-root.ts @@ -93,6 +93,7 @@ export type RunConfigCommandOptions = { const SUBCOMMANDS = Object.freeze([ 'list', 'default', 'agent', 'rm', 'use', 'bind', 'unbind', 'bindings', 'doctor', 'help', 'accounts', + 'proxy', 'setups', // v3's name for `setups`. Kept dispatchable so an install that scripted it // does not break on upgrade; undocumented in USAGE, and it says so once when @@ -140,6 +141,11 @@ const USAGE = `swisscode config — manage profiles and directory bindings swisscode config web [--port ] configure swisscode from a browser [--no-open] + swisscode config proxy run a local gateway that fails over between + [--profile ] profiles when a provider is overloaded, then + [--fallback ] point an agent at it with + [--port ] --cc-base-url http://127.0.0.1:8787 + swisscode config doctor [--json] check binary, endpoint, credential, models, tool calling, env conflicts, permissions [--offline] skip every network probe @@ -215,6 +221,8 @@ export async function runConfigCommand({ return doctorCommand({ deps, args: rest, out, err }) case 'web': return webCommand({ deps, args: rest, out, err }) + case 'proxy': + return proxyCommand({ deps, args: rest, out, err }) case 'upgrade': return upgradeCommand({ deps, args: rest, out, err }) case 'accounts': @@ -865,6 +873,100 @@ async function webCommand({ } } +/** + * `swisscode config proxy` — a local gateway in front of several profiles. + * + * The launcher answers "which provider should this session use?" once, at + * launch. This answers it per request, which is a different question and the + * reason it is a separate process rather than part of a launch: when a + * provider returns 529 mid-session, only something still running can react. + * + * Foreground by design, like `config web`. The promise settles when the server + * closes, so the command holds the terminal and Ctrl-C ends it — there is no + * daemon to leave behind and no PID file to go stale. + */ +async function proxyCommand({ + deps, + args, + out, + err, +}: { + deps: LaunchDeps + args: string[] + out: Emit + err: Emit +}): Promise { + const portFlag = args.indexOf('--port') + let port: number | undefined + if (portFlag !== -1) { + const raw = args[portFlag + 1] + const parsed = Number(raw) + if (!raw || !Number.isInteger(parsed) || parsed < 0 || parsed > 65535) { + err(`swisscode: --port needs a number between 0 and 65535; got "${raw ?? ''}".`) + return 2 + } + port = parsed + } + + const profiles: string[] = [] + const profileFlag = args.indexOf('--profile') + if (profileFlag !== -1) { + const name = args[profileFlag + 1] + if (!name || name.startsWith('-')) { + err('swisscode: --profile needs a profile name.') + return 2 + } + profiles.push(name) + } + const fallbackFlag = args.indexOf('--fallback') + if (fallbackFlag !== -1) { + const raw = args[fallbackFlag + 1] + if (!raw || raw.startsWith('-')) { + err('swisscode: --fallback needs one or more comma-separated profile names.') + return 2 + } + for (const name of raw.split(',').map((n) => n.trim()).filter(Boolean)) profiles.push(name) + } + + const { runProxy, describeRoutes } = await import('./gateway-root.ts') + try { + const started = await runProxy({ deps, profiles, ...(port === undefined ? {} : { port }), out }) + if (!started.ok) { + err(`swisscode: ${started.reason}`) + return 2 + } + + out(`swisscode: gateway on ${started.server.url}`) + for (const line of describeRoutes(started.routes)) out(line) + out('') + out(` point an agent at it: swisscode --cc-base-url ${started.server.url}`) + out('') + + await new Promise((resolve) => { + const stop = () => { + void started.server.close().then(resolve) + } + process.once('SIGINT', stop) + process.once('SIGTERM', stop) + }) + + // Printed on the way out rather than per request: a running gateway should + // not narrate, and the totals are only interesting once. + const totals = [...started.server.usage.entries()] + if (totals.length > 0) { + out('') + for (const [profile, u] of totals) { + out(` ${profile.padEnd(16)} ${String(u.requests).padStart(4)} req in ${u.inputTokens.toLocaleString()} out ${u.outputTokens.toLocaleString()}`) + } + } + return 0 + } catch (e) { + const message = (e as { message?: string }).message ?? 'could not start the gateway' + err(`swisscode: ${/EADDRINUSE/.test(message) ? `port ${port ?? 8787} is already in use — another gateway may be running.` : message}`) + return 2 + } +} + /** * `swisscode config accounts [login …]`. * diff --git a/src/composition/gateway-root.ts b/src/composition/gateway-root.ts new file mode 100644 index 0000000..e832c82 --- /dev/null +++ b/src/composition/gateway-root.ts @@ -0,0 +1,67 @@ +// Composition root for `swisscode config proxy`. +// +// LAZY, like the web UI, the wizard and the doctor: reached only through a +// dynamic import, so the launch path's static closure never grows to carry an +// HTTP server. test/architecture.test.ts bans node:http there by name, and the +// launch path is at its module ceiling besides. +// +// The gateway is the one thing swisscode runs that outlives a command, and it +// is deliberately foreground-only. A daemon would need a PID file, and a PID +// file has to reimplement — badly — what the OS already does when a process +// dies. The port bind is the mutex, exactly as it is for `config web`. + +import { buildTable, type Route } from '../adapters/gateway/table.ts' +import { startGateway, type RunningGateway } from '../adapters/gateway/server.ts' +import { withCustomProviders } from '../adapters/providers/composite.ts' +import type { LaunchDeps } from './launch-root.ts' + +export type RunProxyOptions = { + deps: LaunchDeps + /** Primary first, then fallbacks in order. */ + profiles: string[] + port?: number + out: (line: string) => void +} + +export type ProxyResult = + | { ok: true; server: RunningGateway; routes: Route[] } + | { ok: false; reason: string } + +export async function runProxy({ + deps, + profiles, + port, + out, +}: RunProxyOptions): Promise { + const { state } = deps.store.load() + const registry = withCustomProviders(deps.registry, state) + + // Default to the profile a bare launch would have used, so the gateway and + // the launcher agree about "my current setup" without the user restating it. + const names = profiles.length > 0 ? profiles : state.defaultProfile ? [state.defaultProfile] : [] + if (names.length === 0) { + return { ok: false, reason: 'no profile given and no default profile set.' } + } + + const table = buildTable(state, registry, names, deps.proc.env()) + if (!table.ok) return { ok: false, reason: table.reason } + + const server = await startGateway({ + routes: table.routes, + ...(port === undefined ? {} : { port }), + out, + }) + + return { ok: true, server, routes: table.routes } +} + +/** One line per route, for the banner. Credentials are never printed. */ +export function describeRoutes(routes: Route[]): string[] { + return routes.map((route, index) => { + const role = index === 0 ? 'primary ' : 'fallback' + // Says whose credential travels, which is the part worth being unambiguous + // about: the account's own, or the one the caller arrived with. + const auth = route.credential ? "the account's key" : 'your own login' + return ` ${role} ${route.profile.padEnd(16)} ${route.provider.padEnd(12)} ${route.baseUrl} (sends ${auth})` + }) +} diff --git a/test/adapters/gateway/dispatch.test.ts b/test/adapters/gateway/dispatch.test.ts new file mode 100644 index 0000000..1cadba8 --- /dev/null +++ b/test/adapters/gateway/dispatch.test.ts @@ -0,0 +1,62 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { + DEFAULT_POLICY, isRetryable, isTerminalRateLimit, retryDelay, summarize, +} from '../../../src/adapters/gateway/dispatch.ts' + +test('529 overloaded is retryable — the status this gateway exists for', () => { + assert.equal(isRetryable(529), true) + assert.equal(isRetryable(429), true) + assert.equal(isRetryable(503), true) +}) + +test('a bad request is not retried', () => { + assert.equal(isRetryable(400), false) + assert.equal(isRetryable(401), false) + assert.equal(isRetryable(404), false) +}) + +test('a balance or quota message reads as terminal, not as a rate limit', () => { + assert.equal(isTerminalRateLimit('Insufficient balance or no resource package. Please recharge.'), true) + assert.equal(isTerminalRateLimit('quota exceeded for this month'), true) + assert.equal(isTerminalRateLimit('rate limit exceeded, slow down'), false) +}) + +test('a numeric retry-after wins over local backoff', () => { + assert.equal(retryDelay(0, '2', DEFAULT_POLICY), 2000) +}) + +test('an HTTP-date retry-after is honoured', () => { + const when = new Date(Date.now() + 3000).toUTCString() + const delay = retryDelay(0, when, DEFAULT_POLICY) + assert.ok(delay > 1500 && delay <= 3000, `expected ~3000, got ${delay}`) +}) + +test('retry-after is clamped to the ceiling', () => { + assert.equal(retryDelay(0, '9999', DEFAULT_POLICY), DEFAULT_POLICY.maxDelayMs) +}) + +test('backoff grows exponentially and stays under the ceiling', () => { + const noJitter = () => 0 + assert.equal(retryDelay(0, null, DEFAULT_POLICY, noJitter), 500) + assert.equal(retryDelay(1, null, DEFAULT_POLICY, noJitter), 1000) + assert.equal(retryDelay(2, null, DEFAULT_POLICY, noJitter), 2000) + assert.equal(retryDelay(9, null, DEFAULT_POLICY, noJitter), DEFAULT_POLICY.maxDelayMs) +}) + +test('jitter separates retries that would otherwise land on the same tick', () => { + assert.equal(retryDelay(0, null, DEFAULT_POLICY, () => 0), 500) + assert.equal(retryDelay(0, null, DEFAULT_POLICY, () => 1), 1000) +}) + +test('an error body is summarized to its type and message', () => { + assert.equal( + summarize('{"type":"error","error":{"type":"overloaded_error","message":"Overloaded"}}'), + 'overloaded_error: Overloaded', + ) +}) + +test('a non-JSON or empty body still yields something worth logging', () => { + assert.equal(summarize('502 Bad Gateway'), '502 Bad Gateway') + assert.equal(summarize(''), '(empty body)') +}) diff --git a/test/adapters/gateway/server.test.ts b/test/adapters/gateway/server.test.ts new file mode 100644 index 0000000..257b02c --- /dev/null +++ b/test/adapters/gateway/server.test.ts @@ -0,0 +1,278 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { createServer, type Server } from 'node:http' +import { startGateway } from '../../../src/adapters/gateway/server.ts' +import type { Route } from '../../../src/adapters/gateway/table.ts' + +/** A stub upstream that replays a scripted sequence and records what it saw. */ +type Reply = { status: number; body: string; headers?: Record } +type Upstream = { + url: string + calls: Array<{ path: string; body: string; auth: string | undefined; key: string | undefined }> + close: () => Promise +} + +async function upstream(script: Reply[]): Promise { + const calls: Upstream['calls'] = [] + let n = 0 + const server: Server = createServer((req, res) => { + const chunks: Buffer[] = [] + req.on('data', (c: Buffer) => chunks.push(c)) + req.on('end', () => { + calls.push({ + path: req.url ?? '', + body: Buffer.concat(chunks).toString('utf8'), + auth: req.headers.authorization, + key: req.headers['x-api-key'] as string | undefined, + }) + const reply = script[Math.min(n, script.length - 1)] ?? { status: 500, body: '{}' } + n++ + res.writeHead(reply.status, reply.headers ?? { 'content-type': 'application/json' }) + res.end(reply.body) + }) + }) + await new Promise((r) => server.listen(0, '127.0.0.1', r)) + const port = (server.address() as { port: number }).port + return { + url: `http://127.0.0.1:${port}`, + calls, + close: () => new Promise((done) => { server.close(() => done()) }), + } +} + +const route = (over: Partial & { baseUrl: string }): Route => ({ + profile: 'p', provider: 'anthropic', credential: '', header: 'authorization', + models: { opus: undefined, sonnet: undefined, haiku: undefined, fable: undefined }, + ...over, +}) + +const ok = (body: unknown): Reply => ({ status: 200, body: JSON.stringify(body) }) +const overloaded = (headers?: Record): Reply => ({ + status: 529, + body: '{"type":"error","error":{"type":"overloaded_error","message":"Overloaded"}}', + ...(headers ? { headers: { 'content-type': 'application/json', ...headers } } : {}), +}) + +async function withGateway( + routes: Route[], + run: (url: string, usage: Map) => Promise, +): Promise { + const gateway = await startGateway({ + routes, port: 0, out: () => {}, sleep: async () => {}, + }) + try { + return await run(gateway.url, gateway.usage) + } finally { + await gateway.close() + } +} + +const post = (url: string, body: unknown) => + fetch(`${url}/v1/messages`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(body), + }) + +test('a healthy upstream is forwarded and its response returned unchanged', async () => { + const up = await upstream([ok({ id: 'msg_1', usage: { input_tokens: 5, output_tokens: 2 } })]) + try { + await withGateway([route({ baseUrl: up.url })], async (url, usage) => { + const res = await post(url, { model: 'claude-opus-5' }) + assert.equal(res.status, 200) + assert.deepEqual(((await res.json()) as { id: string }).id, 'msg_1') + assert.equal(usage.get('p')?.inputTokens, 5) + }) + assert.equal(up.calls[0]?.path, '/v1/messages') + } finally { + await up.close() + } +}) + +test('529 is retried against the same route before giving up on it', async () => { + const up = await upstream([overloaded(), overloaded(), ok({ id: 'msg_ok' })]) + try { + await withGateway([route({ baseUrl: up.url })], async (url) => { + const res = await post(url, { model: 'claude-opus-5' }) + assert.equal(res.status, 200) + }) + assert.equal(up.calls.length, 3) + } finally { + await up.close() + } +}) + +test('once a route is exhausted the next profile serves the request', async () => { + const primary = await upstream([overloaded()]) + const backup = await upstream([ok({ id: 'from_backup' })]) + try { + await withGateway( + [route({ profile: 'work', baseUrl: primary.url }), route({ profile: 'glm', baseUrl: backup.url })], + async (url) => { + const res = await post(url, { model: 'claude-opus-5' }) + assert.equal(((await res.json()) as { id: string }).id, 'from_backup') + }, + ) + assert.equal(primary.calls.length, 3, 'primary should exhaust its retries first') + assert.equal(backup.calls.length, 1) + } finally { + await primary.close() + await backup.close() + } +}) + +test('failover rewrites the model to the fallback profile’s tier equivalent', async () => { + const primary = await upstream([overloaded()]) + const backup = await upstream([ok({ id: 'ok' })]) + try { + await withGateway( + [ + route({ + profile: 'work', baseUrl: primary.url, + models: { opus: 'claude-opus-5', sonnet: undefined, haiku: undefined, fable: undefined }, + }), + route({ + profile: 'glm', baseUrl: backup.url, + models: { opus: 'glm-5.2', sonnet: undefined, haiku: undefined, fable: undefined }, + }), + ], + async (url) => { await post(url, { model: 'claude-opus-5' }) }, + ) + assert.equal(JSON.parse(primary.calls[0]?.body ?? '{}').model, 'claude-opus-5') + assert.equal(JSON.parse(backup.calls[0]?.body ?? '{}').model, 'glm-5.2') + } finally { + await primary.close() + await backup.close() + } +}) + +test('a non-retryable status fails immediately without burning attempts', async () => { + const up = await upstream([{ status: 400, body: '{"type":"error","error":{"type":"invalid_request_error","message":"bad"}}' }]) + try { + await withGateway([route({ baseUrl: up.url })], async (url) => { + const res = await post(url, { model: 'claude-opus-5' }) + assert.equal(res.status, 400) + }) + assert.equal(up.calls.length, 1) + } finally { + await up.close() + } +}) + +test('a 429 that reads as a balance problem is not retried', async () => { + const up = await upstream([{ + status: 429, + body: '{"error":{"message":"Insufficient balance or no resource package. Please recharge."}}', + }]) + try { + await withGateway([route({ baseUrl: up.url })], async (url) => { + const res = await post(url, { model: 'claude-opus-5' }) + assert.equal(res.status, 429) + }) + // Retrying a spend cap is pure latency; it will still be 429 in an hour. + assert.equal(up.calls.length, 1) + } finally { + await up.close() + } +}) + +test('a route with a key never forwards the caller’s own credential', async () => { + const up = await upstream([ok({ id: 'x' })]) + try { + await withGateway( + [route({ baseUrl: up.url, credential: 'route-key', header: 'authorization' })], + async (url) => { + await fetch(`${url}/v1/messages`, { + method: 'POST', + headers: { 'content-type': 'application/json', authorization: 'Bearer CALLER-SECRET' }, + body: JSON.stringify({ model: 'claude-opus-5' }), + }) + }, + ) + assert.equal(up.calls[0]?.auth, 'Bearer route-key') + assert.ok(!JSON.stringify(up.calls[0]).includes('CALLER-SECRET')) + } finally { + await up.close() + } +}) + +test('a session-mode route forwards the caller’s credential untouched', async () => { + const up = await upstream([ok({ id: 'x' })]) + try { + await withGateway([route({ baseUrl: up.url, credential: '' })], async (url) => { + await fetch(`${url}/v1/messages`, { + method: 'POST', + headers: { 'content-type': 'application/json', authorization: 'Bearer SUBSCRIPTION' }, + body: JSON.stringify({ model: 'claude-opus-5' }), + }) + }) + assert.equal(up.calls[0]?.auth, 'Bearer SUBSCRIPTION') + } finally { + await up.close() + } +}) + +test('count_tokens is answered locally and never reaches an upstream', async () => { + const up = await upstream([{ status: 401, body: '{"error":{"message":"jwt auth is not yet supported"}}' }]) + try { + await withGateway([route({ baseUrl: up.url })], async (url) => { + const res = await fetch(`${url}/v1/messages/count_tokens`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ model: 'claude-haiku-4-5', messages: [{ role: 'user', content: 'hello world' }] }), + }) + assert.equal(res.status, 200) + assert.ok(((await res.json()) as { input_tokens: number }).input_tokens > 0) + }) + assert.equal(up.calls.length, 0) + } finally { + await up.close() + } +}) + +test('a streamed response is forwarded frame for frame, ping events included', async () => { + const wire = + 'event: message_start\ndata: {"type":"message_start","message":{"usage":{"input_tokens":9}}}\n\n' + + 'event: ping\ndata: {"type":"ping"}\n\n' + + 'event: content_block_delta\ndata: {"type":"content_block_delta","delta":{"text":"hi"}}\n\n' + + 'event: message_stop\ndata: {"type":"message_stop"}\n\n' + const up = await upstream([{ status: 200, body: wire, headers: { 'content-type': 'text/event-stream' } }]) + try { + await withGateway([route({ baseUrl: up.url })], async (url) => { + const res = await post(url, { model: 'claude-opus-5', stream: true }) + const text = await res.text() + // A dropped ping leaves a client with no way to tell a slow stream from + // a dead one, and Claude Code has no inactivity watchdog. + assert.ok(text.includes('event: ping')) + assert.equal(text, wire) + }) + } finally { + await up.close() + } +}) + +test('every route failing surfaces the last upstream error', async () => { + const a = await upstream([overloaded()]) + const b = await upstream([{ status: 429, body: '{"type":"error","error":{"type":"rate_limit_error","message":"slow"}}' }]) + try { + await withGateway( + [route({ profile: 'a', baseUrl: a.url }), route({ profile: 'b', baseUrl: b.url })], + async (url) => { + const res = await post(url, { model: 'claude-opus-5' }) + assert.equal(res.status, 429) + }, + ) + } finally { + await a.close() + await b.close() + } +}) + +test('/health names the routes without leaking credentials', async () => { + await withGateway([route({ profile: 'work', baseUrl: 'http://127.0.0.1:1', credential: 'SECRET' })], async (url) => { + const res = await fetch(`${url}/health`) + const text = await res.text() + assert.match(text, /work/) + assert.ok(!text.includes('SECRET')) + }) +}) diff --git a/test/adapters/gateway/table.test.ts b/test/adapters/gateway/table.test.ts new file mode 100644 index 0000000..989b157 --- /dev/null +++ b/test/adapters/gateway/table.test.ts @@ -0,0 +1,139 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { buildTable, modelFor, tierOf } from '../../../src/adapters/gateway/table.ts' +import { makeDescriptor } from '../../support/fixtures.ts' +import type { State } from '../../../src/ports/config-store.ts' +import type { ProviderRegistryPort, ProviderDescriptor } from '../../../src/ports/provider.ts' + +const anthropic = makeDescriptor({ + id: 'anthropic', + baseUrl: null, + credentialEnv: 'ANTHROPIC_API_KEY', + defaultModels: {}, +}) + +const zai = makeDescriptor({ + id: 'zai', + baseUrl: 'https://api.z.ai/api/anthropic', + credentialEnv: 'ANTHROPIC_AUTH_TOKEN', + defaultModels: { opus: 'glm-5.2', sonnet: 'glm-5.2', haiku: 'glm-5.2', fable: 'glm-5.2' }, +}) + +const registry = (...all: ProviderDescriptor[]): ProviderRegistryPort => ({ + all: () => all, + byId: (id) => all.find((d) => d.id === id) ?? null, +}) + +/** Two profiles, so failover has somewhere to go. */ +const twoProfiles = (): State => + ({ + version: 4, + providerAccounts: { + work: { provider: 'anthropic' }, + glm: { provider: 'zai', apiKey: 'zai-key' }, + }, + setups: { + work: { models: { opus: 'claude-opus-5', sonnet: 'claude-sonnet-5' } }, + glm: {}, + }, + profiles: { + work: { setup: 'work', accounts: ['work'], strategy: 'single' }, + glm: { setup: 'glm', accounts: ['glm'], strategy: 'single' }, + }, + defaultProfile: 'work', + bindings: {}, + settings: {}, + }) as unknown as State + +test('a route is derived per profile, primary first', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work', 'glm']) + assert.ok(table.ok) + assert.deepEqual(table.routes.map((r) => r.profile), ['work', 'glm']) +}) + +test('a null descriptor baseUrl becomes the real Anthropic host', () => { + // The launcher expresses "default endpoint" by leaving the variable unset. + // A proxy has to name the host it will actually dial. + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work']) + assert.ok(table.ok) + assert.equal(table.routes[0]?.baseUrl, 'https://api.anthropic.com') +}) + +test('the credential header follows the provider descriptor', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work', 'glm']) + assert.ok(table.ok) + assert.equal(table.routes[0]?.header, 'x-api-key') + assert.equal(table.routes[1]?.header, 'authorization') +}) + +test('an account with no key resolves to session mode rather than an empty key', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work']) + assert.ok(table.ok) + // Empty credential is the signal that the caller's own login is forwarded. + assert.equal(table.routes[0]?.credential, '') + assert.equal(table.routes[0]?.provider, 'anthropic') +}) + +test('a credential is only ever attached to its own account’s route', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work', 'glm']) + assert.ok(table.ok) + assert.equal(table.routes[1]?.credential, 'zai-key') + // The key entered for z.ai must not appear on the Anthropic row. + assert.equal(table.routes[0]?.credential, '') +}) + +test('an unknown profile fails the whole table rather than silently shrinking it', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work', 'nope']) + assert.equal(table.ok, false) + assert.match(table.ok === false ? table.reason : '', /nope/) +}) + +test('an unregistered provider is reported, not skipped', () => { + const table = buildTable(twoProfiles(), registry(anthropic), ['glm']) + assert.equal(table.ok, false) + assert.match(table.ok === false ? table.reason : '', /zai/) +}) + +test('building the table does not rotate a round-robin profile', () => { + // resolveProfileRefs advances a cursor as a side effect when given one. + // Passing one here would change which account a later launch picks, purely + // because a gateway was started. + const state = twoProfiles() + state.profiles.work = { setup: 'work', accounts: ['work'], strategy: 'round-robin' } + const before = JSON.stringify(state) + const first = buildTable(state, registry(anthropic, zai), ['work']) + const second = buildTable(state, registry(anthropic, zai), ['work']) + assert.ok(first.ok && second.ok) + assert.equal(JSON.stringify(state), before, 'state must not be mutated') + // Rebuilding must be idempotent: if a cursor were threaded through, the + // second build would select a different account than the first. + assert.deepEqual(first.routes, second.routes) +}) + +test('tierOf finds which tier the client asked for', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work', 'glm']) + assert.ok(table.ok) + const primary = table.routes[0] + assert.ok(primary) + assert.equal(tierOf(primary, 'claude-opus-5'), 'opus') + assert.equal(tierOf(primary, 'claude-sonnet-5'), 'sonnet') + assert.equal(tierOf(primary, 'something-else'), null) +}) + +test('failover asks the next profile for its model at the SAME tier', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work', 'glm']) + assert.ok(table.ok) + const [primary, fallback] = table.routes + assert.ok(primary && fallback) + // An opus-tier request must not arrive at z.ai still saying claude-opus-5, + // which is a 404 in a working request's costume. + assert.equal(modelFor(fallback, tierOf(primary, 'claude-opus-5'), 'claude-opus-5'), 'glm-5.2') +}) + +test('a model matching no tier is forwarded verbatim rather than rewritten', () => { + const table = buildTable(twoProfiles(), registry(anthropic, zai), ['work', 'glm']) + assert.ok(table.ok) + const fallback = table.routes[1] + assert.ok(fallback) + assert.equal(modelFor(fallback, null, 'some-custom-model'), 'some-custom-model') +}) diff --git a/test/adapters/gateway/tokens.test.ts b/test/adapters/gateway/tokens.test.ts new file mode 100644 index 0000000..599738c --- /dev/null +++ b/test/adapters/gateway/tokens.test.ts @@ -0,0 +1,49 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { estimateRequestTokens, estimateTokens } from '../../../src/adapters/gateway/tokens.ts' + +test('an empty string costs nothing', () => { + assert.equal(estimateTokens(''), 0) +}) + +test('a request with system, messages and tools counts all three', () => { + const withAll = estimateRequestTokens({ + system: 'be brief', + messages: [{ role: 'user', content: 'hello world' }], + tools: [{ name: 'get_weather', description: 'weather', input_schema: { type: 'object' } }], + }) + const withoutTools = estimateRequestTokens({ + system: 'be brief', + messages: [{ role: 'user', content: 'hello world' }], + }) + assert.ok(withAll > withoutTools) +}) + +test('block content is walked, not just plain strings', () => { + const blocks = estimateRequestTokens({ + messages: [{ + role: 'assistant', + content: [ + { type: 'text', text: 'here you go' }, + { type: 'tool_use', name: 'run', input: { cmd: 'ls -la' } }, + ], + }], + }) + assert.ok(blocks > 0) +}) + +test('an image is charged a flat cost rather than counted as base64 text', () => { + // Counting the base64 blob as prose would inflate a screenshot into tens of + // thousands of tokens and compact the session almost immediately. + const huge = 'A'.repeat(200_000) + const asImage = estimateRequestTokens({ + messages: [{ role: 'user', content: [{ type: 'image', source: { type: 'base64', data: huge } }] }], + }) + assert.ok(asImage < 5000, `image should not scale with payload size, got ${asImage}`) +}) + +test('malformed input yields zero instead of throwing', () => { + assert.equal(estimateRequestTokens(null), 0) + assert.equal(estimateRequestTokens('not an object'), 0) + assert.equal(estimateRequestTokens({ messages: 'not an array' }), 0) +})