refactor: migrate ai-groq + ai-openrouter onto @tanstack/openai-base (#543)#545
Open
tombeckenham wants to merge 24 commits into
Conversation
…543) Adds protected `callChatCompletion`, `callChatCompletionStream`, `extractReasoning`, and `transformStructuredOutput` hooks to `OpenAICompatibleChatCompletionsTextAdapter` so providers with non-OpenAI SDK shapes can reuse the shared stream accumulator, partial-JSON tool-call buffer, RUN_ERROR taxonomy, and lifecycle gates. ai-groq drops `groq-sdk` in favour of the OpenAI SDK pointed at api.groq.com/openai/v1; ai-openrouter keeps `@openrouter/sdk` via hook overrides. ai-ollama remains on BaseTextAdapter (native API has a different wire format). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version Preview4 package(s) bumped directly, 24 bumped as dependents. 🟨 Minor bumps
🟩 Patch bumps
|
|
| Command | Status | Duration | Result |
|---|---|---|---|
nx affected --targets=test:sherif,test:knip,tes... |
❌ Failed | 6m 17s | View ↗ |
nx run-many --targets=build --exclude=examples/** |
✅ Succeeded | 1m 50s | View ↗ |
☁️ Nx Cloud last updated this comment at 2026-05-11 12:34:06 UTC
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
…ons migration Addresses regressions and pre-existing silent failures surfaced by reviewing #545: - `@tanstack/ai`: `toRunErrorPayload` normalizes `AbortError` / `APIUserAbortError` / `RequestAbortedError` to `{ code: 'aborted' }` so consumers can discriminate user-initiated cancellation without matching provider-specific message strings. - `@tanstack/openai-base`: `structuredOutput` throws a distinct "response contained no content" error instead of cascading into a misleading JSON-parse error on an empty string; the post-loop tool-args drain now logs malformed JSON via `logger.errors` so truncated streams don't silently invoke tools with `{}`. - `@tanstack/ai-openrouter`: `stream_options.include_usage` is camelCased to `includeUsage` (Zod was silently stripping it, leaving `RUN_FINISHED.usage` always undefined on streaming); mid-stream `chunk.error.code` is stringified so provider codes (401/429/500) survive `toRunErrorPayload`; assistant `toolCalls[].function.arguments` is stringified to match the SDK's `string` contract; `convertMessage` now mirrors the base's fail-loud guards (empty user content, unsupported content parts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds OpenRouterResponsesTextAdapter on top of @tanstack/openai-base's responses-text base, mirroring the chat-completions migration in #543. - openai-base: protected `callResponse` / `callResponseStream` hooks on OpenAICompatibleResponsesTextAdapter parallel to the existing `callChatCompletion*` hooks, so providers whose SDK has a different call shape can override without forking processStreamChunks. Re-exports the OpenAI Responses SDK types subclasses need. - ai-openrouter: new OpenRouterResponsesTextAdapter routing through `client.beta.responses.send({ responsesRequest })`. Emits the SDK's camelCase TS shape directly via overrides of convertMessagesToInput / convertContentPartToInput / mapOptionsToRequest, annotated with `Pick<ResponsesRequest, ...>` so future SDK field renames break the build instead of silently producing Zod-stripped wire payloads. Bridges inbound stream events camel -> snake so the base's processStreamChunks reads documented fields unchanged. - Function tools only in v1; webSearchTool() throws with a clear error pointing at the chat-completions adapter. - Folds in the silent-failure lessons from 0171b18 (stringified error codes, stringified tool-call arguments, fail-loud on empty user content). - E2E: new `openrouter-responses` provider slot in feature-support / test-matrix / providers / types / api.summarize, reusing aimock's native `/v1/responses` handler. - 10 new unit tests covering request mapping (snake -> camel for top-level fields, function-call camelCasing in input[], variant suffix), stream-event bridge (text deltas, function-call lifecycle, response.failed, top-level error code stringification), webSearchTool() rejection, and SDK constructor wiring. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes `validateTextProviderOptions` (no-op stub never called) and the chain of `ChatCompletion*MessageParam` / `ChatCompletionContentPart*` / `ChatCompletionMessageToolCall` types that were only referenced by it. Unblocks the root `test:knip` CI check. None of the removed exports are re-exported from the package's public `src/index.ts`, so this is internal-only cleanup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The OpenRouter SDK's stream-event schema is built with Speakeasy's
discriminated-union helper, which on a per-variant parse failure falls
back to `{ raw, type: 'UNKNOWN', isUnknown: true }` rather than throwing.
This happens whenever an upstream omits an "optional-looking" required
field — notably `sequence_number` and `logprobs` on text/reasoning delta
events, which aimock-served fixtures don't include.
Before this fix the adapter's switch hit the default branch for UNKNOWN
events and emitted them with no usable `type`, so the base's
processStreamChunks ignored them silently — the run terminated as
`RUN_FINISHED { finishReason: 'stop' }` with no content.
The `raw` payload preserved on the fallback is the original wire-shape
event in snake_case, which is exactly what processStreamChunks reads.
Re-emit it verbatim. Real-OpenRouter responses still flow through the
existing camel -> snake bridge because their events include the required
fields and parse cleanly.
Unblocks the openrouter-responses E2E suite: 11 affected tests now pass
locally against aimock; before this commit they all timed out empty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…i-openrouter-ai-ollama-to-openai-base-+-parameterize-the-base-for-sdk-shape-variance
Replaces ~200 sites of `asChunk({ type: 'X', ... })` (a `Record<string,
unknown> as unknown as StreamChunk` cast) with `({ type: EventType.X, ... })
satisfies StreamChunk` so the type system validates AG-UI event shape at
every emission. The cast was bypassing TypeScript's string-enum nominal
typing and masking a cluster of spec deviations now fixed:
- RUN_STARTED / RUN_FINISHED in openai-base (chat-completions + responses)
and all three summarize adapters were missing the AG-UI-required
`threadId`. Threading `options.threadId ?? generateId(this.name)` through
`aguiState` (matching the existing Gemini/Anthropic pattern) fixes it.
- RUN_ERROR emissions carried a non-existent `runId` field and the
deprecated nested `error: { message, code }` form instead of AG-UI's
top-level `message`/`code`. Both forms now coexist (deprecated kept for
back-compat) and `runId` is dropped — verified no consumer reads it
(chat-client.ts:404 only reads runId on RUN_FINISHED).
- STEP_STARTED / STEP_FINISHED in responses-text.ts were passing only the
deprecated `stepId` alias; AG-UI requires `stepName`. Now passes both.
- `finishReason` in chat-completions-text.ts was typed as `string`,
dropping below the AG-UI vocabulary. Widened `RunFinishedEvent.finishReason`
in `@tanstack/ai` to include OpenAI's `'function_call'` so it narrows
cleanly. responses-text.ts maps Responses-API `'max_output_tokens'` →
`'length'` and passes `'content_filter'` through.
- Per-event timestamps. AG-UI spec: "Optional timestamp indicating when
the event was created." Previously a single `const timestamp = Date.now()`
was captured at run start and reused on every emission across the eight
adapters; each chunk now uses `Date.now()` inline.
`@tanstack/ai/tests/test-utils.ts` `ev.*` builders are typed to return
precise event members via `satisfies StreamChunk`; the loose `chunk(type,
fields)` factory is preserved as a documented escape hatch for tests that
deliberately construct off-spec fixtures. ai-client tests no longer declare
a local `asChunk`. ai-groq's `processStreamChunks` override signature is
updated to include the new `threadId` field on `aguiState`.
Out of scope, flagged for follow-up:
- Framework tests (ai-react / ai-svelte / ai-vue) with inline string-literal
chunk arrays — their test directories aren't currently type-checked, so
they compile despite being off-spec.
- Summarize adapters omit TEXT_MESSAGE_START / TEXT_MESSAGE_END around
content emissions (separate AG-UI lifecycle gap).
Verified: pnpm -r test:types, test:lib, test:eslint, test:build all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tombeckenham
added a commit
that referenced
this pull request
May 12, 2026
#545's asChunk removal added \`threadId\` to RUN_STARTED/RUN_FINISHED on the chatStream path. The structuredOutputStream lift on this branch was emitting those events without \`threadId\`; the new \`satisfies StreamChunk\` checks now catch it. Plumb \`threadId\` through structuredOutputStream's aguiState in both bases. Also drop the residual \`asChunk()\` wrappers in my structuredOutputStream yields and use \`type: EventType.X, ... } satisfies StreamChunk\` directly, matching #545's new convention. While we're here: the chat-completions \`processStreamChunks\` finalisation forwards the SDK's \`finish_reason\` directly into \`RUN_FINISHED.finishReason\`, but the SDK type still includes the legacy \`function_call\` value that AG-UI doesn't accept. #545's \`satisfies\` cleanup exposed the mismatch — collapse \`function_call\` to \`stop\` alongside the existing orphan \`tool_calls\` collapse.
The chat adapter's convertMessage JSON-stringified Array<ContentPart> assistant content (so a multi-part assistant turn would round-trip as the literal JSON of the parts instead of joined text) and emitted `content: undefined` for tool-call-only assistants where the OpenAI Chat Completions contract documents `null`. Use the base's extractTextContent + emit `null` for the tool-call-only case so the override matches the chat-completions base. The Responses adapter's convertMessagesToInput tool branch had the same shape — JSON.stringify(message.content) fed the raw ContentPart shape into function_call_output.output for structured tool results. Use extractTextContent there too. Regression tests assert (a) array-shaped assistant content extracts to joined text rather than JSON, and (b) tool-call-only assistant content emits `null` rather than `undefined`.
The interface declared a single capitalized `Function` key with no
`type` discriminator. The OpenAI / Groq Chat Completions wire format
for a named tool_choice is `{ type: 'function', function: { name } }`.
Construct a literal against the old type and the SDK's Zod schema
would either reject it or treat tool_choice as unset.
No production code constructs this type literally yet — only the
`ChatCompletionToolChoiceOption` union in the same file uses it — so
fixing the shape now is a no-op at runtime but locks the type to the
correct contract going forward.
The module-level pendingMockCreate is only cleared inside applyPendingMock when a factory call consumes it. Tests in the first describe block instantiate the adapter without calling setupMockSdkClient first, so a leaked value from a prior test would inject a stale mock into a later adapter. Reset in beforeEach for deterministic ordering regardless of test-runner permutation.
The feature-support matrix advertises summarize / summarize-stream for both `openrouter` and `openrouter-responses`, but the factories silently substituted `createOpenaiSummarize` against the OpenAI base URL — exercising the OpenAI adapter while reporting OpenRouter coverage. Wire `createOpenRouterSummarize` (a thin wrapper over the OpenRouter chat adapter, used for both rows since the summarize endpoint is chat-completions-only) against the LLMOCK base so the matrix's claim is actually verified.
Sibling adapters (`ai-openai`, `ai-groq`, `ai-grok`) all declare zod as a peerDependency so a consumer that passes a Zod tool schema gets a single zod instance shared with this adapter. Without the peerDep, strict installs (pnpm `strict-peer-dependencies`, yarn berry pnp) can end up with two zod copies — one transitive via `@openrouter/sdk` or `@tanstack/ai`, one direct — and `instanceof ZodType` checks then fail across the boundary.
…override The Groq subclass declared its aguiState parameter with an extra `timestamp: number` field that does not exist on the base class's aguiState type. TypeScript's bivariant method-parameter checks let the wider type pass typecheck, but at runtime the body never reads `timestamp` and the field is never populated by the base, so any caller (or future override) that relied on the declared shape would observe `undefined`. Realign the override's parameter type with the base.
The chunk-level 'error' branch in adaptOpenRouterResponsesStreamEvents already stringifies provider codes so they survive toRunErrorPayload's string-only code filter, but the parallel response.failed / response.incomplete path went through toSnakeResponseResult which forwarded `r.error.code` raw. A provider that returned a numeric code (401/429/500/…) on a terminal failure event would lose it on the way through to RUN_ERROR. Mirror the chunk-level stringification inside toSnakeResponseResult and add a regression test for response.failed with a numeric error.code.
When a base64 image source has no mimeType the override produced a literal `data:undefined;base64,...` URI that the upstream rejects as invalid. The chat-completions base defaults to `application/octet-stream` for exactly this case; mirror the same defaulting in the OpenRouter convertContentPart override. Regression test asserts the data URI no longer contains the literal `undefined`.
The Responses adapter's processStreamChunks marked `runFinishedEmitted` on a top-level chunk.type === 'error' to prevent the synthetic terminal block from firing, but it did not return from the for-await loop. Any subsequent chunks the upstream delivered after a terminal error event (a stray output_text.delta, an output_item.done, etc.) would continue to emit lifecycle events past RUN_ERROR, violating the 'RUN_ERROR is terminal' contract. Mirror the response.failed / response.incomplete branches above: return after yielding RUN_ERROR. Regression test covers the case where the upstream continues delivering chunks after a top-level error event and asserts no further chunks reach the consumer.
…ough transformStructuredOutput hook The Responses base hard-coded transformNullsToUndefined on parsed structured-output JSON, leaving no hook for subclasses to opt out. The changeset's promise of 'transformStructuredOutput for subclasses (like OpenRouter) that preserve nulls in structured output instead of converting them to undefined' was therefore only fulfilled on the chat-completions surface — the matching Responses adapter would silently strip nulls regardless of provider intent. Add the transformStructuredOutput protected hook on OpenAICompatibleResponsesTextAdapter mirroring the chat-completions base's design, and override it as a no-op on OpenRouterResponsesTextAdapter so OpenRouter callers see null sentinels round-trip identically across the two adapter surfaces. Regression test asserts a structuredOutput response containing `nickname: null` round-trips as null (not undefined) through the Responses adapter.
The chat-completions adapter's convertMessage tool branch still
JSON-stringified Array<ContentPart> tool message content, so a tool
result delivered as structured parts (e.g. [{type:'text', content:
'"temp":'}, {type:'text', content:'72'}]) reached the model as the
literal JSON of the parts rather than the joined textual result. The
parallel responses adapter override was fixed earlier; this mirrors
the same fix on the chat-completions path so both surfaces handle
structured tool content identically.
Regression test feeds a structured tool result and asserts the wire
payload's tool message content is the joined text without any
'"type":"text"' leakage.
Every sibling adapter (ai-openai, ai-grok, ai-openrouter, ai-anthropic, ai-gemini, ai-fal, ai-ollama) explicitly lists `@tanstack/ai: workspace:*` under devDependencies in addition to declaring it as a peer. ai-groq omitted the devDep entry, so resolution worked only via pnpm's autoInstallPeers behaviour — toggling that off (strict installs, some yarn berry configs) would silently break ai-groq while every other adapter kept working. Add the dev dep for parity.
…ions
The chat-completions OpenRouter adapter's convertContentPart for audio
unconditionally emitted `{ type: 'input_audio', inputAudio: { data,
format: 'mp3' } }` — but `data` is supposed to be base64. A
URL-sourced audio part therefore shipped the literal URL string into
the base64 slot, which the upstream rejects (or worse, treats as
garbage audio bytes). The Responses adapter already handles this by
routing URL audio through `input_file` (where the URL belongs);
chat-completions has no `input_file` shape on this surface, so
mirror the existing document fallback: emit a text reference noting
the URL. Callers needing real audio URL support should use the
Responses adapter.
The header comment claimed these types "mirror the Groq SDK types", but the migration dropped the groq-sdk dependency entirely in favour of pointing the OpenAI SDK at Groq's /openai/v1 base URL. The file is now the source of truth for Groq-specific wire fields (compound tools, citation/service-tier provider options, …), not a mirror of an external SDK. Update the header to reflect the post-migration role.
The chat-completions convertContentPart 'document' branch unconditionally
returned `{ type: 'text', text: `[Document: ${part.source.value}]` }`.
For URL sources that's a reasonable degradation. For data sources,
`part.source.value` is the raw base64 payload — a multi-megabyte
document would be inlined into the prompt verbatim, blowing the
context window and leaking the document content as plaintext bytes.
Branch on `part.source.type`: URL sources keep the text-reference
fallback, data sources throw with a clear error pointing the caller at
the Responses adapter (which has proper `input_file` support for
inline document data). Mirrors the audio URL/data branching added in
the prior round.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #543 (for groq + openrouter; ollama stays on
BaseTextAdapter— see rationale below).@tanstack/openai-base— adds four protected hooks onOpenAICompatibleChatCompletionsTextAdapterso providers with non-OpenAI SDK shapes can plug in:callChatCompletion/callChatCompletionStream(SDK call sites),extractReasoning(surface reasoning content into the base's REASONING_* + legacy STEP_STARTED/STEP_FINISHED lifecycle), andtransformStructuredOutput(subclasses can opt out of the default null→undefined transform). Defaults preserve existing behaviour for ai-openai / ai-grok.@tanstack/ai-groq— rewritten as a thin subclass (~91 LOC, down from 650). Dropsgroq-sdkin favour of the OpenAI SDK pointed athttps://api.groq.com/openai/v1(the same pattern as ai-grok against xAI). Preserves thex_groq.usagequirk via a small stream wrapper.@tanstack/ai-openrouter— rewritten as a subclass with hook overrides (~396 LOC, down from 807). Keeps@openrouter/sdkfor typed provider routing, plugins, and metadata; a small request shape converter (max_tokens→maxCompletionTokens, etc.) and chunk shape adapter bridge the SDK boundary. Provider routing, app attribution headers (httpReferer/appTitle), reasoning variants, andRequestAbortedErrorhandling are preserved.Net: −731 lines, ~1k LOC of duplicated stream/tool/lifecycle code removed. Unblocks #527's centralised structured-output lift for groq + openrouter.
Why ai-ollama is out of scope
Ollama's native API (
ollamanpm package) uses a different wire format from OpenAI Chat Completions — different request shape (options: { num_ctx, num_gpu, ... }), different chunk shape (chunk.message.{content, tool_calls, thinking},chunk.done), non-incremental tool-call streaming, and a differentformatfield for structured output. The base'sprocessStreamChunks(the bulk of the duplication win) assumes OpenAI Chat Completions chunks; bridging Ollama would require overriding every inherited method, leaving the base doing no useful work. The earlier changeset (refactor-providers-to-shared-packages.md) already documented this.Two notable test-contract changes flagged in the changeset:
ai-openrouterstructuredOutputerror wrapping ("Structured output generation failed: ..."and"Structured output response contained no content") is replaced by the shared base's cleaner unwrapped errors. Two test assertions updated to match.transformStructuredOutputto the identity function.Test plan
pnpm --filter @tanstack/openai-base test:types test:lib test:eslint test:build— 71/71 unit, types/lint/build cleanpnpm --filter @tanstack/ai-groq test:types test:lib test:eslint test:build— 17/17 unit, types/lint/build cleanpnpm --filter @tanstack/ai-openrouter test:types test:lib test:eslint test:build— 43/43 unit, types/lint/build cleanpnpm --filter @tanstack/ai-openai test:lib— 131/131 (regression check on the base hooks)pnpm --filter @tanstack/ai-grok test:lib— 53/53 (regression check on the base hooks)pnpm test:types(Nx affected) — all 32 projects cleanpnpm test:eslint(Nx affected) — all 31 projects cleanpnpm --filter @tanstack/ai-e2e test:e2e -- --grep "groq"— gating signal per CLAUDE.mdpnpm --filter @tanstack/ai-e2e test:e2e -- --grep "openrouter"— gating signal per CLAUDE.mdx_groq.usageoverride)provider: { order: [...] }(validates provider routing through the request shape conversion):thinkingvariant (validatesextractReasoninghook)RequestAbortedError→ RUN_ERROR mapping)🤖 Generated with Claude Code