Skip to content

Anthropic adaptive-thinking models: thinking content invisible (display omitted) and thinking_tokens telemetry missing #1326

Description

@easonLiangWorldedtech

Bug

For the adaptive-thinking model class (supportsReasoningBinary: claude-sonnet-5, claude-opus-4-7, claude-opus-4-8, claude-opus-5, claude-fable-5), the thinking content is never visible in the UI, even though thinking tokens are billed.

Root cause

  1. Thinking text arrives empty. The Anthropic API defaults the adaptive model class to thinking.display: "omitted" — thinking blocks are present (and billed), but their text is stripped server-side. Zoo Code never sends display, so the stream handler's thinking_delta/content_block_start(thinking) cases never receive any text. The UI shows an empty reasoning block with only the elapsed-seconds timer.
    • AWS Bedrock docs (adaptive thinking) confirm the default: "The thinking block uses the same shape as any other thinking block (empty text with a signature under the default omitted display)."
    • The Bedrock provider in this repo already works around it (thinking: { type: "adaptive", display: "summarized" }), while the direct Anthropic API handler and the Vertex handler do not.
  2. No thinking-token telemetry. The API reports reasoning tokens inside usage.output_tokens_details.thinking_tokens (on both message_start and message_delta). Every other provider already surfaces this as reasoningTokens on the usage stream chunk (gemini, openai, openrouter, nanogpt, poe, …) — the Anthropic and Vertex handlers are the only ones missing it.

Repro

  • Any adaptive-class model (e.g. claude-opus-4-7) with reasoning enabled
  • Ask a multi-step question; the reasoning block shows no text (or only an empty block), while billed output tokens include thinking tokens.

Suggested fix

  • src/api/transform/reasoning.ts: return { type: "adaptive", display: "summarized" } from getAnthropicProviderReasoning (SDK 0.109.x already types ThinkingConfigAdaptive.display; "summarized" returns the server-side summary at no extra cost — billing is on full thinking tokens).
  • src/api/providers/anthropic.ts + src/api/providers/anthropic-vertex.ts: parse usage.output_tokens_details?.thinking_tokens into reasoningTokens on the message_start / message_delta usage yields (additive, no behavior change; output_tokens already includes these for billing).

A fix PR is in progress from my fork.

Impact

  • Users of Opus 4.7 / Opus 4.8 / Opus 5 / Sonnet 5 / Fable 5 are paying for thinking they cannot see — the core transparency value of extended thinking is lost.
  • Per-turn thinking telemetry (did the model actually think this step?) is unavailable for Anthropic models while it works for all other providers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions