fix(responses): dispatch SSE on payload type, not just event: line (#434) - #435
Conversation
) The openai-responses parser routed events solely on the SSE `event:` line. `event:` is optional per the SSE spec, and the Bedrock openai-sigv4 gateway (Kong /bedrock/openai-sigv4/v1/responses) omits it, carrying the event type only as a `type` field inside each `data:` payload. With no `event:` lines, currentEvent stayed "" and EVERY frame was dropped — text deltas and the terminal response.completed (usage). Field symptom: empty assistant message, input_tokens:0/output_tokens:0/ tokens_unavailable:true, and an empty finish_reason. - Prefer the payload `type` field (authoritative, present in both the real-OpenAI dual dialect and the event:-less gateway dialect); fall back to the `event:` line when the payload has none. - Bump bufio.Scanner's 64KB line cap: the response.completed frame embeds the full output[] on one data: line and overruns it on large answers, aborting the stream mid-parse. Tests: event:-less stream recovers text + usage (76/73/149) + finish reason "stop"; dual-dialect stream still parses (no regression). Claude-Session: https://claude.ai/code/session_01Hkimw1PDJRY5Dh8BgNQxWJ
initializ-mk
left a comment
There was a problem hiding this comment.
Approve-grade — precise diagnosis, minimal robust fix, no regression path. Traced the full readStream loop against the branch source.
Correct root-cause fix
The event: line is optional per the SSE spec, and the Bedrock openai-sigv4 gateway omits it, carrying the type only as an in-band type field — so the old dispatch left currentEvent=="" and dropped EVERY frame (empty answer, zero usage, empty finish reason, exactly the field symptoms). Preferring the payload type and falling back to the event: line is the right call: type is present in both dialects and is how the OpenAI SDKs route.
Things I verified beyond the happy path:
- No
currentEvent-reset regression —eventTypeis recomputed perdata:frame; sincetypeis present in both dialects the fallback to a possibly-stalecurrentEventeffectively never fires, and where it would, the old code dropped the frame anyway. - Non-JSON /
[DONE]frames are safe —json.Unmarshalon a non-JSONdata:returns an error (never panics), soeventTypefalls back tocurrentEventand the switch no-ops. Correct terminator handling. - Scanner buffer bump uses the right signature (
Buffer(initial, max)) and fixes the realresponse.completedlarge-frame overrun. - Regression coverage — the event-less gateway dialect recovers text + usage (76/73/149) + finish reason
stop, and the real-OpenAI dual dialect still parses. Both green.
Non-blocking trivia
- The 8 MB line cap is a theoretical ceiling — a
response.completedembedding >8 MB ofoutput[]would still abort the stream. Generous and bounded, so fine in practice; awareness note only. - The per-frame double-unmarshal (type probe + full payload) could be collapsed, but the lightweight probe-first approach is cleaner and the cost is negligible for SSE. Not worth changing.
The two deferred items (plain-text read_skill, cached_tokens #431 parity) are legitimately out of scope. All 9 CI checks green.
| var typed struct { | ||
| Type string `json:"type"` | ||
| } | ||
| if json.Unmarshal([]byte(data), &typed) == nil && typed.Type != "" { |
There was a problem hiding this comment.
This is the crux and it is correct. Preferring the in-band type over the event: line is what recovers the gateway dialect, and the guard is well-formed: on a non-JSON data: frame (e.g. a [DONE] sentinel) json.Unmarshal returns an error — not a panic — so eventType cleanly falls back to currentEvent, and && typed.Type != "" avoids clobbering a valid event:-derived type with an empty payload type. The result is robust across all three inputs: gateway (type only), real-OpenAI (both, matching), and junk/terminator frames (neither → no-op).
Fixes #434.
Problem
The
openai-responsesSSE parser (readStream) routed events only on the SSEevent:line. That field is optional per the SSE spec, and the Bedrockopenai-sigv4gateway (Kong/bedrock/openai-sigv4/v1/responses) omits it — it streams the event type as atypefield inside eachdata:payload instead:With no
event:lines,currentEventstayed"", theswitchmatched nothing, and every frame was dropped — deltas and the terminalresponse.completed(usage).Field evidence
llm_call:input_tokens:0, output_tokens:0, tokens_unavailable:trueon a 619ms call.{"finish_reason":"","msg":"llm response"}— empty finish reason is the smoking gun (onlyresponse.completedsets one).Fix
typefield, falling back to theevent:line only when the payload has none (authoritative, present in both dialects; matches how the OpenAI SDKs route).bufio.Scanner's 64KB line cap — theresponse.completedframe embeds the fulloutput[]on onedata:line and overruns it on large answers.Tests
TestResponsesClient_DispatchesOnPayloadType_NoEventLines— the gateway dialect (noevent:lines) recovers text + usage (76/73/149) + finish reasonstop.TestResponsesClient_EventLineDialectStillWorks— real-OpenAI dual dialect still parses (no regression).golangci-lintclean.Out of scope
read_skill("weather")as plainoutput_textin aphase:"commentary"message rather than afunction_callitem, so the skill never loads — a separate model/gateway-prompt behavior issue.usage.input_tokens_details.cached_tokensfor cache-usage parity with Anthropic llm_call undercounts tokens: capture cache_read/creation + emit total_input_tokens (parity with initializ-sdk#10) #431.https://claude.ai/code/session_01Hkimw1PDJRY5Dh8BgNQxWJ