Skip to content

fix(responses): dispatch SSE on payload type, not just event: line (#434) - #435

Merged
initializ-mk merged 1 commit into
mainfrom
fix/responses-sse-event-dispatch
Sep 1, 2026
Merged

fix(responses): dispatch SSE on payload type, not just event: line (#434)#435
initializ-mk merged 1 commit into
mainfrom
fix/responses-sse-event-dispatch

Conversation

@initializ-mk

Copy link
Copy Markdown
Contributor

Fixes #434.

Problem

The openai-responses SSE parser (readStream) routed events only on the SSE event: line. That field is optional per the SSE spec, and the Bedrock openai-sigv4 gateway (Kong /bedrock/openai-sigv4/v1/responses) omits it — it streams the event type as a type field inside each data: payload instead:

data: {"delta":"Loading","type":"response.output_text.delta"}
data: {"response":{...,"usage":{"input_tokens":76,"output_tokens":73,"total_tokens":149}},"type":"response.completed"}

With no event: lines, currentEvent stayed "", the switch matched nothing, and every frame was dropped — deltas and the terminal response.completed (usage).

Field evidence

  • llm_call: input_tokens:0, output_tokens:0, tokens_unavailable:true on a 619ms call.
  • Log: {"finish_reason":"","msg":"llm response"} — empty finish reason is the smoking gun (only response.completed sets one).
  • User got an empty answer; the raw stream shows the gateway did return content + usage.

Fix

  1. Dispatch on the payload type field, falling back to the event: line only when the payload has none (authoritative, present in both dialects; matches how the OpenAI SDKs route).
  2. Bump bufio.Scanner's 64KB line cap — the response.completed frame embeds the full output[] on one data: line and overruns it on large answers.

Tests

  • TestResponsesClient_DispatchesOnPayloadType_NoEventLines — the gateway dialect (no event: lines) recovers text + usage (76/73/149) + finish reason stop.
  • TestResponsesClient_EventLineDialectStillWorks — real-OpenAI dual dialect still parses (no regression).
  • All existing responses/provider tests pass; golangci-lint clean.

Out of scope

https://claude.ai/code/session_01Hkimw1PDJRY5Dh8BgNQxWJ

)

The openai-responses parser routed events solely on the SSE `event:`
line. `event:` is optional per the SSE spec, and the Bedrock
openai-sigv4 gateway (Kong /bedrock/openai-sigv4/v1/responses) omits it,
carrying the event type only as a `type` field inside each `data:`
payload. With no `event:` lines, currentEvent stayed "" and EVERY frame
was dropped — text deltas and the terminal response.completed (usage).
Field symptom: empty assistant message, input_tokens:0/output_tokens:0/
tokens_unavailable:true, and an empty finish_reason.

- Prefer the payload `type` field (authoritative, present in both the
  real-OpenAI dual dialect and the event:-less gateway dialect); fall
  back to the `event:` line when the payload has none.
- Bump bufio.Scanner's 64KB line cap: the response.completed frame
  embeds the full output[] on one data: line and overruns it on large
  answers, aborting the stream mid-parse.

Tests: event:-less stream recovers text + usage (76/73/149) + finish
reason "stop"; dual-dialect stream still parses (no regression).

Claude-Session: https://claude.ai/code/session_01Hkimw1PDJRY5Dh8BgNQxWJ

@initializ-mk initializ-mk left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve-grade — precise diagnosis, minimal robust fix, no regression path. Traced the full readStream loop against the branch source.

Correct root-cause fix

The event: line is optional per the SSE spec, and the Bedrock openai-sigv4 gateway omits it, carrying the type only as an in-band type field — so the old dispatch left currentEvent=="" and dropped EVERY frame (empty answer, zero usage, empty finish reason, exactly the field symptoms). Preferring the payload type and falling back to the event: line is the right call: type is present in both dialects and is how the OpenAI SDKs route.

Things I verified beyond the happy path:

  • No currentEvent-reset regressioneventType is recomputed per data: frame; since type is present in both dialects the fallback to a possibly-stale currentEvent effectively never fires, and where it would, the old code dropped the frame anyway.
  • Non-JSON / [DONE] frames are safejson.Unmarshal on a non-JSON data: returns an error (never panics), so eventType falls back to currentEvent and the switch no-ops. Correct terminator handling.
  • Scanner buffer bump uses the right signature (Buffer(initial, max)) and fixes the real response.completed large-frame overrun.
  • Regression coverage — the event-less gateway dialect recovers text + usage (76/73/149) + finish reason stop, and the real-OpenAI dual dialect still parses. Both green.

Non-blocking trivia

  • The 8 MB line cap is a theoretical ceiling — a response.completed embedding >8 MB of output[] would still abort the stream. Generous and bounded, so fine in practice; awareness note only.
  • The per-frame double-unmarshal (type probe + full payload) could be collapsed, but the lightweight probe-first approach is cleaner and the cost is negligible for SSE. Not worth changing.

The two deferred items (plain-text read_skill, cached_tokens #431 parity) are legitimately out of scope. All 9 CI checks green.

var typed struct {
Type string `json:"type"`
}
if json.Unmarshal([]byte(data), &typed) == nil && typed.Type != "" {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the crux and it is correct. Preferring the in-band type over the event: line is what recovers the gateway dialect, and the guard is well-formed: on a non-JSON data: frame (e.g. a [DONE] sentinel) json.Unmarshal returns an error — not a panic — so eventType cleanly falls back to currentEvent, and && typed.Type != "" avoids clobbering a valid event:-derived type with an empty payload type. The result is robust across all three inputs: gateway (type only), real-OpenAI (both, matching), and junk/terminator frames (neither → no-op).

@initializ-mk
initializ-mk merged commit 327f0b8 into main Sep 1, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openai-responses provider drops entire SSE stream when gateway omits event: lines (empty content, zero usage)

1 participant