Skip to content

STT audit: fix VAD event routing, add missing streaming session params, retry-safe transcribe body, document hand-patched files#1

Open
abhishekmishragithub wants to merge 3 commits into
mainfrom
fix/stt-streaming-audit
Open

STT audit: fix VAD event routing, add missing streaming session params, retry-safe transcribe body, document hand-patched files#1
abhishekmishragithub wants to merge 3 commits into
mainfrom
fix/stt-streaming-audit

Conversation

@abhishekmishragithub

@abhishekmishragithub abhishekmishragithub commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

What

Live-tested the STT surface (batch + realtime WebSocket) against api.smallest.ai and fixed what the audit found. All fixes verified against the live API before and after.

Bug fixes

  1. VAD events routed to the wrong handler. `speech_ended` messages fired the `speech_started` handler; the `speech_ended` handler never fired. The generated router matched messages by field presence (`type`+`session_id`+`timestamp`), which is identical for both events, and the enum wrapper tolerates unknown strings, so deserialization succeeded on the wrong type. Now routes on the `type` value. For voice-agent turn-taking this inverted barge-in logic.
  2. Retries sent an empty body on batch STT. `transcribePulse` used a one-shot InputStream-backed request body; on 408/429/5xx the RetryInterceptor re-sent the request with a drained stream, POSTing zero bytes and surfacing a confusing 400 instead of the real error. Now uses a byte[]-backed body (sync + async clients).
  3. NPE in `disconnect()`/`close()` when `connect()` was never called. Guarded.

Missing API coverage

The streaming connect options exposed only 5 of the session params the server supports. Added typed options for `keywords`, `eouTimeoutMs`, `endpointing`, `sentenceTimestamps`, `redactPii`, `redactPci`, `diarize`, `format`, `punctuate`, `capitalize`, `itnNormalize`, `finalizeOnWords`, `maxWords`. `additionalProperty(...)` entries are now forwarded as query params, so future server params work without an SDK release. All verified live (keywords boosting, eou_timeout_ms, sentence_timestamps/diarize confirmed active in responses).

New example

`examples/AzureStyleSttRecognizer.java`: recognizing/recognized/canceled/sessionStopped event adapter matching the Azure Speech SDK shape, for teams migrating an existing Azure STT integration.

Hand-written vs generated (maintenance model)

This SDK was generated manually with fern-java-sdk 4.13.0 from a dirty local checkout of smallest-ai-documentation@04f10a9 (see `.fern/metadata.json`). There is no `java-sdk` group in `generators.yml` yet, so nothing regenerates it in CI.

Following the smallest-python-sdk convention, this PR adds a `.fernignore` that separates:

  • Permanently hand-written files (examples/, build.gradle, README, wrapper, jitpack.yml). The generator never produces these.
  • Temporarily hand-patched generated files (4 files: the streaming WS client, its connect options, and the sync/async raw waves clients), each annotated with why it was patched and where the fix must be upstreamed before regenerating:
    • VAD routing + retry body: fern-java-sdk generator bugs, to be reported to Fern.
    • Streaming session params: spec wiring. Expose the full ws query binding from `fern/apis/waves/asyncapi/pulse-stt-ws(-overrides).yml` so the generator emits them.

The README gains a "Generation and maintenance" section with the rules: do not regenerate over `.fernignore`d files until their patches are upstreamed; end state is a `java-sdk` group in `fern/apis/unified/generators.yml` (github pull-request mode, like Python) and a `.fernignore` shrunk to hand-written files only.

Coverage audit (vs Python SDK)

Diffed all `waves` models and client methods against smallest-python-sdk: parity except the deliberately removed Lightning v2 TTS, and `sample_rate.py` (a hand-written Python file, not generator output). Java is ahead on STT: it has the VAD event message types Python lacks.

Testing

  • `./gradlew compileJava test` green.
  • Live: batch (plain + all flags + 401 path), streaming (interims/finals/finalize/close_stream/is_last), VAD events (both handlers verified firing correctly post-fix), new session params confirmed active server-side.

…ing session params, retry-safe transcribe body

- Fix WS message dispatch: speech_ended events were delivered to the
  speech_started handler (presence-based checks could not tell them apart);
  route on the type field value instead. Verified against the live API.
- Add typed streaming connect options from the current Pulse WS API:
  sentenceTimestamps, redactPii, redactPci, format, punctuate, capitalize,
  eouTimeoutMs, endpointing, diarize, keywords, itnNormalize,
  finalizeOnWords, maxWords. additionalProperty(...) entries are now
  forwarded as query params as an escape hatch for future params.
- transcribePulse: use a byte[]-backed request body so the RetryInterceptor
  can re-send it; the previous one-shot InputStream body sent an empty body
  on 408/429/5xx retries (sync + async clients).
- Guard disconnect() against NPE when connect() was never called.
- examples/AzureStyleSttRecognizer.java: recognizing/recognized/canceled
  event-style adapter for teams migrating from the Azure Speech SDK.
…iles, README section

- .fernignore (Python SDK convention) lists hand-written files and the four
  generated files carrying manual STT fixes, each with its upstream path
  (spec change in smallest-ai-documentation vs generator bug report to Fern).
- README explains the SDK is manually generated (see .fern/metadata.json),
  not yet in the Fern CI pipeline, and the rules for regenerating safely.
- gitignore temp/ (scratch clones used for coverage audits).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant