Skip to content

VAPI-3991: implement the Bandwidth StartStream source and wire /bw/stream - #30

Open
mramasubramanian-bw wants to merge 4 commits into
mainfrom
VAPI-3991/bw-stream-source
Open

mramasubramanian-bw wants to merge 4 commits into
mainfrom
VAPI-3991/bw-stream-source

Conversation

@mramasubramanian-bw

@mramasubramanian-bw mramasubramanian-bw commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The Media Streams bridge spoke Twilio's protocol to the customer's bot, but the Bandwidth side was only an interface. Nothing in src/ implemented BwStreamSource, and the server never wired a StartStream destination to the bridge, so a translated Connect/Stream opened a Bandwidth stream to a destination that could not relay anything.

This PR implements Bandwidth's side of the bridge and routes StartStream to it. All four ticket items are done: a live end-to-end call passed on 2026-09-25 (see Live verification), so the README rows for Connect/Stream and the bridge now read Live.

Changes

src/streams/bw-source.ts (new): BwWebSocketSource speaks Bandwidth's StartStream WebSocket protocol.

  • start: parses metadata (accountId, callId, streamId, streamName, tracks) and streamParams. waitForStart() gives the server a timeout.
  • media: relays the inbound (caller) track. Outbound-track frames and non-string payloads are ignored. Frames that arrive before release() are buffered (bounded to 10 s), so caller audio spoken while the bot's socket is still opening is delivered in order rather than dropped.
  • stop: emitted exactly once, on a stop event or socket close.
  • sendMedia becomes playAudio with audio/pcmu, flush becomes clear.
  • No dtmf: Bandwidth delivers digits via BXML Gather webhooks, not the media socket.

src/server/app.ts

  • The translator rewrites a Twilio <Stream url> to wss://<PUBLIC_BASE_URL>/bw/stream?dest=<bot url> (ws:// for an http base in local dev).
  • The HTTP upgrade is taken off Fastify's underlying server and handed to ws. The upgrade must carry the same Basic-auth credentials as the /bw/* webhooks (401), dest is required (400), must be ws/wss (400), and passes the egress guard (400). Any other path is 404.
  • After Bandwidth's start, a TwilioStreamBridge is built with callSid from callId, accountSid from config, customParameters from streamParams, and the configured playout latency pad. Caller audio is released once the bot's socket is open.
  • No start within streamStartTimeoutMs (default 5 s), an unreachable bot, or a bot that accepts TCP but never completes the WebSocket handshake within streamBotConnectTimeoutMs (default 10 s) closes the Bandwidth socket so <StopStream wait="true"> returns and the call's BXML moves on.
  • The raw upgrade socket gets an error listener before the egress check awaits DNS, so a reset in that window cannot become an uncaught exception.
  • Teardown runs in Fastify's preClose hook, terminating bridges and any upgraded socket that has not sent start yet, so app.close() does not wait on Bandwidth.

src/streams/bridge.ts: new connectTimeoutMs (ws handshakeTimeout, default 10 s) so ready() rejects on a black-hole bot. When the bot closes its socket, the Bandwidth source is closed. On Twilio that ends <Connect> and TwiML resumes. Here it ends the StartStream so BXML resumes likewise.

src/translator/translate.ts: destinationUsername / destinationPassword are stamped on StartStream alongside the existing callback credentials. Bandwidth presents them as the Authorization header on the upgrade.

src/twilio/egress-guard.ts: assertPublicUrl takes a schemes option so the stream route can require ws/wss.

src/server/capture.ts: with TRANSLATOR_CAPTURE_DIR set, raw Bandwidth frames are appended to <dir>/streams/<hash>.jsonl (start, first 10 media, stop). Same hashed-filename and private-mode guardrails as the TwiML capture; the key is a per-connection UUID, never Bandwidth input. The per-stream tally is dropped when the stop frame is captured.

src/server/index.ts: optional STREAM_PLAYOUT_LATENCY_PAD_MS, STREAM_START_TIMEOUT_MS, STREAM_BOT_CONNECT_TIMEOUT_MS.

Fixture

test/fixtures/bandwidth/stream-frames.json. The start.metadata block is the real event recorded on a live call on 2026-09-17, ids redacted. streamParams, media, and stop follow the documented shapes, with the media payload being 20 ms of synthetic PCMU silence. The live call below confirmed these shapes field for field. The frames it captured are deliberately not checked in: they came from a personal line and this repository is public. The _meta block says so and how to refresh from a shared test account.

Verification

  • npm run typecheck && npx vitest run: 46 files, 363 passed, 6 skipped. The four WebSocket test files were run five times in a row with no flakes.
  • test/streams-bw-source.test.ts: parsing, buffering and release, track filtering, the buffer cap, stop-once, playAudio/clear, onFrame.
  • test/server-stream.test.ts: a fake Bandwidth client drives the listening Fastify server to a fake Twilio bot. Both directions end to end, buffered audio order, bot hangup, unreachable bot, and black-hole bot (TCP accepted, no handshake) ending the stream, start timeout, app.close() with a pre-start socket, all four upgrade rejections, the URL rewrite with destination credentials, and frame capture including the 10-frame cap.
  • test/translate-callback-auth.test.ts: the new StartStream stamping.

Known limitation, pre-existing

assertPublicUrl resolves dest at upgrade time and the bridge re-resolves when it connects, so a DNS-controlled host could pass the check and rebind to an internal address. The HTTP egress path in src/twilio/egress.ts has the identical check-then-connect pattern, so this is consistent rather than new. A fix (pin the resolved address, or re-check in a lookup hook on the outbound socket) should cover both paths in one follow-up.

Live verification

A real inbound call on 2026-09-25 ran through the translator behind an ngrok tunnel, with TRANSLATOR_CAPTURE_DIR set and a Twilio-protocol echo bot as the customer's bot. Bandwidth fetched the TwiML through /bw/initiate, opened its StartStream WebSocket to /bw/stream (upgrade 101 with the destination credentials), and the bridge relayed it. The caller heard the bot's one-second tone and then their own voice echoed back.

Check Result
Caller audio to bot 878 frames over 17.6 s
Bot audio to caller 853 frames, audible on the phone
<StreamParam> to customParameters both values delivered
Mark held for playout (VAPI-3990) greeting mark returned after 999 ms, echo marks 40 to 51 ms
Stop propagation bot received stop when the call ended
Captured frames vs fixture same fields, nesting, and formats; Bandwidth orders eventType last

The first attempt returned 401 on /bw/initiate because the Voice application had no callback credentials configured. Setting them to WEBHOOK_USER / WEBHOOK_PASSWORD fixed it, which is the behaviour AGENTS.md Phase 3 already documents.

Docs

AGENTS.md describes the route, auth, capture, the tunnel requirement for WebSocket upgrades, and the optional latency pad. README.md names the new source in the tree.

Reference: StartStream docs.

@mramasubramanian-bw
mramasubramanian-bw requested review from a team as code owners September 25, 2026 13:31
@bwappsec

bwappsec commented Sep 25, 2026 •

Copy link
Copy Markdown

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues
✅ Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

…ream

The Media Streams bridge spoke Twilio's protocol to the bot, but the
Bandwidth side was only an interface: nothing in src/ implemented
BwStreamSource and the server never wired a StartStream destination to the
bridge. A translated Connect/Stream therefore opened a Bandwidth stream to
a destination that could not relay anything.

src/streams/bw-source.ts (new): BwWebSocketSource speaks Bandwidth's
StartStream WebSocket protocol over the socket Bandwidth opens to us.
- start: parses metadata (accountId, callId, streamId, streamName, tracks)
  and streamParams; waitForStart() with a timeout for the server.
- media: relays the inbound (caller) track; outbound-track frames and
  frames without a string payload are ignored. Frames arriving before
  release() are buffered (bounded to 10 s) so caller audio spoken while the
  bot's socket is still opening is delivered in order, not dropped.
- stop: emitted exactly once, on a stop event or on socket close.
- sendMedia -> playAudio (audio/pcmu); flush -> clear; close -> ws.close.
- onFrame hook for fixture capture. No dtmf: Bandwidth delivers digits via
  BXML Gather webhooks, not the media socket.

src/server/app.ts: the translator now rewrites a Twilio <Stream url> to
wss://<PUBLIC_BASE_URL>/bw/stream?dest=<bot url> (ws:// for http bases).
The HTTP upgrade is taken off Fastify's underlying server and handed to ws.
The upgrade must carry the same Basic-auth credentials as the /bw/*
webhooks (401 otherwise), dest is required (400), must be ws/wss (400) and
passes the egress guard (400), and any other path is 404. After Bandwidth's
start event a TwilioStreamBridge is built with callSid from callId,
accountSid from config, customParameters from streamParams, and the
configured playout latency pad; caller audio is released once the bot's
socket is open. No start within streamStartTimeoutMs (default 5 s), or an
unreachable bot, closes the Bandwidth socket so the translator's
<StopStream wait="true"> returns and the call's BXML moves on. Bridges are
closed on app close.

src/streams/bridge.ts: when the bot closes its socket, close the Bandwidth
source. On Twilio that ends <Connect> and TwiML resumes; here it ends the
StartStream so BXML resumes likewise.

src/translator/translate.ts: stamp destinationUsername/destinationPassword
on StartStream alongside the existing callback credentials. Bandwidth
presents them as the Authorization header on the WebSocket upgrade.

src/twilio/egress-guard.ts: assertPublicUrl accepts a schemes option so the
stream route can require ws/wss while HTTP callers keep http/https.

src/server/capture.ts: captureStreamFrame appends raw Bandwidth frames to
<captureDir>/streams/<hash>.jsonl (start, first 10 media, stop) so the
first live call yields checked-in fixtures. Same hashed-filename and
private-mode guardrails as captureTwiml; the key is a per-connection UUID,
never Bandwidth input.

test/fixtures/bandwidth/stream-frames.json: start.metadata is the real
event recorded on a live call 2026-09-17 (ids redacted); streamParams,
media, and stop follow the documented shapes, with the media payload being
20 ms of synthetic PCMU silence. The _meta block says so and how to
replace it from a capture.

Tests: streams-bw-source.test.ts covers parsing, buffering and release,
track filtering, the buffer cap, stop once, playAudio/clear, and onFrame.
server-stream.test.ts drives a fake Bandwidth client through the listening
server to a fake Twilio bot: both directions end to end, buffered audio
order, bot hangup and unreachable bot ending the stream, start timeout,
all four upgrade rejections, the URL rewrite with destination credentials,
and frame capture. translate-callback-auth.test.ts covers the new
StartStream stamping.

Docs: AGENTS.md describes the route, auth, capture, and the tunnel
requirement; README notes the source. The README capability rows stay at
Unit until an end-to-end call passes.
…tdown

Review follow-ups:

- Attach an error listener to the raw upgrade socket at the top of the
  handler. Node hands it over with none, ws only adds one inside
  handleUpgrade, and the egress check awaits a DNS lookup in between; a
  reset in that window was an uncaught exception that would kill the
  server.
- Bound the bot's WebSocket handshake. TwilioStreamBridge takes
  connectTimeoutMs (default 10 s) and passes it to ws as handshakeTimeout,
  so a bot that accepts TCP but never answers the upgrade makes ready()
  reject and the server ends the Bandwidth stream instead of holding the
  caller in silence until the OS gave up. Wired as
  ServerConfig.streamBotConnectTimeoutMs / STREAM_BOT_CONNECT_TIMEOUT_MS;
  STREAM_START_TIMEOUT_MS added alongside for symmetry.
- Move stream teardown from onClose to preClose. Fastify waits for HTTP
  connections to drain before onClose runs and an upgraded socket counts
  as one, so close() hung until Bandwidth hung up. preClose also
  terminates streamServer.clients (noServer mode does not on close()),
  covering a socket that upgraded but never sent start. The waitForStart
  timer is unref'd.
- captureStreamFrame forgets a stream's media tally when its stop frame
  is captured, so the module map no longer grows per call.
- State the credential-stamping invariant in a comment: callbackAuth only
  travels with the server's rewriter, which always points stream URLs at
  the translator.

Tests: black-hole bot at both the bridge and server level, app.close()
with a pre-start socket, the 10-frame capture cap (12 sent, 10 kept), and
a loosened upper bound on the start-timeout wall-clock assertion.
…d call

A real inbound call on 2026-09-25 ran through the translator: Bandwidth
opened its StartStream WebSocket to /bw/stream, the bridge relayed it to a
Twilio-protocol echo bot, and the caller heard the bot's tone and then their
own voice echoed back. Over 17.6 s: 878 caller frames in, 853 bot frames
out, StreamParam values delivered as customParameters, the bot's greeting
mark returned after 999 ms (held for the 1 s of audio queued ahead of it),
and stop propagated when the call ended.

README: split the stream row so Connect -> Stream and the Media Streams
bridge read Live; Start/Stop Stream forks and transcription stay Unit.

Fixture: frames unchanged. The _meta note now records that the live
frames matched these shapes field for field, and that the recorded frames
are deliberately not checked in because they came from a personal line and
this repository is public.
The Bandwidth start-event literal in streams-wire.test.ts carried a real
account id from the live repro. Replace it with the StartStream docs'
example value; the test only needs the shape.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants