fix(voice): bound the redirect-wait loop instead of looping forever - #15
Closed
michaela-band wants to merge 1 commit into
Closed
michaela-band wants to merge 1 commit into
michaela-band wants to merge 1 commit into
Conversation
Reported directly by the voice platform team (AJ): the server exposes
/callbacks/voice/continue/{call_id}, and when Bandwidth POSTs to it with
no BXML queued yet, the server replies with <Redirect
redirectUrl=/callbacks/voice/continue/{call_id}/> -- a redirect back to
itself with no bound. If the agent driving the call is slow, crashed, or
never shows up, this loops indefinitely: Bandwidth keeps POSTing, the
server keeps redirecting to the same URL, forever.
Same root mechanism the internal PoC already flagged as a race condition
(agent tool-call latency vs. Bandwidth's callback retry window) -- this
is the failure mode when the agent loses that race and never recovers,
rather than losing it once and getting a silent hangup.
Fix: CallState now tracks how long it's been waiting for an agent
response (waiting_since, reset whenever BXML is actually queued). Once
BW_MCP_REDIRECT_TIMEOUT_SECONDS (default 12, configurable -- the real
number Bandwidth itself tolerates is unmeasured) elapses with nothing
queued, the server stops redirecting and returns a terminal response
(a short spoken apology + Hangup) instead of another redirect. Applies
at every point a call can be sitting in this state: the initial /answer
redirect, a /gather turn, and a bare /continue poll.
- src/event_store.py: CallState.waiting_since + start_waiting_if_unset(),
reset in try_set_bxml() once a response actually lands.
- src/callbacks.py: _wait_or_give_up() replaces the three separate
unconditional _redirect_bxml() calls in the answer/gather/continue
handlers.
- src/specs/AGENTS.md, README.md: documented the timeout and its env var.
- test/test_callbacks.py: four new tests -- timeout fires from answer,
gather, and continue, plus waiting_since resets once BXML lands.
Full suite: 121 passed (117 existing + 4 new), no regressions.
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Reported directly by the voice platform team (AJ): the server exposes
/callbacks/voice/continue/{call_id}, and when Bandwidth POSTs to it with no BXML queued yet, the server replies with<Redirect redirectUrl=/callbacks/voice/continue/{call_id}/>— a redirect back to itself, unconditionally. If the agent driving the call is slow, crashed, or never shows up at all, this loops indefinitely: Bandwidth keeps POSTing, the server keeps redirecting to the same URL, forever.Why
Same underlying mechanism the internal PoC already flagged as a race condition — an LLM agent's tool-call latency vs. Bandwidth's callback retry window. This is the failure mode when the agent loses that race and never recovers (crashes, hangs, or the session just ends), rather than losing it once and getting a clean silent hangup. Left unbounded, it's a real production risk, not just a PoC annoyance — any call where the driving agent disappears mid-session loops forever instead of terminating.
What changed
src/event_store.py—CallStatenow trackswaiting_since(when the current redirect-wait period started) viastart_waiting_if_unset(), reset back toNoneintry_set_bxml()once an agent's response actually lands.src/callbacks.py— new_wait_or_give_up()replaces the three separate unconditional_redirect_bxml()calls in the/answer,/gather, and/continuehandlers. OnceBW_MCP_REDIRECT_TIMEOUT_SECONDS(default 12, configurable — the real number Bandwidth itself tolerates before giving up on its own side is unmeasured, flagged in the accompanying conversational-voice spike proposal) elapses with nothing queued, the server returns a terminal response (a short spoken apology +Hangup) instead of another redirect.src/specs/AGENTS.md,README.md— documented the timeout, the env var, and that this is the server's own patience bound, not a measured Bandwidth-side limit.test/test_callbacks.py— four new tests: timeout fires correctly from/answer,/gather, and/continue, andwaiting_sinceresets once BXML actually lands (so a later, unrelated stall gets a fresh window instead of inheriting an old one).Testing
Full suite: 121 passed (117 existing + 4 new), no regressions. Also manually verified the timeout path directly against
_wait_or_give_upwith a fast simulated timeout — confirmed it returns a<Redirect>before the bound and a terminal<Hangup>response after, rather than looping.Based on
main, independent of #14 (respondToGather) — different bug, same callback surface, easier to review separately.