Skip to content

feat(voice): add respondToGather to collapse the conversational-turn round trip - #14

Open
michaela-band wants to merge 1 commit into
mainfrom
feat/respond-to-gather-single-round-trip
Open

michaela-band wants to merge 1 commit into
mainfrom
feat/respond-to-gather-single-round-trip

Conversation

@michaela-band

Copy link
Copy Markdown

What

Adds respondToGather(call_id, verbs, auto_gather) — a single tool that generates BXML from verbs and queues it as the callback response for an active call, in one round trip. It's the merge of the existing generateBXML + respondToCallback two-step, kept as-is for callers who still want them separate (e.g. inspecting BXML before sending).

Why

Every turn of a live, agent-driven voice conversation races Bandwidth's answer/gather callback retry window: the callback lands, and there's a bounded window to queue a BXML response via respondToCallback before the call gives up. The current two-tool-call pattern (generateBXML then respondToCallback, with a full model-reasoning step in between) burns real wall-clock time inside that window.

This isn't theoretical — ran an internal PoC (Claude Code driving this server against a live Bandwidth Build account) attempting exactly this pattern for a one-line greeting on call answer (turn zero, not even a full back-and-forth). 8 of 9 live call attempts connected and hung up with no audio delivered — the agent's tool-call round trip lost the race against the retry window every time but one.

Collapsing the two calls into one doesn't guarantee winning the race (the remaining latency is model inference time, which this doesn't touch), but it removes one full reasoning gap from the loop — the cheapest, lowest-risk fix available without changing the callback timing model itself. Michaela's asked me to separately scope a bigger fix (fast, non-agent-driven turn generation inside the callback handler, for genuine multi-turn conversation) as a spike in DX operations — this PR is the narrower, immediately-actionable half of that conversation.

What changed

  • src/tools/voice.py — respond_to_gather_flow() (mirrors the existing generate_bxml_flow / respond_to_callback_flow pattern in this file) plus the respondToGather tool registration. Same first-write-wins semantics as respondToCallback.
  • src/profiles.py — added to the voice profile.
  • src/instructions.py — voice section now tells agents to prefer respondToGather for every turn after the first, and explains why (the race).
  • src/specs/AGENTS.md, README.md — tool inventory updated.
  • test/test_bxml.py — two new tests: generates + queues BXML in one call, and first-write-wins matches respondToCallback's behavior.

Testing

Full suite: 119 passed (117 existing + 2 new), no regressions. Also smoke-tested the flow function directly against a live EventStore instance to confirm the queued BXML round-trips correctly.

Flagging for review rather than assuming this is the final shape — happy to discuss whether the tool should also read the pending gather event itself (folding in getCallbackEvents) rather than leaving that as a separate call, if that's a more natural round-trip reduction.

…round trip

Every turn of a live agent-driven conversation is a race against
Bandwidth's answer/gather callback retry window: the callback lands,
and the agent has a bounded window to queue a BXML response via
respondToCallback before the call gives up and hangs up.

The current pattern for a conversational turn is two separate tool
calls -- generateBXML, then respondToCallback -- with a full model
reasoning step in between. Each of those is wall-clock time that
narrows the window before the race is lost.

Ran an internal PoC (Claude Code + this server, live Bandwidth Build
account) attempting exactly this pattern for a scripted greeting (not
even a full conversation, just turn zero). 8 of 9 live call attempts
connected and hung up with no audio delivered -- the agent's tool-call
round trip lost the race against the callback retry window every time
but one.

respondToGather(call_id, verbs, auto_gather) merges generateBXML +
respondToCallback into a single tool call: generate the BXML from
verb descriptions and queue it as the callback response in one round
trip. Same first-write-wins semantics as respondToCallback. Does not
by itself guarantee winning the race -- it removes one full
model-reasoning gap from the loop, which is the highest-leverage,
lowest-risk fix available without changing the callback timing model
itself (a separate, bigger conversation -- see the accompanying spike
proposal for making the server's callback loop respond fast enough
for genuine multi-turn conversation without depending on the outer
agent's per-turn latency at all).

- src/tools/voice.py: new respond_to_gather_flow() (mirrors the
  generate_bxml_flow / respond_to_callback_flow pattern already in
  this file) plus the respondToGather tool registration.
- src/profiles.py: added to the voice profile.
- src/instructions.py: voice section now tells agents to prefer
  respondToGather for every turn after the first.
- src/specs/AGENTS.md, README.md: tool inventory updated.
- test/test_bxml.py: two new tests -- generates+queues in one call,
  and first-write-wins semantics match respondToCallback.

Full suite: 119 passed (117 existing + 2 new), no regressions.
@michaela-band
michaela-band requested review from a team as code owners September 24, 2026 15:13
@bwappsec

Copy link
Copy Markdown

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants