Skip to content

buzz-acp: online mentions can be silently lost before enqueue and in native-steer withholding #4903

Description

@Dilosaurus

Summary

On Buzz Desktop / buzz-acp v0.5.4 on Windows, correctly p-tagged mentions to an online managed agent can be accepted and persisted by the relay but never reach the agent queue or session. We observed two distinct silent-loss seams in one incident:

  1. A live-subscription/pre-queue gap: the harness was connected and subscribed before the event was published, but the event produced no queue acceptance, lazy-pool wake, or session.
  2. A per-channel native-steer/withhold gap: events published during an active turn never appeared in the ACP session and had no steer-ack receipt; a later event after the visible turn also spawned nothing. The state was volatile and a later process replacement discarded it.

This is a silent-success failure: the relay accepts the event and the agent can appear online/ready, but the mention disappears without a sender- or operator-visible error.

Sanitized incident timeline (UTC, 2026-08-04)

  • 15:56:54: harness connected to a hosted private relay.
  • 15:57:00: subscribed to the DM channel.
  • 15:58:34: event 7d3dda2b… persisted with the correct recipient p tag. No pool wake, queue receipt, or session followed.
  • 16:20:57: a fresh process reported agent_pool_ready agents=10.
  • 16:21:30: a session started from an earlier catch-up event.
  • 16:22:22 and 16:23:42: events 2bd93407… and e5293983… persisted with the correct p tag while the turn was active. A literal scan of the session transcript found zero occurrences of both IDs, and no steer-ack was logged.
  • 16:24:12: the visible turn ended.
  • 16:27:52: event f5aed6c9… persisted with the correct tag but spawned no follow-on session.
  • 16:29:21: the managed process was replaced, discarding any volatile queued/withheld state.
  • A fresh subscription later recovered an event that already existed on the relay before subscribe, showing that catch-up could work even while live delivery was unreliable.

Relevant v0.5.4 paths

The source itself warns that losing the steer acknowledgement can leak a withheld event until the long in-flight deadline.

Evidence boundary

Current INFO logs do not record event IDs at receive, filter, enqueue, withhold, ack, dequeue, or spawn boundaries. Therefore:

  • The first event is narrowed to live relay delivery vs. an unlogged pre-queue filter branch; it definitively never reached session spawn.
  • The mid-turn events are code-supported narrowing to the native-steer/withheld-event seam, not proof of the exact internal branch.

Expected behavior

  • Every accepted matching event is durably queued or replayed after reconnect/restart.
  • A lost steer acknowledgement is bounded by a short timeout and releases the withheld event to normal dispatch.
  • “Online/ready” reflects working relay ingress, not only process liveness.
  • Operators can trace an event ID through receive → filter → enqueue → withhold → ack → dequeue → session.

Suggested fixes

  1. Persist a per-channel high-water mark and accepted/withheld queue.
  2. Catch up from that mark on reconnect and periodically; dedupe by event ID.
  3. Bound steer-ack waits to seconds and atomically release unacknowledged events on timeout or prompt completion.
  4. Persist queue state across harness replacement.
  5. Add event-ID transition logs and an end-to-end external canary.

Related: #4743 covers hosted-relay fan-out / silently dead subscriptions; #4189 covers a different native-steer deadlock for non-steerable ACP agents.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions