Skip to content

refactor(net)!: route everything through create_broadcast, gate announce on Route.live#2396

Merged
kixelated merged 10 commits into
devfrom
claude/broadcast-routing-api-b9a743
Jul 18, 2026
Merged

refactor(net)!: route everything through create_broadcast, gate announce on Route.live#2396
kixelated merged 10 commits into
devfrom
claude/broadcast-routing-api-b9a743

Conversation

@kixelated

@kixelated kixelated commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

What

Replaces the origin's announce/unannounce surface with liveness on the route, per the design discussion:

  • broadcast::Route gains announce: bool (default false, builder .with_announce(bool)). The origin announces a path only while its best source's route asks for it. A non-live broadcast stays in the tree: request_broadcast / consume_broadcast / subscribes / fetches all still resolve it, it just never appears in announced(). Toggling announce via set_route announces/unannounces without touching the broadcast, so an offline broadcast keeps serving cached or on-demand content.
  • origin::Producer::create_broadcast(path, route) is the sole entry point, returning a plain broadcast::Producer that acts as a route source. Calling it again at the same path attaches another source; the origin owns the spliced broadcast consumers actually see and always serves from the best route (announced first, then cost, then shortest hops with the deterministic tie-break). set_route is only needed to update the route afterwards.
  • Lifecycle: finish() detaches immediately (graceful unannounce, and a later create at the path is a fresh broadcast); a bare drop is treated as a failure, so the path lingers ~5s for a reconnecting session to splice back in invisibly. This replaces the Publish/Broadcast guards, Route::unannounce(), and the guard-vs-producer lifetime split.
  • Sessions use the same entry point. The lite subscriber now creates a source per received announce and serves the origin's track requests through broadcast::Producer::dynamic(), the shape the IETF subscriber already had; attach_route and the whole Assignments/Assignment/Serving surface are gone. Attach is deferred to the source watcher's first route_changed observation, which gives every creator a synchronous window to register tracks/handlers before the broadcast becomes visible (this closes the handler-registration race for free).

Deleted from the public surface: origin::Publish, origin::Broadcast, publish_broadcast, attach_route, Assignments, Assignment, Serving.

Semantic changes to be aware of

  • Consumers now always receive the origin-owned broadcast, never a clone of a publisher's own producer channel. Two local publishers at one path are redundant sources of a single broadcast: replacement/failover is an invisible splice at a group boundary instead of an unannounce/announce pair.
  • Requesting a nonexistent track through the origin now yields Unroutable after 3 rejected attempts (same as the network path did) instead of an instant NotFound; local and remote consumers behave identically.
  • A superseded source winds down through demand (its segment is capped, the sub pauses) instead of the explicit retire signal; a route flapping back re-splices the same upstream subscription instead of opening a new one.

Simplifications

The origin was carrying two mechanisms (direct publish with active/backup queues, and the route-fed front from #2241). It now has one. Concretely removed: the per-path backup queue and its promote/replace logic, the served-track map + requeue plumbing, per-route assignment queues with their parking/wake edge cases, three guard types, and the dual face/front node slots with cross-identity checks (now a single OriginBroadcast slot; the join-or-create attach race loop collapsed into one lock acquisition). Net: -372 lines workspace-wide, -321 in moq-net alone, while adding the announce-gating and offline-routability features. Each spliced track is now one task that watches the source table, rather than state smeared across queues.

Bugs found and fixed along the way

  • track::Consumer::poll_complete (new) re-locked a kio channel while the Err(Ref) closed-guard from the same poll was still alive; kio's sync API returns closure as a live lock guard, so this self-deadlocked the runtime (caught by the moq-ffi suite wedging). Fixed by reading through the guard.
  • The dispatcher burned its whole MAX_TRACK_RETRIES budget instantly on a source that had closed but whose watcher hadn't detached it yet, aborting tracks as Unroutable instead of failing over. A closed source is now parked out (no strike) until the table moves on; route ids are never reused, so this cannot wedge.

The open question: offline-but-fetchable over the wire

Nothing on the wire signals "no longer live but still fetchable" yet, and this PR deliberately doesn't invent it. Two paths from here:

  1. Today, app-supplied via Origin::dynamic() (works now): an edge handler serves fetches for cold content by calling create_broadcast(path, Route::new()) — announce=false — and filling it from storage. It's routable and cacheable in the tree without ever being announced.
  2. Later, on the wire: a liveness bit on ANNOUNCE (or a distinct GONE-but-routable state vs ANNOUNCE_END) in lite-07. The receiver's mapping is already one line: attach with .with_announce(flag) instead of hardcoded true, and everything downstream (gating, selection preferring announced sources, fetch serving) already works. Route.announce is exactly the model hook for it.

Relatedly: with announce in the route, a dynamic handler can now promote a broadcast to announced (announce=true) — dynamically accepted broadcasts are no longer condemned to invisibility. A possible follow-up is folding the dynamic served weak-cache into announce=false tree residency entirely.

Cross-package sync

  • rs/*: every crate migrated (gateways, cli, relay, stats, bench, examples); moq-ffi/libmoq reshaped (create_broadcast + set_announce replace announce/MoqAnnounce/unannounce; libmoq's moq_publish_create removed since a standalone broadcast can no longer reach an origin); hand-written py/swift/kt/go wrapper code, cpp/obs, and doc/lib/* updated.
  • Deferred: the js/net mirror. The TS side already uses producer-lifetime-driven unpublish and has no Route/guard surface, so nothing breaks; adding announce there is a follow-up.
  • Drafts unchanged: no wire format change in this PR.
  • Verified: just rs check green (moq-net 512 tests, moq-ffi 50, libmoq 35, plus all migrated crates), cargo doc -D warnings, and just test smoke.

(written by Fable 5)

…nce on Route.live

Announce/unannounce is no longer a separate origin-level concern: a broadcast's
liveness now lives on its route. broadcast::Route gains a live flag, the origin
announces a path only while its best source's route is live, and a non-live
broadcast stays reachable by exact path for subscribes and fetches. Toggling
live via set_route announces or unannounces without touching the broadcast.

origin::Producer::create_broadcast(path, route) is now the sole way content
enters an origin, returning a plain broadcast::Producer that acts as a route
source. Every source created at a path feeds one origin-owned spliced
broadcast; the best route (live first, then cost, hops, deterministic
tie-break) serves each track, and handovers resume at the first missing group.
finish() detaches immediately (graceful unannounce); a bare drop lingers so a
reconnecting session can splice in invisibly. Sessions attach announces through
the same entry point and serve tracks via a dynamic handler, the same shape the
IETF subscriber already used.

Deleted: publish_broadcast, the Publish/Broadcast guards, attach_route, and the
whole per-route Assignments/Assignment/Serving surface plus the served-track
requeue machinery. The node tree now holds a single origin-owned broadcast per
path instead of parallel face/front slots with identity checks, and the
join-or-create attach race loop collapsed into one lock acquisition.

Also fixes two bugs found while landing this: track::Consumer::poll_complete
re-locked a kio channel whose closed guard (Err(Ref)) was still held,
deadlocking the runtime; and the dispatcher burned its whole retry budget on a
closed source before its watcher could detach it, aborting tracks as
Unroutable instead of failing over.

moq-ffi/libmoq reshape accordingly (create_broadcast + set_live replace
announce/unannounce), with the hand-written py/swift/kt/go wrappers, cpp/obs,
and docs updated in tow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @kixelated, your pull request is larger than the review limit of 150000 diff characters

kixelated and others added 3 commits July 18, 2026 07:26
…blish API

Merged latest dev; #2389's new tests used the one-argument create_broadcast
and relied on an abrupt producer drop propagating immediately, which now
lingers for a reconnect by design. Pass a live route and finish() the
broadcast at the deliberate teardown, keeping the unfinalized-live-edge
assertion intact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review fixes on top of the create_broadcast/Route.live redesign:

- sync_front now snapshots and applies under the leaf lock. Two concurrent
  syncs could otherwise apply stale snapshots out of order, leaving the
  announce flag (or route advert) contradicting the current source table.
- The dispatcher parks on a source that was deliberately ended (finish/abort
  marked but handles still alive), not just a fully closed one, so the
  teardown window can't burn the retry budget and abort a routable track.
- attach_source reads the live flag from the route in hand instead of
  re-locking the just-built table.
- The lite and IETF subscribers share one SourceGuard (finish = graceful
  detach, drop = abort + linger) instead of two hand-rolled copies of the
  same contract.
- Gateways finish() their broadcast on deliberate teardown (RTMP clean stream
  end and dial, SRT finish, GStreamer stop, WHIP DELETE), restoring the
  prompt unannounce they had before the rework; a dying connection still
  lingers so a reconnect can splice in.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kixelated
kixelated enabled auto-merge (squash) July 18, 2026 14:51
@kixelated
kixelated disabled auto-merge July 18, 2026 14:51
The flag's effect is announcement, so name it that: `Route.announce` +
`with_announce()` in moq-net, `MoqRoute.announce` / `set_announce` in moq-ffi,
`moq_publish_set_announce` in libmoq, and the matching renames in the
py/swift/kt/go wrappers and docs. No behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kixelated
kixelated enabled auto-merge (squash) July 18, 2026 14:56
@kixelated
kixelated merged commit 7671feb into dev Jul 18, 2026
10 checks passed
@kixelated
kixelated deleted the claude/broadcast-routing-api-b9a743 branch July 18, 2026 22:18
@moq-bot moq-bot Bot mentioned this pull request Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant