Skip to content

feat(net)!: extract stats publishing into moq-stats with compressed tracks#2380

Merged
kixelated merged 4 commits into
devfrom
claude/moq-net-stats-compression-d2b7a6
Jul 18, 2026
Merged

feat(net)!: extract stats publishing into moq-stats with compressed tracks#2380
kixelated merged 4 commits into
devfrom
claude/moq-net-stats-compression-d2b7a6

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

What

Splits moq_net::stats into collection and publication, and adds compressed stats tracks:

  • moq_net::stats is now collection-only. Producer is renamed Registry (Registry::new(exclude) / Registry::disabled()); the publishing task, Config, and frame types are gone from moq-net. The three duplicated counter shapes (RawCounts, TrafficTotals, BroadcastFrame) merge into one serde-both-ways Traffic type, and (SessionTotals, SessionFrame) into Presence, so the wire shape and the rollup are the same type. A new Registry::report() returns the per-(broadcast, tier) detail and prunes dead entries (the GC that used to live in the task); snapshot() is unchanged and never prunes.
  • New rs/moq-stats crate (depends on moq-net + moq-json), with both halves:
    • Producer builds the registry itself (the config prefix is wired as the registry's exclude prefix, so the stats-of-stats cycle break can't be misconfigured) and drains it on an interval. Each track is now a pair: the plain .json track is written through moq_json::snapshot with delta_ratio: 0, compression: false, which is wire-identical to the old hand-rolled output (one full JSON object per single-frame group, same field order); the new .json.z sibling uses merge-patch deltas inside a group-shared DEFLATE window, which is a fraction of the bytes since successive frames are nearly identical.
    • Consumer yields typed TrafficFrame / SessionsFrame off either flavor (ConsumerConfig { compression }), plus parse_node_path(prefix, depth, path) and traffic_track / sessions_track naming helpers, so downstream aggregators (e.g. the moq.pro dash/edge slurp) stop hand-duplicating the wire types, track tables, and announce-path parsing.
  • moq-relay: Cluster.stats is now the Registry; StatsConfig::build returns the moq_stats::Producer, which main.rs keeps alive for the run. No config surface changes.

Why

The stats frames are cumulative counter maps that barely change tick to tick: the best possible case for merge-patch deltas + a shared compression window, which moq-json already implements for catalog.json.z. moq-json depends on moq-net, so moq-net could not use it directly; splitting publication out of moq-net (which only collects now) fixes the dependency direction and deletes a hand-rolled subset of moq-json in the process (net -760 lines before the new consumer half).

Compatibility

  • The plain .json tracks are byte-compatible with the old output; .json.z siblings are additive. Existing consumers keep working unchanged.
  • Breaking pub changes are confined to moq_net::stats (rename + type merge) and moq-relay's Cluster::with_stats signature, hence the dev target.

Cross-package sync

  • No IETF draft change: the stats broadcast is an application-layer JSON format published by the relay, not part of the moq-lite/moq-transport wire protocol, and is not specified in any draft (same as catalog.json.z).
  • No js mirror: js/net has no stats module (its only "stats" are WebTransport getStats() calls); a @moq/stats consumer can follow up if needed.
  • Docs: doc/bin/relay/config.md stats section documents the .z siblings; rs/CLAUDE.md crate map updated.

Testing

  • The moq-net registry tests now run synchronously (no task to tick) and cover report()'s prune-after-final-values contract plus serde forward/backward compat for Traffic.
  • The publishing tests moved to moq-stats largely as-is; new coverage: compressed siblings exist alongside plain tracks, and a producer→consumer round trip asserts the plain and .json.z flavors decode to identical frames across an update (exercising the delta path).
  • just check and RUSTDOCFLAGS="-D warnings" cargo doc pass.

🤖 Generated with Claude Code

…racks

moq_net::stats is now collection-only: Producer becomes Registry, the
publishing task moves to the new moq-stats crate, and every stats track
gains a DEFLATE-compressed .json.z sibling encoded via moq-json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @kixelated, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

kixelated and others added 3 commits July 17, 2026 19:11
Registry::new took a bare Option<PathOwned> exclude prefix, which was a
future breaking change waiting to happen. It now takes a non_exhaustive
stats::Config (new() + with_exclude), and exclude generalizes to a list
of prefixes while we are at it.

Also gives the moq_json snapshot configs the same with_* builder setters
the stream module already has, and moq_stats::ConsumerConfig a matching
new() + with_compression, replacing default-then-field-set at the call
sites (and a clippy field_reassign_with_default warning in a test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The main-into-dev merge (fe9b782) left a double blank line in
moq-ffi's error enum, so cargo fmt --check fails on every PR merge
ref targeting dev.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kixelated
kixelated enabled auto-merge (squash) July 18, 2026 02:53
@kixelated
kixelated merged commit 7739c68 into dev Jul 18, 2026
3 checks passed
@kixelated
kixelated deleted the claude/moq-net-stats-compression-d2b7a6 branch July 18, 2026 03:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant