Skip to content

perf(hub): stop heartbeat replay from scanning session history - #1745

Open
ZJUFangzh wants to merge 1 commit into
tiann:mainfrom
ZJUFangzh:fix/hub-heartbeat-scan-1689
Open

perf(hub): stop heartbeat replay from scanning session history#1745
ZJUFangzh wants to merge 1 commit into
tiann:mainfrom
ZJUFangzh:fix/hub-heartbeat-scan-1689

Conversation

@ZJUFangzh

Copy link
Copy Markdown

Summary

  • query the immediate replay queue before checking the OpenCode clear delivery gate, so the common empty-queue heartbeat path does not scan and parse session metadata
  • add a matching partial index for immediate queued-message lookups
  • migrate the internal SQLite schema from v25 to v26 and cover the query plan in a migration test

Fixes #1689.

Root cause

Each CLI session-alive heartbeat calls replayImmediateQueuedMessages(). The existing order checked isOpenCodeClearDeliveryGated() first, which scans all session metadata in the namespace even when there is nothing to replay. With many connected sessions, that turns the 2-second heartbeats into repeated namespace-wide scans.

Moving the queue lookup first removes that scan from the empty-queue path. The queue lookup itself previously had no index matching all of its predicates, so this PR also adds idx_messages_immediate_queued on (session_id, seq) with the exact partial predicate used by the query.

Production validation

The same change is running on a self-hosted v0.29.0 hub with 123 sessions and 322,007 messages:

  • SQLite read throughput: about 117 MB/s -> about 3 KB/s
  • /health: timeout/unresponsive -> 3-4 ms
  • socket receive queues drained (Recv-Q=0)
  • no service restarts observed after deployment

Testing

  • bun typecheck
  • bun run test
  • focused Hub suite: 94 tests covering message replay, OpenCode clear handling, and v23/v25 migrations
  • migration test verifies schema v25 -> v26 and confirms EXPLAIN QUERY PLAN uses the new partial index

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • None.

Summary

  • Review mode: initial
  • No issues found in the added or modified lines. Residual operational risk: the v25→v26 upgrade builds the partial index synchronously during Hub startup; this review did not execute the PR code to measure migration time on a large database.

Testing

  • Not run (automation). At review time, the integration check passed; the main test and PR-review checks were pending.

HAPI Bot

@alexistojims-sys alexistojims-sys mentioned this pull request Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(hub): sessions table is full-scanned several times per second; hub pegs a core and all requests queue as session count grows

1 participant