Skip to content

perf(chat): block-level transcript units and windowed history - #318

Merged
su-fen merged 1 commit into
mainfrom
features
Jul 28, 2026
Merged

perf(chat): block-level transcript units and windowed history#318
su-fen merged 1 commit into
mainfrom
features

Conversation

@su-fen

@su-fen su-fen commented Jul 28, 2026

Copy link
Copy Markdown
Member

Fixes visible scroll jank and render lag in the desktop GUI on very long conversations. Two root causes: one assistant reply was a single virtualized row (a long reply meant one enormous DOM subtree measured and re-rendered as a unit), and opening a conversation hydrated its entire lifetime history into runtime state.

Rendering

  • Assistant replies split into block-level render units — each text/thinking/tool/hostedSearch block, plus a footer, is its own virtualized row. Only the mutable live tail stays force-mounted; completed units of a streaming reply virtualize normally and are reused by identity so streaming does not churn finished blocks.
  • New cost-budgeted overscan replaces the force-mount-everything-live extractor: one warm neighbor per side, then a small render-cost budget, so a giant Markdown unit consumes the budget alone instead of dragging expensive offscreen siblings into WKWebView.
  • @tanstack/react-virtual 3.14.6 → 3.14.8 with directDomUpdates in transform mode, so item positioning and total size bypass React commits. Row gap moves into a per-row spacer; the measurements-LRU layout key is versioned so stale pre-split measurements cannot be restored.
  • ChatTranscript drops the Base UI ScrollArea for a native overflow viewport, removing its per-scroll geometry, computed-style and inherited CSS-variable work from WebKit's hot path.
  • Long fenced code blocks collapse to a 24-line plain-text preview (no Shiki); row estimates count the same 24 visible lines per fence so estimates match what is rendered.
  • LazyCollapse gains retainWhileClosed: running tools/searches keep their body while collapsed, settled ones release it immediately.

History

  • Conversations load in 360-message windows with revision-based optimistic concurrency; scrolling to the top pages earlier history in with scroll anchoring. The two-phase open (paint active segment, then hydrate the full record at idle) is removed entirely along with its idle scheduler, hydrating phase and openFullFailed state; the shared open controller is now single-phase on both ends.
  • History JSON parsing moves to a Web Worker.
  • Edit-resend becomes an atomic Rust transaction (revision CAS → truncate → FTS rebuild → consistency check → new window) instead of client-side truncation followed by a persist. Subagent invalidation and pruning move into the single send preflight shared with WebUI-relayed resends.
  • Runtime state keeps a small persistence cursor instead of a full persisted-state snapshot per conversation.

Review fixes folded in

  • Mirror drift: mirrored files had only been updated on the GUI side. Markdown.tsx, rowEstimates.ts, openController.ts and the new markdownCodeBlockPolicy are byte-identical again, with the WebUI app, i18n keys and tests adapted to the single-phase controller.
  • Permanent persist failure: normalizeConversationState now subtracts the messages normalizeSegment filters out from the wire totalMessageCount. Legacy conversations carrying non-runtime messages would otherwise fail the Rust segment-sum consistency check on every persist, forever.
  • HostedSearchGroupView's new isLive prop was never passed, so a running search still dropped its body on collapse.
  • The history parser leaked a pending entry when postMessage threw and had no onmessageerror handler (a deserialization failure hung the open with no error).
  • Dead code removed (flattenSegmentsToTimeline, three unused re-exports, a pass-through helper, a duplicated timestamp reader); the 360 window size and the window-to-state assembly are single-sourced; chat_history_get is no longer a webview command (only the gateway bridge calls it).

Verification

  • GUI frontend: 1359 tests pass; WebUI: 477 tests pass; Rust: 658 tests pass.
  • tsc --noEmit clean on both frontends; biome reports no errors on either.
  • node scripts/check-mirror.mjs passes (115 files).
  • rustfmt --check clean on all touched Rust files.

Known trade-offs (intentional, not regressions)

  • Paging up is silently unavailable while a run streams — appends null the transcript revision and the CAS conservatively declines rather than risk a torn window.
  • Renaming a conversation no longer bumps updated_at (keeps open windows' revisions valid), so a rename no longer floats the conversation to the top of the sidebar.
  • Reopening a conversation whose active segment is checkpoint-born with zero messages does not render its summary card for that session.

Fixes visible scroll jank and render lag in the desktop GUI on very long
conversations. Two root causes: one assistant reply was a single
virtualized row (a long reply meant one enormous DOM subtree measured and
re-rendered as a unit), and opening a conversation hydrated its entire
lifetime history into the runtime state.

Rendering:
- Assistant replies split into block-level render units (rowModel
  buildAssistantUnits): each text/thinking/tool/hostedSearch block, plus
  a footer unit, is its own virtualized row. Only the mutable live tail
  stays force-mounted; completed units from the same streaming reply
  virtualize normally. Live builds reuse settled unit objects by identity
  through liveUnitCache so streaming does not churn finished blocks.
- New cost-budgeted overscan (renderUnitRangeExtractor) replaces the
  force-mount-everything-live extractor: one warm neighbor per side, then
  a small render-cost budget, so a giant Markdown unit consumes the
  budget alone instead of dragging expensive offscreen siblings into
  WKWebView. The GUI copy of liveRangeExtractor is deleted (the WebUI
  keeps its own; it is no longer a mirrored file).
- @tanstack/react-virtual 3.14.6 -> 3.14.8 with directDomUpdates in
  transform mode plus containerRef, so item positioning and total size
  bypass React commits. Row gap moves from the virtualizer option into a
  per-row spacer; the measurements-LRU layout key is versioned so stale
  pre-split measurements cannot be restored.
- ChatTranscript drops the Base UI ScrollArea for a native overflow
  viewport, removing its per-scroll geometry, computed-style and
  inherited CSS-variable work from WebKit's hot path.
- Long fenced code blocks collapse to a 24-line plain-text preview (no
  Shiki) behind markdownCodeBlockPolicy; rowEstimates counts the same 24
  visible lines per fence so estimates match what is rendered.
- LazyCollapse gains retainWhileClosed: running tools/searches keep their
  body while collapsed, settled ones release it immediately.

History:
- Conversations load in 360-message windows with revision-based
  optimistic concurrency; scrolling to the top pages earlier history in
  with scroll anchoring. The two-phase open (paint active segment, then
  hydrate the full record at idle) is removed entirely along with its
  idle scheduler, "hydrating" phase and openFullFailed state; the shared
  openController is now single-phase on both ends.
- History JSON parsing moves to a Web Worker (chatHistoryParser).
- Edit-resend becomes an atomic Rust transaction
  (chat_history_replace_from_message: revision CAS, truncate, FTS
  rebuild, consistency check, new window) instead of client-side
  truncation followed by a persist; subagent invalidation and pruning
  move into the single send preflight shared with WebUI-relayed resends.
- Runtime state keeps a small persistence cursor instead of a full
  persisted-state snapshot per conversation.

Review fixes on top of the base work:
- Mirrored files were only updated on the GUI side: Markdown.tsx,
  rowEstimates.ts, openController.ts and the new markdownCodeBlockPolicy
  are now byte-identical again, with GatewayApp, the WebUI i18n keys and
  the WebUI tests adapted to the single-phase controller.
- normalizeConversationState now subtracts the messages normalizeSegment
  filters out from the wire totalMessageCount. Legacy conversations
  carrying non-runtime messages would otherwise fail Rust's segment-sum
  consistency check on every persist, permanently.
- HostedSearchGroupView's new isLive prop was never passed, so a running
  search still dropped its body on collapse.
- chatHistoryParser leaked a pending entry when postMessage threw and had
  no onmessageerror handler (a deserialization failure hung the open).
- Dead code removed: flattenSegmentsToTimeline, three unused AssistantBubble
  re-exports, the branch_stable_message_id pass-through, a duplicated
  timestamp reader; chat_history_get is no longer a webview command (only
  gateway_bridge calls it). The 360 window size and the window-to-state
  assembly are single-sourced.
@su-fen su-fen changed the title chat: transcript performance overhaul, windowed history, and reliable desktop ingress mirror perf(chat): block-level transcript units and windowed history Jul 28, 2026
@su-fen
su-fen merged commit 503dc6e into main Jul 28, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant