Skip to content

feat(widget): add Cache Hit / Read / Write widgets#423

Open
Viperwow wants to merge 1 commit into
sirmalloc:mainfrom
Viperwow:feat/cache-hit-widgets
Open

feat(widget): add Cache Hit / Read / Write widgets#423
Viperwow wants to merge 1 commit into
sirmalloc:mainfrom
Viperwow:feat/cache-hit-widgets

Conversation

@Viperwow

@Viperwow Viperwow commented Jun 8, 2026

Copy link
Copy Markdown

Summary

Adds a Cache widget category that exposes prompt-cache efficiency. Today Tokens Cached sums cache_read_input_tokens (served from cache, the cache hit) and cache_creation_input_tokens (written to cache this turn) into one opaque number, so there's no way to see your cache hit rate or how much context is being re-written cold. These widgets surface exactly that, to help users measure and maximize cache reuse.

New widgets

Each widget toggles between per-turn ("last action", default) and session scope via the t keybind (mirrors the existing context-inverse used/remaining toggle and the speed-widget config pattern).

Widget Example Meaning
Cache Hit Cache Hit: 88.0% read / (read + creation) — share of cacheable context served hot vs rewritten cold
Cache Read Cache Read: 88.0k (84.5%) cache-read tokens + share of context
Cache Write Cache Write: 12.0k (11.5%) cache-creation tokens + share of context

Inline: Cache Hit: 88.0% │ Cache Read: 88.0k (84.5%) │ Cache Write: 12.0k (11.5%)

Approach

  • No new data source. Per-turn values come straight from the live status JSON (context_window.current_usage); session values are summed from the transcript in getTokenMetrics.
  • TokenMetrics gains optional cacheReadTokens / cacheCreationTokens (a split of the existing cachedTokens, which is unchanged — the Tokens Cached widget and its tests are untouched, fully backward compatible).
  • Shared helpers keep it DRY: parseCurrentUsageTokens (reused by getContextWindowMetrics), cache-scope.ts (scope toggle), cache-metrics.ts (formulas + tokens (pct) formatter).
  • Defaults: Cache Hit/Read green, Cache Write yellow (recolorable in the TUI). Widgets return null on missing data / zero denominator, like existing widgets.

Testing

  • New src/widgets/__tests__/CacheWidgets.test.ts (turn/session scope, raw mode, null/zero-denominator edges, scope-toggle keybind, preview).
  • Updated jsonl-metrics tests for the read/creation split.
  • bun test (new + updated suites green) and bun run lint (tsc + eslint, zero warnings) both pass.

Adds a new "Cache" widget category that surfaces prompt-cache efficiency,
which previously was hidden: TokensCached summed cache_read and
cache_creation into one opaque number.

New widgets, each toggling between per-turn ("last action") and session
scope via the 't' keybind:
- Cache Hit   - read / (read + creation), %
- Cache Read  - cache_read tokens with context share, e.g. "88.0k (84.5%)"
- Cache Write - cache_creation tokens with context share, e.g. "12.0k (11.5%)"

No new data source: turn scope reads context_window.current_usage from the
live status JSON; session scope sums the transcript. TokenMetrics gains
optional cacheReadTokens / cacheCreationTokens (split of cachedTokens, which
is unchanged for backward compatibility).

Defaults: Cache Hit/Read green, Cache Write yellow (recolorable in the TUI).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Viperwow

Viperwow commented Jun 8, 2026

Copy link
Copy Markdown
Author

Hi,

@sirmalloc I've been using your project for about a week and am pretty happy! Thank you for your work!

In the meantime, I've been struggling to measure cache hits during experiments with different tool setups, and that's my PoC for widgets to implement to address the issue. Co-authored with AI, but tests seem to be working, covering happy paths, and I've reviewed the code part too manually.

I hope this PR could be a good starting point for an improvement or even be merged.

@Viperwow Viperwow changed the title feat: add Cache Hit / Read / Write widgets feat(widget): add Cache Hit / Read / Write widgets Jun 8, 2026
@Viperwow

Viperwow commented Jun 9, 2026

Copy link
Copy Markdown
Author

I know that there's another way of implementing this without modifying the source code by adding custom command execution. I haven't used the recommended https://github.com/ryoppippi/ccusage and cannot tell whether it can show only the values for those items to implement it as a custom Widget.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant