feat(widget): add Cache Hit / Read / Write widgets#423
Conversation
Adds a new "Cache" widget category that surfaces prompt-cache efficiency,
which previously was hidden: TokensCached summed cache_read and
cache_creation into one opaque number.
New widgets, each toggling between per-turn ("last action") and session
scope via the 't' keybind:
- Cache Hit - read / (read + creation), %
- Cache Read - cache_read tokens with context share, e.g. "88.0k (84.5%)"
- Cache Write - cache_creation tokens with context share, e.g. "12.0k (11.5%)"
No new data source: turn scope reads context_window.current_usage from the
live status JSON; session scope sums the transcript. TokenMetrics gains
optional cacheReadTokens / cacheCreationTokens (split of cachedTokens, which
is unchanged for backward compatibility).
Defaults: Cache Hit/Read green, Cache Write yellow (recolorable in the TUI).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Hi, @sirmalloc I've been using your project for about a week and am pretty happy! Thank you for your work! In the meantime, I've been struggling to measure cache hits during experiments with different tool setups, and that's my PoC for widgets to implement to address the issue. Co-authored with AI, but tests seem to be working, covering happy paths, and I've reviewed the code part too manually. I hope this PR could be a good starting point for an improvement or even be merged. |
|
I know that there's another way of implementing this without modifying the source code by adding custom command execution. I haven't used the recommended https://github.com/ryoppippi/ccusage and cannot tell whether it can show only the values for those items to implement it as a custom Widget. |
Summary
Adds a Cache widget category that exposes prompt-cache efficiency. Today
Tokens Cachedsumscache_read_input_tokens(served from cache, the cache hit) andcache_creation_input_tokens(written to cache this turn) into one opaque number, so there's no way to see your cache hit rate or how much context is being re-written cold. These widgets surface exactly that, to help users measure and maximize cache reuse.New widgets
Each widget toggles between per-turn ("last action", default) and session scope via the
tkeybind (mirrors the existingcontext-inverseused/remaining toggle and the speed-widget config pattern).Cache Hit: 88.0%read / (read + creation)— share of cacheable context served hot vs rewritten coldCache Read: 88.0k (84.5%)Cache Write: 12.0k (11.5%)Inline:
Cache Hit: 88.0% │ Cache Read: 88.0k (84.5%) │ Cache Write: 12.0k (11.5%)Approach
context_window.current_usage); session values are summed from the transcript ingetTokenMetrics.TokenMetricsgains optionalcacheReadTokens/cacheCreationTokens(a split of the existingcachedTokens, which is unchanged — theTokens Cachedwidget and its tests are untouched, fully backward compatible).parseCurrentUsageTokens(reused bygetContextWindowMetrics),cache-scope.ts(scope toggle),cache-metrics.ts(formulas +tokens (pct)formatter).green, Cache Writeyellow(recolorable in the TUI). Widgets returnnullon missing data / zero denominator, like existing widgets.Testing
src/widgets/__tests__/CacheWidgets.test.ts(turn/session scope, raw mode, null/zero-denominator edges, scope-toggle keybind, preview).jsonl-metricstests for the read/creation split.bun test(new + updated suites green) andbun run lint(tsc + eslint, zero warnings) both pass.