fix(cache): route_cache and depends.py caches now bounded - #14
Merged
Merged
Conversation
Router.route_cache had a CachedRoute.is_valid()/ttl meant to gate cache
freshness, but nothing ever called it, and nothing ever pruned the cache
by size either -- a dynamic route like /users/{id} got one permanent
CachedRoute per distinct id ever requested. depends.py's _sig_cache,
_type_hints_cache, and _plan_cache (keyed by id(func)) had the same shape
of problem: cleanup_caches() existed to trim them but was deleted as dead
code in #9 since nothing called it either.
Both now prune on write, mirroring Velocix._prune_response_cache's
sweep-then-cap: route_cache sweeps expired entries first (finally giving
is_valid()/ttl a job), then drops oldest by created_at if still over
_ROUTE_CACHE_MAX_SIZE (1024, per method). The depends.py caches have no
expiry concept (identity-keyed, not time-keyed), so they just drop the
oldest entries via dict's insertion order once over _CACHE_MAX_SIZE (1000).
Added tests/test_cache_eviction.py: resolves/handlers well past each cap
and asserts the caches stay bounded. Fixes #8.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8. Router.route_cache had a CachedRoute.is_valid()/ttl meant to gate freshness but nothing ever called it, and nothing pruned by size either — a dynamic route like /users/{id} got one permanent CachedRoute per distinct id ever requested. depends.py's three id(func)-keyed caches had the same shape of bug (cleanup_caches() existed for exactly this but was itself deleted as dead code in #9, since nothing called it).
Both now prune on write, mirroring Velocix._prune_response_cache's sweep-then-cap: route_cache sweeps expired entries first (is_valid()/ttl finally does something), then drops oldest by created_at if still over the cap (1024/method). depends.py's caches have no expiry concept (identity-keyed), so they drop oldest by dict insertion order once over 1000.
Added tests/test_cache_eviction.py proving both stay bounded well past their caps. Verified in a fresh venv: 252/252 tests, mypy clean, ruff clean.