Skip to content

fix(query): a planner stats view is keyed by the index root it came from - #2200

Open
aaj3f wants to merge 4 commits into
mainfrom
test/stats-cache-isolated
Open

aaj3f wants to merge 4 commits into
mainfrom
test/stats-cache-isolated

Conversation

@aaj3f

@aaj3f aaj3f commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

This started as a test flake in the store-less stats cache, and looking at that cache's key turned up a production case it can't tell apart, so the PR does both.

The key. The planner's stats view is cached under the ledger id, the snapshot's t, the query t, the overlay's epoch and a few flags, and none of those name the index the stats came from. A ledger dropped and created again under its name, then indexed to the same t with different data, matches that key, and the Fluree instance's leaflet cache outlives the drop, so the recreated ledger gets planned with the dropped ledger's stats. That's not just a worse plan: those stats can license folding FILTER(?x = ?y) into a join on the term, and on the recreated ledger, where ex:a is 1 and ex:b is 1.0, the filter then returns no rows instead of one. it_stats_view_identity does exactly that.

A snapshot now carries the content id of the index root its stats, schema and annotation arena were decoded from (LedgerSnapshot::from_root(root_id, bytes); every load path already had the id in hand, so nothing is hashed), and the key folds it in. Two loads of one root agree, and a different root never does. A snapshot that has stats but no root id to name them, which only tests build, gets its view built rather than shared. The store-backed key keeps the overlay's epoch(), as before, and the store-less cache still adds the content version, as before.

I first tried keying by the index store's id, which works for the collision but is new on every load, and a historical read loads its store again on every call, so it was rebuilding the view on every query. I measured the root key against main on each lane that reloads, counting stats-view builds per 20 queries (5,000 subjects, index at t=2):

Lane main store id root id
head, ledger cache 0 0 0
db_at_t below the index t 0 40 0
db_at_t at the index t 0 40 0
db_at_t above the index t (novelty replayed) 0 20 0
db() with no ledger cache, with and without novelty 0 20 0

So every lane that hits on main still hits. The key is also hashed field by field now instead of format!-ed into a String first, which makes the hit itself cheaper: 0.24× main's instructions per lookup (about 50 ns vs 160 ns, 2M lookups in a dev-fast build, 5 alternating rounds), with or without a store. Loading a 1.3 MB root is flat (instructions 1.0003× main), since the id is just carried along.

One thing I left alone: a replayed novelty keeps the old epoch-based term on the store-backed path, which is what lets the above-t lane hit, but it also means two divergent clones of one novelty at the same t would share a slot there, same as on main. I haven't found a path that reaches that. Giving a replayed novelty a deterministic content version, from the commit at its t and the index t, would let that key use the content version too, but it changes what that version promises to every other cache that keys on it, so it probably wants its own look rather than riding along here.

The flake. stats_cache::tests::storeless_stats_view_is_reused_at_the_same_content_version failed once in a full fluree-db-query lib run under load, with "an unchanged overlay must hit the cache, not rebuild", and passed alone. The store-less views live in one process-wide eight-slot LRU, lib tests run as threads of one process, and 14 of the 15 views a full lib run puts there come from stats_cache's own tests, which run side by side. Eight insertions between the reuse test's two calls evict its slot. Its sibling, storeless_stats_view_misses_when_content_version_diverges, has the quieter half: under the same interleaving it passes against a cache that ignores the content version.

The LRU is now a small type, StorelessStatsCache. cached_stats_view_for_db keeps its signature and owns the one process-wide instance, as before, and every test in the module builds through its own. A new test pins the LRU itself (eight views fit, a hit refreshes one, the ninth insertion displaces the least recently used). binary_range's an_empty_overlay_probe_leaves_the_translation_lru_alone had the same shape, so it now reads a record of its store's insertions that nothing evicts, rather than an eight-slot LRU that churn can empty.

Proof.

  • it_stats_view_identity fails with stats_cache.rs as on main, and when the root id is left out of the key; it passes with the fix. Leaving the no-root-id check out fails stats_with_no_index_root_to_name_them_are_not_shared, and leaving the content version out of the store-less key fails the miss test.
  • With the stats_cache module looped 16,000 times, 8 at once, main failed the reuse test 25 times; this branch fails it 0 times. With eight other views pushed into the process-wide cache between the two calls, main fails it 5/5 and this branch passes 5/5.
  • The tests still catch what they're for: the reuse test fails with eight insertions into its own cache and with a cache that never hits, the miss test fails when a slot ignores the version, and the LRU test fails with nine slots or with a hit that doesn't refresh.

aaj3f added 4 commits October 26, 2026 18:38
… own

The store-less stats views of memory-mode ledgers live in one process-wide
eight-slot LRU. Lib tests run as threads of one process, and 14 of the 15
views a full lib run inserts into that cache come from this module's own
tests, which libtest runs side by side. Eight insertions landing between
the reuse test's two calls evict its slot: the second call rebuilds and
the `Arc::ptr_eq` assertion fails. The same interleaving lets the
content-version miss test pass against a cache that ignores the version,
since its first view is gone before the second call could be served it.

The LRU is now a small type. `cached_stats_view_for_db` keeps its
signature and owns the one process-wide instance as a function-local
static, as before; the body moves to `stats_view_for_db_in`, which takes
the cache. Every test in the module builds through its own instance, and
a new test pins the LRU itself: eight views fit, a hit refreshes one, and
the ninth insertion displaces the least recently used.

No production behavior change.
…empty

`an_empty_overlay_probe_leaves_the_translation_lru_alone` counted this
store's entries in the process-wide eight-slot translation LRU. Lib tests
share that cache across threads, so eight insertions from other tests
between the probes and the count could evict an entry a regression had
just added, and the test would pass on the regression.

Each insertion now also lands, under `cfg(test)`, in a set of store ids
nothing evicts, and the test asserts its store is absent from that.
Production code is unchanged.
The planner stats view was cached under (ledger id, snapshot `t`, query
`t`, overlay epoch, a few flags), which doesn't name the index the stats
came from. A ledger dropped and created again under its name, indexed
to the same `t` with different data, matches that key, and the
`Fluree` instance's leaflet cache outlives the drop, so the recreated
ledger was planned with the dropped ledger's stats. Those can license
folding `FILTER(?x = ?y)` into a join on the term, which loses rows
whose values are equal but whose terms differ (`1` and `1.0`).
`it_stats_view_identity` drops and recreates a ledger and checks that
filter.

A snapshot now carries the content id of the index root its stats,
schema and annotation arena were decoded from: `LedgerSnapshot::from_root`
takes it with the bytes, `LedgerSnapshotMetadata` carries it, and every
production load already has it. The key folds it in, so two loads of one
root agree and a different root never does. A snapshot with stats but no
root id (only tests build one through `from_root_bytes`) gets its view
built rather than shared. The store-backed key keeps the overlay's
`epoch()`, as before: a historical read replays its novelty on every
call and reaches the same epoch, so it stays on one view. The store-less
cache still adds the content version, as before.

The key's fields are hashed directly instead of being formatted into a
string, which makes a hit 0.24x main's instructions. Stats-view builds
per 20 queries are 0, as on main, on each reload lane: head, `db_at_t`
below, at and above the index `t`, and `db()` with no ledger cache.
Loading a root is flat.
@aaj3f aaj3f added bug Something isn't working as expected area:query Query execution, planning, fast paths, overlay, result formatting labels Oct 28, 2026
@aaj3f
aaj3f requested a review from bplatz October 28, 2026 11:32

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:query Query execution, planning, fast paths, overlay, result formatting bug Something isn't working as expected

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant