[aaj3f]

#2159's list view (an owner edge s p head plus the rdf:first/rdf:rest cells, with the members no longer values of their predicate) is the right model for SPARQL, and nothing here changes what it answers. The cost was in how one common shape gets read: a SPARQL ?s <p> ?o on a predicate that holds lists, with no input, was planned as a member-hiding scan followed by a read of every list on the predicate. For a predicate whose rows are mostly list members that's two full reads, and a LIMIT stopped neither of them:

• the hiding scan fills its batch, so when the only rows are members, one next_batch walks the whole predicate and returns nothing (binary_scan.rs, the hide_members && o_i != u32::MAX skip);
• then read_lists_in reads the predicate again through range_with_overlay as flakes, groups the members by owner, and list_rows emits every list triple in one batch.

So SELECT ?s ?t WHERE { ?s ex:tags ?t } LIMIT 5 cost exactly what the unlimited query did, and both grew with the ledger: about 22× a list-free ledger at 300k subjects for the LIMIT, and 9.4× for the full read.

What changed

One pass. For that shape the list triples are exactly the owner edges, so the scan emits them itself. Where the hiding scan skips a member, BinaryScanOperator::listing_owners instead emits that list's owner edge, once per list, naming the head cell through the request's ListView. The planner builds that scan (DatasetOperator::scan_listing_owners) in place of the hiding scan plus ListAugmentOperator when the pattern has no input (list_view::owner_edges_answer: ?s <p> ?o, two distinct variables, <p> not rdf:first/rdf:rest, no datatype constraint). Everything the scan's row loop does differently sits inside the existing member-skip branch, which is only taken in leaflets that hold list rows, so a scan of anything else runs the same code it did.

Stopping at the batch. A cursor batch is a whole leaflet, and the scan converts all of it before returning, so a scan listing owners would name every owner the leaflet held even when a LIMIT needed five rows. It now stops at the first owner edge past its batch budget, keeps the rest of the leaflet, and resumes there on the next batch (again only in the member branch).

Naming lists by id. Resolving every owner's subject through the dictionary and hashing its strings into the list table was most of the remaining cost (about 9k instructions and seven allocations per list). The list table now also names a list by (graph, s_id, p_id), and a scan over persisted ids (no novelty) names the lists it meets that way, keeping the subject encoded the way it binds stored rows; a cell is resolved back to its owner's terms only if something decodes it (e.g. a following ?l rdf:first ?x). Once a plan holds such a scan, a list named by its terms elsewhere in the request is entered under its ids too, so a list gets one name however it's met. With novelty present the scan resolves subjects as before.

Planner precondition. needs_empty_seed now admits Pattern::ListView alongside Triple/Values/Bind. Without that, a WHERE that opens on a list-view triple got an EmptyOperator seed, so its first pattern was planned as a join and never took the one-pass path. This also gives ?s ?p ?o on a list-bearing ledger its first scan back.

List predicates in one order. Not introduced here, but the label tests turned it up: with no subject, read_lists_in walked the view's list predicates in a HashSet's order, whose hasher differs per request, so lists on two predicates could swap labels between two runs of the same query (?c rdf:first ?x, say). It now reads them sorted.

Not changed: patterns on rdf:first/rdf:rest, a variable predicate, and list-view triples joined to an input keep #2159's tee-and-augment path.

Answers

The new tests ask every case twice: as written, where the first list-view triple has no input and is read in one pass, and after VALUES ?z { 1 } FILTER(?z = 1), which gives every list-view triple an input and so routes it through the existing tee and list read in the same binary. The plans are checked too, so a case can't pass by comparing a plan with itself. The ledger mixes lists with stored values on one predicate, has two lists on one subject, a list in a named graph, a stored blank node and a predicate only novelty holds; it's read in three states (novelty only, indexed, indexed with novelty), over the default graph, a named graph and their union, in GRAPH ?g, with fast paths on and off. A second ledger mixes 6k stored values with 2k lists on one predicate, so the scan stops and resumes leaflets mid-way. The W3C suite (testsuite-sparql) passes unchanged.

Cell labels are numbered in the order a request reads its lists, so a GROUP BY ?o (which reads the predicate in object order) may number them differently from main; it's the same RDF up to a consistent renaming of the cells. Everywhere the scan reads in subject order, the tests hold the labels to exactly the ones the list read gives. Within one result a cell has one label wherever it appears (the owner edge's object, and the cell's own rdf:first/rdf:rest rows), the same query gives the same labels on every run, and the labels stay in the list view's own _:l… space, apart from stored blank nodes (_:fdb-…) and BNODE()'s.

Numbers

Instructions per query (release, proc_pid_rusage), main 42bb9646c against this branch, on a ledger where every tenth subject owns a two-item ex:tags list and a two-item ex:mixed list (the others hold one plain ex:mixed value), beside a list-free ledger holding the same values as plain multi-values. Medians, three interleaved passes at 3k/30k, one at 300k.

query (list ledger)	N	main	this PR	÷ main	÷ list-free
?s ex:tags ?t LIMIT 5	30k	40.9M	2.67M	0.07	0.56
	300k	435.5M	4.16M	0.01	0.22
?s ex:tags ?t	30k	41.0M	6.51M	0.16	1.37
	300k	435.3M	65.8M	0.15	1.42
… ORDER BY ?s LIMIT 5	300k	478.4M	230.0M	0.48	0.89
?s ex:mixed ?m	300k	2,493M	269.6M	0.11	1.06
?s ex:mixed ?m LIMIT 5	30k	17.98M	3.01M	0.17	0.16
	300k	17.84M	2.99M	0.17	0.15
?s ?p ?o	30k	548.4M	396.9M	0.72	—

Allocations drop from about 12 per list to about one (the cell's name), and a LIMIT over a list predicate now costs one batch whatever the ledger's size. Shapes that don't touch lists (non-list predicates, joins, COUNT(*), GROUP BY, plain ?s ?p ?o, and JSON-LD on both ledgers) stay within −1.7% to +0.4% of main, with identical allocations.

Tests

it_sparql_list_single_pass (own binary, since it flips the fast-path switch):

• one_pass_answers_as_the_list_view_with_an_input — the differential and plan checks above (labels compared exactly where the read is in subject order), including one list met twice in a request (?s ex:items ?o . ?s2 ex:items ?o, where the second pattern reads the lists by their terms and the first names them by id, so the head cells only join if both give a list one name);
• cell_labels_are_one_name_per_cell_and_stable — an owner edge joined to its first cell, a UNION holding each head twice, six repeat runs, and labels apart from stored blank nodes and BNODE();
• a_large_mixed_predicate_answers_as_with_an_input — the mixed ledger across batches, as with an input, labels included, and LIMIT 5 well under the full read's fuel;
• one_pass_answers_by_hand — owner edges beside stored values, by hand;
• a_limit_over_lists_reads_one_batch — LIMIT 5 fuel over 16k and 48k lists is the same, and under half the full read's.

On the branch, each of these changes is caught when reverted: dropping the id alias in ListView::cell loses rows in the one-list-met-twice case; dropping ListView from needs_empty_seed fails the plan and cost checks; dropping the batch stop puts the mixed LIMIT 5 at the full read's fuel; and reading the predicates in HashSet order changes labels between runs.

Since this reshapes how the list view gets read, I'd especially value a close look at the id-keyed list table (ListView::scanned_head / name_lists_by_id and the alias in cell) and the batch stop in batch_to_bindings, and on whether there's a shape I'm missing where a scan's owner edge and the old list read could disagree. Happy to talk through any of it.

Fwiw, it_transact_update_indexed::a_multi_op_update_ends_where_its_last_op_leaves_a_datetime_held_in_novelty failed twice under heavy load with a WAL append ENOENT in create_ledger (2 of 5 full runs here, 0 of 5 on main), then not at all in 20 more full-suite rounds of each run side by side, so it's not attributed to this change or to main.