Skip to content

WIP: ReceiptStore Add littidx backend (LittDB bodies + pebble tag index)#3620

Open
Kbhat1 wants to merge 3 commits into
mainfrom
littidx
Open

WIP: ReceiptStore Add littidx backend (LittDB bodies + pebble tag index)#3620
Kbhat1 wants to merge 3 commits into
mainfrom
littidx

Conversation

@Kbhat1

@Kbhat1 Kbhat1 commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

No description provided.

littidx stores receipt bodies in LittDB — point lookup by tx hash, where
litt's keymap IS the tx-hash index (eth_getTransactionReceipt is one lookup,
no separate index) — and supports eth_getLogs via a small pebble tag index:
one empty-valued key per log tag (address + each topic position), queried by
seeking the matching (block, tag) entries, intersecting across criteria
dimensions, and point-reading only the matching receipts from litt. getLogs
never decodes non-matching receipts, and receipt bytes are never duplicated.

- Bodies live in litt immutable segments, so large values never enter LSM
  compaction and expiry reclaims whole segments.
- Tag key: 't'+block+kind+tag+txIndex+firstLogIndex+txHash -> nil. Pruning a
  block range is a single pebble range tombstone (adds DeleteRange to the
  pebbledb wrapper).
- Retention: litt TTL reclaims bodies; a read-time floor bounds visibility to
  KeepRecent regardless of GC timing.

Wired as receipt backend "littidx", reusing existing helpers (matchLog,
getLogsForTx, key consts). Tests cover read/write, getLogs semantics
(address/topic AND/OR/wildcard, range bounds), multi-log numbering, reopen
persistence, and prune.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cursor

cursor Bot commented Jun 22, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
New default-selectable storage path for EVM receipts and log queries; correctness and retention depend on Litt TTL, async flush, and tag-index pruning staying aligned with KeepRecent.

Overview
Adds a littidx receipt-store backend selectable via receipt-store.rs-backend, alongside the existing Pebble path.

Receipt payloads are stored in LittDB with tx-hash secondary keys for single-lookup GetReceipt, while a separate Pebble tag index powers FilterLogs / eth_getLogs via address/topic tag keys, candidate intersection, and point reads back into Litt. Writes commit Litt values first, then batch index tags and version metadata (m:latest / m:earliest); retention uses Litt TTL plus block-range index pruning (new DeleteRange on Pebble for O(1) tag tombstones), a read-time earliest-version floor, background flush, and legacy KV fallback for pre-migration receipts.

Factory wiring in NewReceiptStore, config validation, and a focused test suite (read/write, filters, multi-part blocks, reopen, prune) complete the change.

Reviewed by Cursor Bugbot for commit 0771fd0. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedJun 23, 2026, 4:16 AM

}
}
}
firstLogIndex += uint32(len(record.Receipt.Logs)) //nolint:gosec // log counts fit within uint32

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multi-part block log index

High Severity

When the same block is written in multiple SetReceipts calls (extra litt parts), stageTagKeys always starts firstLogIndex at zero for that batch. Tag keys and getLogsForTx then use wrong block-wide log indices for later parts, so eth_getLogs can return incorrect or colliding log.Index values even when filtering matches.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7383fb0. Configure here.

@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.49746% with 132 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.17%. Comparing base (4d449ae) to head (0771fd0).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
sei-db/ledger_db/receipt/litt_receipt_store.go 59.13% 64 Missing and 30 partials ⚠️
sei-db/ledger_db/receipt/litt_tag_index.go 76.62% 20 Missing and 16 partials ⚠️
sei-db/db_engine/pebbledb/db.go 50.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3620      +/-   ##
==========================================
- Coverage   59.01%   58.17%   -0.85%     
==========================================
  Files        2224     2152      -72     
  Lines      182699   174544    -8155     
==========================================
- Hits       107823   101536    -6287     
+ Misses      65197    63978    -1219     
+ Partials     9679     9030     -649     
Flag Coverage Δ
sei-chain-pr 65.18% <66.49%> (?)
sei-db 70.41% <ø> (ø)
sei-db-state-db ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-db/config/receipt_config.go 77.77% <100.00%> (ø)
sei-db/ledger_db/receipt/receipt_store.go 66.83% <100.00%> (+7.58%) ⬆️
sei-db/db_engine/pebbledb/db.go 71.30% <50.00%> (-0.77%) ⬇️
sei-db/ledger_db/receipt/litt_tag_index.go 76.62% <76.62%> (ø)
sei-db/ledger_db/receipt/litt_receipt_store.go 59.13% <59.13%> (ø)

... and 75 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Review cleanups for the littidx backend — no behavior change:

- Drop the dead u32-length framing in writeBlock. Every read fetches a
  receipt via its tx-hash secondary key (a value sub-range alias), and the
  part value is never read whole, so the per-receipt length prefixes were
  unused. Saves 4 bytes/receipt and clarifies the format.
- Remove the redundant Exists check in writeBlock: nextPartIndex already
  returns the first free part slot, so the check was always false.
- Replace deleteIndexRange's iterate-and-point-delete fallback (dead code —
  the index is always pebble) with a required rangeDeleter assertion that
  errors loudly if a future backend lacks range delete.
- Rename the orphaned ledgerTxIndexLen to littTxIndexLen and name the log
  index width (littLogIndexLen) so the tag-key suffix layout is explicit.
- Document that firstLogIndex restarts per part for a block split across
  SetReceipts calls (legacy migration only) — harmless, the RPC layer
  recomputes log indices from canonical block data.
- Add TestLittIdxBlockWideLogIndex covering firstLogIndex accumulation
  across transactions (previously only single-tx numbering was tested).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
for {
pruneBefore := s.latestVersion.Load() - s.keepRecent
if pruneBefore > 0 {
if err := s.pruneBlocksBelow(uint64(pruneBefore)); err != nil {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Background prune retains extra block

Medium Severity

Background pruning passes latestVersion - keepRecent directly into pruneBlocksBelow, which treats that value as the first visible block. The pebble receipt pruner removes versions through that same difference inclusive, so one additional block stays queryable via GetReceipt and FilterLogs than KeepRecent intends.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6531f47. Configure here.

}
}
if err := batch.Commit(dbtypes.WriteOptions{}); err != nil {
return err

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Index commit failure leaves orphan receipts

Medium Severity

SetReceipts writes receipt bodies to LittDB inside writeBlock before the pebble tag batch commits. If batch.Commit fails after one or more blocks were persisted, those receipts remain hash-addressable while their log tags were never indexed, so GetReceipt can succeed while FilterLogs omits the same transactions.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6531f47. Configure here.

// and stall throughput, so raise the key-count and key-file caps past a
// retention window and let only the value-file size bind segment seals.
littConfig.MaxSegmentKeyCount = 100_000_000
littConfig.TargetSegmentFileSize = 512 << 20

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit-shift segment size literal

Low Severity

TargetSegmentFileSize is set with 512 << 20 even though this file already imports unit and uses unit.GB nearby. sei-db convention is byte sizes via unit.MB / unit.GB, not bit-shift literals.

Fix in Cursor Fix in Web

Triggered by learned rule: sei-db: use unit.MB/GB constants for byte sizes, not bit-shift literals

Reviewed by Cursor Bugbot for commit 6531f47. Configure here.

@Kbhat1 Kbhat1 changed the title feat(receipt): add littidx backend (LittDB bodies + pebble tag index) ReceiptStore Add littidx backend (LittDB bodies + pebble tag index) Jun 22, 2026
@Kbhat1 Kbhat1 changed the title ReceiptStore Add littidx backend (LittDB bodies + pebble tag index) WIP: ReceiptStore Add littidx backend (LittDB bodies + pebble tag index) Jun 22, 2026
…iest

Address independent review findings:

- Make SetEarliestVersion monotonic (only move the retention floor forward),
  matching SetLatestVersion in the same file so the floor can never move
  backward and re-expose pruned blocks.
- Add TestLittIdxMultiPart: a block written across two SetReceipts calls
  exercises nextPartIndex and multi-part read/filter (previously untested).
- Add TestLittIdxLegacyFallback: GetReceipt falling back to the legacy KV
  store for a receipt absent from litt (the one untested behavioral branch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 5 total unresolved issues (including 4 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0771fd0. Configure here.

}
}
if err := batch.Commit(dbtypes.WriteOptions{}); err != nil {
return err

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failed flush appends duplicate parts

High Severity

SetReceipts writes each block to LittDB via receipts.Put before the pebble index batch commits. If writeBlock or batch.Commit fails after a successful Put, a retry uses nextPartIndex and appends another part for the same block instead of replacing the failed write, duplicating receipt data and secondary tx-hash keys.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 0771fd0. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant