TRACKING PR OF 0.2.* DEVELOPMENT - #57
Open
williamwutq wants to merge 15 commits into
Open
Conversation
Backport of #51 to the 0.2.x line. An interrupted non-tail-shrink `realloc` in `CheckedSlabBStackAllocator` (Rust) / `checked_slab_bstack_allocator_realloc` (C) could make recovery corrupt an unrelated live allocation: the shrink committed the block's smaller count before scrubbing the excess into the free list, so a fault in between left the excess holding stale payload while the header already claimed the smaller span. `recover`'s linear scan then read those orphaned bytes as a valid multi-block in-use marker, strode past a neighbouring live allocation's header, and reclaimed its interior as leaked blocks -- writing free-list links over live data. Invert the order: scrub the excess to a clean zero-overhead free run (`write_free_run`) before committing the smaller count. The non-atomic tail shrink keeps its commit-then-discard fast path (safe at the arena tail). Magic bumped 0.1.1 -> 0.1.2 (patch byte only; the 6-byte compat prefix is unchanged, so existing 0.1.x files still open). Unlike the 0.4.x original, the 0.2.x allocator API has no surviving-handle-on-failure mechanism, so the port keeps only the on-disk crash-ordering (no `recovered`/`-2` bookkeeping). Correctness strictly increases: the neighbour-corruption path is gone; the worst remaining outcome is a leak or an allocation with zeroed tail bytes. Tests (all green): Rust alloc,set / alloc,set,atomic; C test-checked-slab / test-checked-slab-atomic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Backport of two first_fit fixes to the 0.2.x line, both surfaced by the allocator fault-injection fuzz. #28 (crash-atomic realloc tail-shrink): reclaiming a shrunk tail block rewrote the block header and footer and discarded the tail as separate operations, so a fault mid-sequence left header, footer, and physical size disagreeing -- a state the block-walking recovery cannot repair (it would truncate the whole block, losing live data). A tail shrink now narrows only the user-visible length and keeps the block at its physical size (an oversized block, as a non-tail shrink already does); the tail is reclaimed on free. Behaviour change: a tail realloc shrink no longer returns space to the file immediately. #35 (two recovery bugs): * Interrupted tail *grow*: extend zero-fills the payload before the header/footer are rewritten, so a crash left a valid block followed by a headerless all-zero region that the recovery scan read as a size-0 block and rejected -- turning a recoverable crash into a hard open failure. Recovery now rolls an all-zero trailing region back by truncation (a real block is never all-zero); genuine mid-arena corruption still fails loudly. * Coalescing free commits the merged size to the header before the footer, so a crash left a stale footer that the header-following walk missed, later letting a neighbour's coalesce overlap two blocks and desync the walk into a hard open failure. Recovery now normalizes every block's footer to its authoritative header as it walks. Both fixes are self-contained recovery/realloc logic with no dependency on the 0.4.0 surviving-handle API or tail-replace primitives. Added two targeted recovery tests that construct the corrupted on-disk state directly (no fault-injection framework); updated realloc_tail_shrink to assert the new oversized-block behaviour. Tests (all green): Rust alloc,set / alloc,set,atomic; C test-first-fit / test-first-fit-atomic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…sync Opt-in compile define that turns plat_durable_sync into a no-op on both the Windows and POSIX paths. In-process test/fuzz runs tear the store down logically and reopen it in-process rather than surviving a real power loss, so skipping the physical sync leaves both the exercised logic and the on-disk bytes unchanged, while on macOS F_FULLFSYNC otherwise dominates C test runtime (minutes -> seconds). Inert unless the define is set; the default build and all production paths still sync. Never enable for a build that must survive a real crash. Mirrors the tooling introduced upstream (#39) so the 0.2.x C allocator test suites can run quickly during backport validation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Under `atomic` / BSTACK_FEATURE_ATOMIC, GhostTreeBstackAllocator's in-place tail shrink discarded the freed tail (try_discard) BEFORE zeroing the retained block's sub-block padding [new_len, aligned_new). A crash between the two left that padding holding the caller's stale bytes. A later same-block grow does not re-zero the newly-exposed region (it trusts the zeroed-memory invariant, ghost_tree.rs:60), so it would hand those stale bytes back to the caller. Zero the padding BEFORE discarding the tail, matching the non-atomic path, which was already correct. A crash now leaves at worst a zeroed retained block plus an unreclaimed tail (a benign leak ghost_tree already tolerates), never stale padding. Operation ordering only -- no on-disk format change, no magic bump. This is NOT the 0.4.x ghost_tree fix (8d5c9d9 / f226b76): that one fuses the two steps with the in-sequence tail-replace primitive (Atrunc / BSTACK_GEN_SPLICE, absent here) and reorders the non-atomic path to discard-first purely to drive the 0.4.0 surviving-handle-on-failure API (also absent here) -- porting it verbatim would REGRESS the 0.2.x non-atomic path. The zeroed-memory invariant violation is the part that matters without handles, and the zero-before-discard reorder closes it. Added realloc_tail_shrink_then_grow_reads_zeros as an invariant guard. Tests (all green): Rust alloc,set / alloc,set,atomic; C test-ghost-tree / test-ghost-tree-atomic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port of the 0.4.x ghost_tree AVL optimization (#39: 3dbeaef + 8631408 Rust, 90f60ac C) to the 0.2.x line. The AVL internals operate on raw offsets and were byte-identical to the optimization's base, so the diffs applied cleanly; no dependency on any 0.4.0 primitive or the three-type handle API. alloc/dealloc/realloc of non-tail blocks do less work under the allocator mutex: the rebalance up-pass no longer re-reads and re-writes each ancestor through a redundant balance-factor pass (the bf and height from the node write are threaded into avl_rebalance), and each node now caches its two child heights in the AVL header's previously-reserved bytes, so the up-pass and rotations write one node per level and read no children in the common in-balance case. Rust also swaps the per-op heap Vec path buffer for a stack array of the fixed MAX_AVL_DEPTH bound. Purely internal -- ~25-33% lower per-op latency under real F_FULLFSYNC, no API or observable-behaviour change. On-disk: magic bumped ALGT\x00\x01\x02\x00 -> ALGT\x00\x01\x03\x00 for the child-height cache. Existing 0.1.x files stay compatible: only the first 6 bytes are checked on open, and coalesce_and_rebalance (run every open) rebuilds the whole tree bottom-up via avl_write_and_update, which recomputes every node's cache from its children's own maintained height fields -- so a legacy/zeroed/crash-torn cache is healed on open and never trusted across a reopen. Added reopen_rebuilds_stale_child_height_cache to guard that contract. Tests (all green): Rust alloc,set (25) / alloc,set,atomic (28); C test-ghost-tree (34) / test-ghost-tree-atomic (37). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…SED_RESULT Compile-time annotation sweeps ported from the 0.4.x line (Rust: e8d323f, 2d2ec56, 6bc23e2; C: f194a41), adapted to the 0.2.x public API. Attributes only -- no logic, signatures, or behaviour changed. - #[must_use] (Rust) / BSTACK_WARN_UNUSED_RESULT (C): public functions whose return reports success/failure or hands back a result that should not be silently discarded now warn if the caller ignores them. Result-returning Rust fns are left alone (Result is already must_use). 36 Rust annotations; the C macro plus ~84 declaration annotations across bstack.h / bstack_alloc.h / bstack_bytevec.h, and two `(void)` casts on intentional cleanup-path discards in bstack_bytevec.c. - #[track_caller]: BStackSlice / BStackByteVec methods with a documented panic precondition now report the caller's source location on panic. - #[inline]: short public functions across the crate, for cross-crate inlining. 0.4.x annotations on types absent from 0.2.x (BStackOwnedSlice, BStackChunk, BStackAllocError/BStackBulkAllocError, the segregated allocator, and 0.4.x-only BStack ops) were skipped. Verified: cargo check (default / alloc,set / alloc,set,atomic) clean; C libbstack-alloc-set{,-atomic}.a compile clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported from the 0.4.x line (Rust cd96b7a, C 6078bdd), adapted to 0.2.x which has no commit_grow/commit_shrink helpers or fault-injection macro -- the grow/shrink commit-and-rollback is inlined to match this branch's extend/discard. - resize(target): grow (zero-filled) or shrink the payload to exactly target bytes; returns the size before the call. Growth follows extend's crash-consistency, shrink follows discard's (truncation is the commit point). Rejects a shrink below the locked length. - ensure(target): grow-only, no-op if already >= target; the unconditional counterpart of resize. - ensure_with(target, f) [Rust atomic / C BSTACK_FEATURE_ATOMIC]: grow only if shorter, handing the freshly zeroed tail to f for initialization before commit. The grown region sits beyond the committed length until the final header write, so it is crash-atomic on extend's terms without needing a journal. Tests: Rust tests::resize (6), tests::ensure (4), tests::ensure_with (3, atomic). C base test 71/71, test-atomic 111/111 (7 resize/ensure + 2 ensure_with added). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported from the 0.4.x line (6244267), adapted to 0.2.x: the shared commit helper lives in lib.rs (no io_core.rs), inlines the grow commit (no commit_grow/commit_sparse_extend helpers), uses seek+write_all for scattered writes (no write_at), and drops all fault_point! calls. - extend_sparse(buf, length) / extend_sparse_batched(writes, length): base API. Grow the payload by `length` with a single set_len (OS zero-fills the gaps), writing only the supplied buffer(s) into the new region. The batched form scatters (relative_offset, data) writes, validated as in-range and pairwise non-overlapping; empty data ignored; length == 0 is a no-op. - try_extend_sparse(s, buf, length) / try_extend_sparse_batched(s, writes, length): atomic. Add a try_extend-style size guard `s` (apply only if the current payload size equals `s`, else Ok(false)/*ok=0; a malformed request is still rejected regardless of the size match). No journal is needed: the whole grown region sits beyond the committed length, so a crash before the header commit rolls back by truncation, exactly like extend. C reuses the existing bstack_iovec_t (its typedef moved into the base section so the base batched API can use it, and its comment refreshed). Tests: Rust tests::extend_sparse (11) + tests::try_extend_sparse (7, atomic). C base test 80/80, test-atomic 127/127 (9 base + 7 atomic added). New C decls carry BSTACK_WARN_UNUSED_RESULT. Not ported (out of scope): the process_gen Sparse / BSTACK_GEN_SPARSE in-sequence variant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ust + C) `repeat(offset, pattern, count)` overwrites [offset, offset + count*pattern.len()) with `count` back-to-back copies of `pattern`; an empty pattern or count == 0 is a no-op. The general form of `zero`. `set` feature (Rust) / BSTACK_FEATURE_SET (C). Ported from the 0.4.x line, but WITHOUT its fixed-size write-in-progress journal (this branch has none): the full count*pattern.len() bytes are staged in memory and written directly, then durably synced -- slower for a large region and O(n) memory, but the same result and the same durability as `set`. The API is now present so the fill-based ergonomic methods (BStackSlice::fill, BStackByteVec::fill) can build on it. Tests: Rust tests::repeat (fill/offset/single-byte/noop/past-end-reject/ reopen). C test-set 101/101, test-set-atomic 194/194 (5 repeat tests). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported from the 0.4.x line (939a089), Rust only (master has no C for these) and on BStackSlice alone (no BStackOwnedSlice on this branch). Adapted to this branch's `BStackSlice<'a, A>`: the stack is reached via the `self.stack()` method rather than master's `self.stack` field. - Read-only (alloc): get, head/tail, contains, starts_with/ends_with, find/rfind, position/rposition, split_at/split_at_mut. - Write (set): fill (one BStack::repeat call), fill_with, copy_from_slice. - Atomic compound (set + atomic, each one crash-atomic BStack call): copy_from_bstack_slice, copy_within, swap (cross_exchange), reverse, rotate_left/rotate_right (process). #[track_caller] on the methods with a panic precondition (split_at, split_at_mut, copy_from_slice, copy_from_bstack_slice, copy_within, swap, rotate_left, rotate_right) and #[must_use] on head/tail, matching master. Tests: 21 in src/test.rs (alloc_tests) — 20 under set, 29 under set,atomic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported from the 0.4.x line (03929eb), Rust only (master has no C for these), adapted to this branch's BStackByteVec<'a, A: BStackSliceAllocator> over BStackSlice. - set(index, value): single crash-atomic write; Ok(None) if index >= len (get-style convention). - fill(value): overwrite the populated region via one BStack::repeat (no-op on empty). On this line repeat has no journal, so a large fill writes the whole region directly. - reserve_exact(additional): grow to exactly len + additional (no amortised over-allocation, unlike reserve). - shrink_to(min_capacity) / shrink_to_fit(): realloc the backing block down to max(len, min_capacity) / len. The internal capacity helper grow_to was renamed realloc_to and now handles shrink as well as growth (the allocator realloc already reallocs in either direction); its two existing call sites were updated. On-disk header format unchanged. Tests: 9 added to the existing bytevec test module in vec.rs (set/fill/ reserve_exact/shrink_to/shrink_to_fit + reopen); alloc::vec::tests 40 passed under alloc,set and alloc,set,atomic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported from the 0.4.x line (03929eb), Rust only, adapted to this branch's BStackByteVec<'a, A: BStackSliceAllocator> over BStackSlice (no BStackOwnedSlice here). All gated #[cfg(feature = "atomic")] since they ride BStack::copy / cross_exchange (set + atomic on this branch). Append-only movers (benign push-style crash model): extend_from_within, extend_from_bstack_slice, append_from_owned. In-place movers (crash-atomic per step, logically torn if interrupted): insert, remove, swap_remove, move_tail_into. copy_into_bstack_slice copies vec bytes out to a same-BStack slice. append_from_owned/move_tail_into take/return BStackSlice in the positions master used BStackOwnedSlice. append_from_owned consumes and frees its argument on EVERY path (foreign-stack, append-error, and success: `let freed = alloc.dealloc(other); appended.and(freed)`), never leaking. OOB index/range or u64 overflow -> Ok(None); a cross-BStack handle on a cross-slice method -> Err(InvalidInput). master doc links to the nonexistent extend_from_slice were repointed at push. Tests: 13 added to the inline bytevec test module (happy path + OOB->None + cross-BStack misuse + reopen; the foreign-append test also asserts the rejected slice is reclaimed, proving no leak). alloc::vec::tests 53 passed under alloc,set,atomic; compiles with the methods cfg'd out under alloc,set. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported verbatim from the 0.4.x line (03929eb) — all the helpers it needs (read_header, reserve, write_bytes_at, write_len_field) already exist on this branch. Appends an entire &[u8] in one shot: reserve once, write all bytes with a single durable set, then commit the new len (vs a grow/write/len cycle per byte). Empty input is a no-op. Crash-consistent like the other multi-step methods — a crash before the len commit leaves the appended bytes beyond the committed length, invisible, and re-running recovers. Tests: 3 added (bulk append, empty no-op, persist via raw block). alloc::vec::tests 43 (alloc,set) / 56 (alloc,set,atomic) passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Crate version 0.2.5 -> 0.2.6 (Cargo.toml, Cargo.lock). Following the per-release convention, the BStack format-version stamp is bumped BSTK\x00\x01\x0f\x00 (0.1.15) -> BSTK\x00\x01\x10\x00 (0.1.16) in src/lib.rs, c/bstack.c, c/test_bstack.c, and the README/doc comments. This is compat-neutral: `open` gates only on the first 6 bytes (BSTK\x00\x01), so files written by any 0.1.x still open, and 0.2.6 reads older files unchanged. The on-disk layout itself is identical to 0.2.5. CHANGELOG: the [Unreleased] section is stamped [0.2.6] - 2026-08-23 and a fresh empty [Unreleased] opened. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE IT! DO NOT CLOSE IT! This is used for CI purposes