Skip to content

📖 feat: Stream Bounded Workspace File Reads - #303

Merged
danny-avila merged 4 commits into
mainfrom
lia/b2-stream-reads
Oct 4, 2026
Merged

danny-avila merged 4 commits into
mainfrom
lia/b2-stream-reads

Conversation

@lia-by-librechat

@lia-by-librechat lia-by-librechat Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Ordinary read_file rejected small windows from files larger than 1 MiB because it loaded the entire file. Stream complete-line windows through the existing verified descriptor instead.

  • Default 200 lines, maximum 500; cap returned UTF-8 content at 1 MiB, including separators.
  • Advance continuation past returned complete lines. An oversized first requested line returns actionable READ_LIMIT_EXCEEDED.
  • Bound memory with 64 KiB chunks. Check cancellation and a monotonic 10-second deadline during scanning, collection, and pending reads. Interruption settles before queued I/O drains; Node owns eventual file/root cleanup.
  • Freeze the scan extent at the opened size. Growth cannot prolong it; truncation stops at observed EOF; replacement never redirects the handle.

Contract and compatibility

Read contract. Request/result shapes, capabilities, and protocol version are unchanged. Preserve UTF-8 replacement decoding, BOM-marked UTF-16, CRLF, and empty-file behavior. Binary bytes remain decoded text; unread content is not validated. Pagination assumes unchanged content across requests.

Search, preview, edit, write, instruction snapshots, and safe parent creation retain their existing limits and semantics. No LibreChat changes or deployment.

Verification

  • Real-filesystem coverage: large-file windows, byte/line pagination, oversized lines, EOF, split encodings, cancellation/deadlines, growth/truncation/replacement, descriptor cleanup, legacy consumers, and current protocol acceptance.
  • Focused workspace, root-access, protocol, and instruction tests on Linux Node 20.11.0 and 24.16.0: 140 passed per runtime; 2 existing skips per runtime (alternate-group permissions and case-insensitive filesystem). Actual queued FileHandle I/O cases cover cancellation/deadline settlement and eventual cleanup with legacy and held roots.
  • Package tsc --noEmit, build, Prettier on new files/touched regions, and git diff --check: passed. Existing unrelated formatting is preserved.
  • Early-timer regression: actual pending I/O, clock 0.25 ms before deadline, bounded settlement and eventual cleanup verified for legacy and held roots. Inspector confirmed explicit interruption at cleanup.
  • Local platform: Linux, Node 20.11.0 and 24.16.0. Ordinary reads use the default I/O backend; only queued-I/O test subprocesses disable io_uring to enforce saturation. CI covers Linux Node 20/22/24 and macOS root containment. New large-file/queued-I/O cases were not run on macOS locally; Windows not exercised.

Review

  • R1 (P2), pending-I/O cleanup blocked interruption: fixed in 0c0b30308b89c34f47c500d53c6fc2766f1a48fc.
  • R2 (P2), fractional timer/clock recheck race: fixed in 0578cb79634b218f9571ba17bf629f882d72acad.
  • Swept admission, scan checks, race settlement, file/root ownership, worker error handling, bounded legacy consumers, and validators by invariant. Cleanup now follows explicit interruption, never a clock recheck.
  • CI1 (P2), Node 20 test subprocesses bypassed the saturation harness: fixed in 95d50b848088ebcc48515c61bf0c639edff55fdb. Worker runtime settings are unchanged.

Head: 95d50b848088ebcc48515c61bf0c639edff55fdb. Independent exact-head review complete: no findings. CI: 10 checks passed; Lambda MicroVM Runner Image (arm64) remains in progress.

Independent review additionally checked 1,605 deterministic real-file windows against whole-text semantics and current validators. Its executable coverage was Linux Node 24 with non-native tests; held-root checks, typecheck, build, and formatting are covered by the implementation checks above, not claimed as independently executed.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head: bfe284e. B2 streams bounded, complete-line workspace reads through the verified descriptor. Wire shapes and whole-file limits for other consumers remain unchanged. Focused verification and independent exact-head review are in progress.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head: 0c0b303. Fixed R1 (P2): interruption no longer waits for queued-I/O cleanup. Actual FileHandle reads verify cancellation/deadline settlement before drain and eventual file/root closure. Focused checks: 138 passed, 2 existing skips; typecheck, package build, and touched-code formatting passed. Fresh independent exact-head review running.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head: 0578cb7. Fixed R2 (P2): file/root cleanup follows explicit interruption, not a clock recheck. Swept read admission, scanning, race settlement, ownership, worker handling, and compatibility by invariant. Real queued-I/O tests cover timers firing before the monotonic deadline. Focused checks: 140 passed, 2 existing skips; typecheck, build, touched-code formatting passed. Independent exact-head review running.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head: 95d50b8. Test-only portability correction: queued-I/O subprocesses disable io_uring so Node 20 file reads cannot bypass the saturation harness. Worker runtime settings are unchanged. Linux Node 20.11.0 and 24.16.0 focused checks each passed 140 tests with 2 existing skips; typecheck, build, and formatting passed. Independent exact-head review running.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review complete for head 95d50b8: no findings. Prior P2 findings R1, R2, and CI1 are fixed; none rejected. Independent checks also validated 1,605 real-file windows against existing line semantics and protocol validators. Local Linux Node 20/24 focused checks: 140 passed each, 2 existing skips each. Typecheck, package build, and touched-code formatting passed. CI: 10 passed; Lambda MicroVM Runner Image (arm64) still running. No deployment.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head, final review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-03T23:59:08.643001Z 95d50b8 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 95d50b8480

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila
danny-avila merged commit 192f61c into main Oct 4, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants