Snapshotted Event Archive (SEA): Collaboration server and client in Rust with fluid driver by CraigMacomber · Pull Request #28217 · microsoft/FluidFramework

Craig Macomber (Microsoft) (CraigMacomber) · 2026-09-14T18:34:51Z

Description

In progress hackathon project focused on learning. This code is not intended for merging, and draft PR is mainly to make inspecting the diff/divergence simpler.

This will need a crates.io upstream added to our ADO feed (and likely a few other things) before it can pass CI.

Examples using the new example-utils (currently just text-editor and inventory-app) supporting selecting the SEA service, and it's also included in integration tests.

As this was partly a learning project about agentic coding, a lot of historical artifacts and learnings are noted in the "historical" folder.

Reviewer Guidance

Not intended for review.

github-actions · 2026-09-14T18:36:15Z

Hi! Thank you for opening this PR. Want me to review it?

Based on the diff (169010 lines, 677 files), I've queued these reviewers:

Correctness — logic errors, race conditions, lifecycle issues
Security — vulnerabilities, secret exposure, injection
API Compatibility — breaking changes, release tags, type design
Performance — algorithmic regressions, memory leaks
Testing — coverage gaps, hollow tests

How this works

Adjust the reviewer set by ticking/unticking boxes above. Reviewer toggles alone don't trigger anything.
Tick Start review below to dispatch the review fleet.
After review finishes, tick Start review again to request another run — it auto-resets after each dispatch.
This comment updates as new commits land; your reviewer selections are preserved.
Start review

Retain command provenance, source inventories, optimized binary hashes, delivery checks, per-process resource samples, and failures. Isolate service and generator CPU affinity and alternate backend order. Document stack differences and measurement limits. Validated both minimal-client paths and 32-client Sea admission, source counts, script syntax, documentation, and policy.

Predeclare ten fresh runs per matched-resource and tested-throughput point, using longer warmup and alternating backend order. Retain failures instead of replacing them.

Compare native WebTransport and benchmark-only WebSocket clients with matching paced workloads and explicit storage/CPU metadata. Add syntax-aware Rust test spans and cloc test/support breakdowns. Native smoke tests, focused Clippy, Rustdoc, workspace build, root build and policy pass. Full workspace tests had one native round-trip timeout which passed in isolation; full Clippy remains blocked by unchanged server main's 102-line function.

Preserve integer total rates while allowing fractional per-worker Node pacing. Retain the original invalid probes and add an explicit retry matrix plus ten-repeat native transport and Tinylicious comparison points. Validated syntax, script documentation links, and eight corrected Tinylicious probes: all four 750 ops/s points passed; 1250 ops/s exceeded the latency threshold.

Select sea-benchmarks for existing cargo run commands after adding presentation tool binaries. No measured service or generator code changes. Validated CLI help and bounded smoke, formatting, rustdoc, workspace build/tests, policy, and root build. Full workspace Clippy still fails the existing server main function-length limit.

Allow memory or durable-file storage per Sea cell, verify the server-reported mode, and record selected modes in the manifest. Preserve memory defaults. Document history-dependent synchronized journal costs. Validated syntax, formatting, documentation links, and a native durable-file smoke with retained overload evidence. No service or generator data-path changes.

Allow buffered-file and eight service cores with distinct physical CPU affinity from the unchanged four generator cores. Preserve existing one/four-core placements and memory guard. Validated syntax, Biome, documentation links, topology, and a passing eight-core buffered native smoke. No server or native generator code changes.

Remove operation IDs, submission resolution, and server operation deduplication across Rust, wire, WASM, and TypeScript APIs. Reconcile accepted submissions through ordered session history while preserving terminal append authority and suffix recovery. Advance archive markers to SEAQ4/SEAM3 and wire protocol to version 9. Previous formats are intentionally rejected; clients and servers must rebuild together. Update tests, generated API reports, documentation, and the decision record. Keep sequential server setup together with a local Clippy expectation. Validation: canonical Rust checks, root build and policy checks, Sea integration and browser suites, and general Fluid E2E with sea-websocket (658 passing, 526 pending, no failures).

Retain summaries, configurations, provenance and linked validation logs. Consolidate 1,368 raw evidence files into a verified archive, losslessly compress large service logs, and remove disposable runtime storage. Record cleanup and extraction instructions.

Apply repository Biome formatting without changing parsed measurement data. Integration validation passed: canonical Rust formatting, Clippy, rustdoc, build, tests, documentation, scoped policy, and root build.

Update code, comment, and test/support counts at revision 470af45. Document regeneration from Git history instead of retaining new source-count artifacts; preserve historical performance measurements.

Checkpoint ordered admission, memory/file batching, append-only durable storage assumptions, regression tests, and local performance evidence before session-sequence integration.

Reconcile bounded batching with session-local submission sequences and terminal-prefix recovery. Preserve ordered admission and cancellation settlement; validate native, WASM, Node/browser, policy and repository builds.

Poll idle appends once before allocating queued completion state, retain pending work in the shared driver, and stack-pin completion polling. Cover idle ambiguous failure and retain baseline, candidate, repeat, and validation evidence. Memory throughput improves across measured payload/window cases. Buffered singleton overhead and durable tail variability remain unresolved.

Wait for actual archive reopening instead of observing an Arc strong count before destruction completes. Preserve exclusion and committed-prefix assertions. Add native connection failure diagnostics and record the unresolved transport timeout investigation without increasing deadlines or suppressing failures.

Register the checkpoint collection and preserve filter identity fields when inserting LevelDB upserts. Add checkpoint update, isolation, deletion, and reopen regression coverage, configuration documentation, and a changeset. Validated with Tinylicious compilation, all 22 tests, ESLint, scoped policy checks, and real client workloads.

…parisons Refresh source counts and optimized browser, service, transport, and storage measurements at 92ecf30. Preserve raw outcomes and provenance in compact evidence archives. Update the native/WASM benchmark call and add explicit Tinylicious memory and LevelDB selection with readiness-based verification. Record both original LevelDB failures and repaired measurements, including latency failures and durability caveats. Validated with documentation checks, benchmark smoke controls, artifact and report assertions, policy checks, and pnpm build:fast.

…n evidence Add explicit memory-backed WebSocketStream benchmark cases for Fluid and direct clients, pin Tinylicious memory storage, and record transport metadata. Retain paired dummy DDS/SharedTree measurements, including the failed browser attempt and successful retry. Condense the presentation with inline Mermaid charts, an alternatives matrix, dependency counts, compatibility-test breakdowns, and labelled payload sizes. Validation: canonical Cargo checks, complete ./test.sh, root pnpm build:fast, benchmark typechecks/lint/format, scoped policy, documentation checks, measured-value assertions, and wide/narrow Mermaid rendering passed.

Remove obsolete benchmark campaigns and profiles while preserving current measurement evidence and iteration narratives. Pin historical citations to the pre-cleanup revision and update documentation checks.

Separate historical plans, learnings, overview, and retained measurement evidence from current contributor guidance. Add an agentic-development reading guide, update iteration records and documentation checks, and streamline reusable benchmark scripts while preserving recorded evidence.

github-actions · 2026-09-21T05:23:24Z

Bundle size comparison

Base commit: unresolved
Head commit: 4ac0f79dd1b1057ecb5ce56d46b6bbddb41c0fcb

⚠️ Comparison unavailable.

The PR's CI build failed — fix the build and the comment will update once the next run succeeds.

github-actions · 2026-09-21T05:24:11Z

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output

$ start-server-and-test "npm run serve -- --host 127.0.0.1 --no-open" http://127.0.0.1:3000 check-links
1: starting server using command "npm run serve -- --host 127.0.0.1 --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-website@0.0.0 serve
> docusaurus serve --host 127.0.0.1 --no-open

[SUCCESS] Serving "build" directory at: http://127.0.0.1:3000/

> fluid-framework-website@0.0.0 check-links
> linkcheck http://127.0.0.1:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  658102 links
    3579 destination URLs
    3898 URLs ignored
       0 warnings
       0 errors

github-actions · 2026-09-21T05:25:41Z

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output

$ start-server-and-test "npm run serve -- --host 127.0.0.1 --no-open" http://127.0.0.1:3000 check-links
1: starting server using command "npm run serve -- --host 127.0.0.1 --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-website@0.0.0 serve
> docusaurus serve --host 127.0.0.1 --no-open

[SUCCESS] Serving "build" directory at: http://127.0.0.1:3000/

> fluid-framework-website@0.0.0 check-links
> linkcheck http://127.0.0.1:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  658102 links
    3579 destination URLs
    3898 URLs ignored
       0 warnings
       0 errors

github-actions Bot added area: tools area: build Build related issues area: repo Repo related work dependencies Pull requests that update a dependency file area: website base: main PRs targeted against main branch labels Sep 14, 2026

github-actions Bot added area: dds Issues related to distributed data structures area: dds: tree labels Sep 14, 2026

Fluid Bot added 21 commits September 17, 2026 02:17

docs(rust-service): start crate cleanup iteration

cf77b2b

chore(sea-benchmarks): improve docs and tests

dc24acf

docs(rust-service): complete sea-benchmarks report

2175131

test(sea-compression): cover payload edge cases

6ae54f0

docs(rust-service): complete sea-compression report

e1a5657

test(sea-conformance): cover storage initial state

c589493

docs(iteration-0013): complete sea-conformance report

627d880

fix(sea-content-addressed): verify immutable publication

3535401

docs(iteration-0013): report sea-content-addressed audit

1883ddd

fix(sea-core): harden directory decoding

68f3695

docs(iteration-0013): complete sea-core report

6c6b5bb

test(sea-encryption): cover envelope boundaries

c847017

docs(rust-service): report sea-encryption cleanup

303c887

test(sea-file): strengthen persistence coverage

d47477e

docs(iteration-0013): report sea-file audit

c1a91c9

test(sea-file-durable): cover recovery errors

82e0de4

docs(rust-service): report sea-file-durable cleanup

ce0b5ea

test(sea-memory): cover storage edge cases

b755579

docs(iteration-0013): complete sea-memory report

1f10246

fix(sequencer): reject zero event lag

4457602

docs(iteration): complete sea-sequencer report

77d19a2

Copilot added 18 commits September 20, 2026 01:56

test(sea): define repeated presentation measurement points

7820513

Predeclare ten fresh runs per matched-resource and tested-throughput point, using longer warmup and alternating backend order. Retain failures instead of replacing them.

Merge branch 'rust-service-session-sequence' into rust-service

c43b46c

style(sea): format retained measurement JSON

470af45

Apply repository Biome formatting without changing parsed measurement data. Integration validation passed: canonical Rust formatting, Clippy, rustdoc, build, tests, documentation, scoped policy, and root build.

docs(sea): refresh presentation source counts after session refactor

7753aba

Update code, comment, and test/support counts at revision 470af45. Document regeneration from Git history instead of retaining new source-count artifacts; preserve historical performance measurements.

perf(sea): add bounded storage pipeline and grouped durable appends

47159ea

Checkpoint ordered admission, memory/file batching, append-only durable storage assumptions, regression tests, and local performance evidence before session-sequence integration.

Merge branch 'rust-service' into rust-service-storage-optimization

32ee6e9

Reconcile bounded batching with session-local submission sequences and terminal-prefix recovery. Preserve ordered admission and cancellation settlement; validate native, WASM, Node/browser, policy and repository builds.

github-actions Bot added the area: server Server related issues (routerlicious) label Sep 20, 2026

Copilot added 2 commits September 21, 2026 00:40

More tests, cleanup report comparison

d01db72

Craig Macomber (Microsoft) (CraigMacomber) changed the title ~~Rust service~~ Snapshotted Event Archive (SEA): Collaboration server and client in Rust with fluid driver Sep 21, 2026

Copilot added 5 commits September 21, 2026 04:27

Presentation tweaks

3e95385

docs: repurpose Sea presentation report as project overview

74f3736

chore(sea): remove superseded benchmark artifacts

85631aa

Remove obsolete benchmark campaigns and profiles while preserving current measurement evidence and iteration narratives. Pin historical citations to the pre-cleanup revision and update documentation checks.

chore: remove intermediate SEA changesets

4ac0f79

Navigation Menu

Snapshotted Event Archive (SEA): Collaboration server and client in Rust with fluid driver - #28217

Snapshotted Event Archive (SEA): Collaboration server and client in Rust with fluid driver#28217
Craig Macomber (Microsoft) (CraigMacomber) wants to merge 504 commits into
microsoft:mainfrom
CraigMacomber:rust-service

Craig Macomber (Microsoft) (CraigMacomber) commented Sep 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Craig Macomber (Microsoft) (CraigMacomber) commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Reviewer Guidance

Uh oh!

github-actions Bot commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 21, 2026

Bundle size comparison

Uh oh!

github-actions Bot commented Sep 21, 2026

linkcheck output

Uh oh!

github-actions Bot commented Sep 21, 2026

linkcheck output

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Craig Macomber (Microsoft) (CraigMacomber) commented Sep 14, 2026 •

edited

Loading

github-actions Bot commented Sep 14, 2026 •

edited

Loading