LLM Benchmark: sequential test for MongoDB#5439

Open

bradleyshep wants to merge 103 commits into

bradley/sequential-mongodb-test

Commits on Jun 16, 2026

some minor polish/fixes
bradleyshep
committed
model flag; runbook
bradleyshep
committed
Track benchmark run output for git-revert; L1 MongoDB baseline
bradleyshep
committed
L1 MongoDB final — presence fix verified, bug report cleared
bradleyshep
committed
Record L1 MongoDB grades: 12/12 (Features 1-4 all 3/3), 1 fix iteration
bradleyshep
committed
L2 MongoDB generate — Scheduled Messages added (pre-grading restore point)
bradleyshep
committed
L2 MongoDB final — 15/15 (Features 1-5 all 3/3), 0 fix iterations
bradleyshep
committed
L3 MongoDB generate — Ephemeral Messages added (pre-grading restore point)
bradleyshep
committed
L3 MongoDB final — 18/18 (Features 1-6 all 3/3), 0 fix iterations
bradleyshep
committed
L4 MongoDB generate — Message Reactions added (pre-grading restore point)
bradleyshep
committed
L4 MongoDB final — 21/21 (Features 1-7 all 3/3), 0 fix iterations
bradleyshep
committed
L5 MongoDB generate — Message Editing + history added (pre-grading restore point)
bradleyshep
committed
L5 MongoDB final — 24/24 (Features 1-8 all 3/3), 0 fix iterations
bradleyshep
committed
L6 MongoDB generate — Real-Time Permissions added; backfill L2-L6 telemetry (failed L6 attempt excluded)
bradleyshep
committed
L6 MongoDB final — 27/27 (Features 1-9 all 3/3), 0 fix iterations
bradleyshep
committed
L7 MongoDB generate — Rich Presence added (pre-grading restore point)
bradleyshep
committed
L7 MongoDB final — 30/30 (Features 1-10 all 3/3), 1 fix iteration (3 presence bugs); preserve L1/L7 bug reports in snapshots
bradleyshep
committed
L8 MongoDB generate — Message Threading added (pre-grading restore point)
bradleyshep
committed
L8 MongoDB final — 33/33 (Features 1-11 all 3/3), 1 fix iteration (thread-reply leak); failed API-500 fix attempt excluded
bradleyshep
committed
L9 MongoDB generate — Private Rooms & DMs added (pre-grading restore point)
bradleyshep
committed
Add cross-backend LEADERBOARD (cost/fixes/quality per level, through L8)
bradleyshep
committed
L9 MongoDB final — 36/36 (Features 1-12 all 3/3), 0 fix iterations; leaderboard through L9
bradleyshep
committed
L10 MongoDB generate — Room Activity Indicators added (pre-grading restore point)
bradleyshep
committed
L10 MongoDB final — 39/39 (Features 1-13 all 3/3), 1 fix iteration (activity-decay bug); leaderboard through L10
bradleyshep
committed
L11 MongoDB generate — Draft Sync added (pre-grading restore point)
bradleyshep
committed
L11 MongoDB final — 42/42 (Features 1-14 all 3/3), 0 fix iterations; leaderboard through L11
bradleyshep
committed
LEADERBOARD: add time-to-complete (wall-clock) section through L11, with rigor caveat
bradleyshep
committed
L12 MongoDB generate — Anonymous Migration added (pre-grading restore point)
bradleyshep
committed
update
bradleyshep
committed
L12 MongoDB final — 45/45 (Features 1-15 all 3/3), 0 fix iterations; run COMPLETE, leaderboard finalized through L12
bradleyshep
committed
perf-benchmark: add MongoDB backend (client adapter + stress/realistic scenarios + --backend mongo)
bradleyshep
committed
perf-benchmark: add MongoDB optimized reference + results (clean sonnet-4.6 pass; ~800->1394 msg/s, ~1.7x); document cross-machine/throttle/prompt caveats
bradleyshep
committed

Commits on Jun 17, 2026

Commits on Jun 18, 2026

Commits on Jun 23, 2026

Commits on Jun 24, 2026