Skip to content

perf(task): reuse keys during journal union - #80

Merged
earakely-scale merged 1 commit into
scaleapi:mainfrom
morluto:codex/perf-journal-merging
Oct 6, 2026
Merged

earakely-scale merged 1 commit into
scaleapi:mainfrom
morluto:codex/perf-journal-merging

Conversation

@morluto

@morluto morluto commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Closes #77.

Compute journal union keys once per incoming/existing item, removing repeated prefix slices and key projections. Keep equality-based membership so unhashable values, deployed-environment identity and replay ordering remain supported.

Incoming deployed-environment items Before After Speedup
100 0.189 ms 0.024 ms 7.7×
1,000 17.275 ms 1.797 ms 9.6×
3,000 151.949 ms 13.110 ms 11.6×

For 1,000 items merged into an empty list, key projections drop from 501,500 to 1,000. Membership comparisons remain quadratic. Timings are local union microbenchmarks, not complete task runs: macOS arm64, Python 3.12.13, baseline 602d730, median of five samples.

Reproduce:

.venv/bin/python tst/benchmarks/journal_union.py --base 602d730 --repeats 5

Validation:

  • 63 focused journal tests passed; a new deterministic regression verifies projection counts and ordinary/move-to-end output for unhashable items.
  • Full unit and protocol suites: 6,231 passed, 13 skipped.
  • Benchmark compares production function bodies and checks equal outputs.
  • Independent review completed with no actionable findings; plugin API check and diff checks passed.

This PR is independent of the task-graph, Explorer and existing context-diff PRs.

RetriggerConfidence Score: 4/5

The PR appears safe to merge; the benchmark should also measure re-recording to show the cost of the changed move branch.

Fix All in CursorFindings

  1. P2 Re-record speed goes unmeasured ▶
Fix with agent prompt
### Issue 1
tst/benchmarks/journal_union.py:71-75
The benchmark always passes an empty `existing` list and leaves `move_to_end` off. Re-recording a step uses the changed move branch, which now builds `existing_keys` for the existing items. Add a nonempty re-record case so maintainers can see whether that extra list affects its speed.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The task journal now computes union keys once per incoming and existing item, reducing repeated work while keeping equality matching and replay order. A benchmark and regression test cover the change.

  • Task journal unions compute each item's key once.

Reviews (1) · Last reviewed commit: "perf(task): reuse journal union keys"

@morluto
morluto marked this pull request as ready for review October 6, 2026 19:54
@morluto
morluto requested a review from a team as a code owner October 6, 2026 19:54
Comment thread tst/benchmarks/journal_union.py
@earakely-scale

Copy link
Copy Markdown
Collaborator

Taking a look

@earakely-scale
earakely-scale merged commit 652c93a into scaleapi:main Oct 6, 2026
11 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce repeated key projection during journal union

2 participants