Conversation
…d occurrence `AgentScheduler.claim` advanced a due schedule by exactly one cron step from its own stored `nextRunAt`. The row therefore stayed due after the claim, and `tick` — which the worker runs on `* * * * *` — dispatched again on the next minute, and the minute after that, until the backlog drained. Every one of those dispatches is a real `stories.start`: a workspace provisioned and an engine run charged to the project budget. An hourly schedule after a day of worker downtime is 24 paid runs in 24 minutes, against a budget sized for one an hour, in a window narrow enough that each one sees the last as still in flight. A five-minute schedule is 288. Nothing bounded it; the replay was as long as the outage. Downtime is not exotic: `tsx watch` restarts the worker on every file change. The occurrences being replayed are stale by construction. A security audit due 05:00 Monday and run 14:32 Tuesday is not the audit anyone asked for — it is the same audit against a repository that has moved on, and the work it was meant to precede already happened. Compute the next occurrence from `now` instead. That makes the loop level-triggered: it converges on "this schedule is due" and satisfies it once, however long the gap, rather than replaying every edge it missed. A schedule claimed on time is unaffected, because the next occurrence after `now` and the next after its own due instant are the same one. The decision is a pure function, `scheduleAdvance`, so the semantics are testable against a fixed clock with no database: on-time claims land exactly where they used to, a week-long backlog collapses to one claim, a single missed period is still reported as coalesced, catching up across a daylight-saving shift keeps the trigger timezone, and a clock behind the due instant never moves a schedule backwards. What was already right is untouched: the claim remains a compare-and-swap on `(nextRunAt, lastScheduledAt)`, so two workers still cannot take the same occurrence, and the dispatch still identifies itself by the occurrence it satisfies — `lastScheduledAt`, the message dedupe key and `trigger.scheduledFor` all keep the observed due instant rather than the wall clock. `tick` now also returns `coalesced`, the number of claims that absorbed at least one further occurrence. `due` counts rows, so a 24-occurrence catch-up and an ordinary minute were indistinguishable in the worker log; they no longer are. It costs one extra cron step per due schedule, never a walk over the backlog, so an outage of any length is the same work. Closes theam#330.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
A schedule that fell behind runs once when the worker comes back, instead of
once per missed occurrence.
AgentScheduler.claimadvanced a due schedule by exactly one cron step from itsown stored
nextRunAt. The row therefore stayed due after the claim, andtick— which the worker runs on
* * * * *— dispatched again the next minute, andthe minute after, until the backlog drained. The next occurrence is now computed
from
now.tickalso returnscoalesced: the number of claims that absorbed at least onefurther occurrence.
Why
Every one of those replays is a real
stories.start— a workspace provisionedand an engine run charged to the project budget. An hourly schedule after a day
of worker downtime is 24 paid runs in 24 minutes, against a budget sized for
one an hour, in a window narrow enough that each sees the last as still in
flight. A five-minute schedule is 288. Nothing bounded it: the replay was as long
as the outage. Downtime is not exotic —
tsx watchrestarts the worker on everyfile change.
The replayed occurrences are also stale by construction. A security audit due
05:00 Monday and run 14:32 Tuesday is not the audit anyone asked for; it is the
same audit against a repository that has moved on, and the work it was meant to
precede already happened.
Computing from
nowmakes the loop level-triggered: it converges on "thisschedule is due" and satisfies it once, however long the gap, rather than
replaying every edge it missed. A schedule claimed on time is unaffected, because
the next occurrence after
nowand the next after its own due instant are thesame one.
duecounts rows, so a 24-occurrence catch-up and an ordinary minute wereindistinguishable in the worker log.
coalescedcosts one extra cron step perdue schedule — never a walk over the backlog — so an outage of any length is the
same work.
What is deliberately untouched. The claim stays a compare-and-swap on
(nextRunAt, lastScheduledAt), so two workers still cannot take the sameoccurrence. The dispatch still identifies itself by the occurrence it satisfies:
lastScheduledAt, the message dedupe key andtrigger.scheduledForall keep theobserved due instant rather than the wall clock, which is what keeps the dispatch
idempotent.
#330 offered three shapes — coalesce, bound the catch-up the way a Kubernetes
CronJob uses
startingDeadlineSeconds, or make it the manifest's choice. Thisimplements coalescing, which is what every agent kickstart ships wants. It is
here as a concrete proposal, cheap to reject: if bounded catch-up is the intended
contract instead, the semantics live in one pure function and the test matrix
flips with it.
Verification
The decision is extracted as a pure function,
scheduleAdvance, so the semanticsare testable against a fixed clock with no database.
Unit — 3 of the 5 new cases are red against the previous behaviour, and the
on-time and clock-skew cases are green both ways because they must not regress:
Integration —
security-audit:nightlyput a week in arrears, then three ticks atthe same instant. Without the fix:
Three ticks, three paid runs. With the fix, one run, one new story message, and
the schedule left ahead of the clock at
2026-01-11T02:00:00Z.The integration test asserts this project's own rows, not the tick counters:
agent_schedulesis global and the shared test database accumulates rows acrosssuites, so a counter assertion there would pass or fail on history. I hit exactly
that while writing it.
scheduledis safe because the fixture's catalog throwsfor any other project, and the run was repeated three times consecutively to
confirm it.
pnpm verifypasses locallypnpm verifydoes not pass on this machine and not because of this change:test:devreports 116 tests, 92 pass, 24 fail here, and I measured theidentical 92/24 on a clean
main— Windows noise in the patchedimage-sizecases,
tarfailing to resolveC:, and the registry publication tests. Beyondthe suite, the failure was reproduced end to end against a real PostgreSQL: three
ticks producing three dispatches before the change and one after.
Closes #330.