🤖 fix: keep a refused or failed migration pending until its command exits - #5579
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
c3a42e8 to
01b34a8
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 01b34a8ce5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Review budget status for head 7ee1ce8. The PR is not merged yet.
Generated with |
|
Readiness record for head 7ee1ce8:
Generated with |
Summary
When a foreground command's move to the background is refused (a removal or archive has sealed the workspace) or fails (the record cannot be created), the bash tool aborts the command and waits at most 5 s for it to exit. Before this PR, the migration's pending entry ended with that wait, so a command whose kill took effect late ran on untracked, and the removal went on to delete its checkout. Now the pending entry stays until the command's exit actually settles. A removal or archive cleanup keeps waiting for it, and fails closed at its existing 60 s drain deadline instead of deleting the checkout.
Fixes #5522
Builds on #5575 (merged: the #5465 model and repros). This PR turns its two case-1 repros into plain tests.
Implementation
tools/bash.ts: the migration handle was ausingdeclaration, so it ended with the block. It is now a plain handle, ended by a smallusingguard unless the failed-migration path hands it toexecStream.exitCode. It ends only when that promise resolves, which confirms the exit. A rejection (for example a remote transport error) does not confirm the stop, so the migration then stays pending for the session, and removal and archive of that workspace keep failing closed. The 5 s join and the tool's reply are unchanged.pendingAdmissionsentry thatcleanup()already drains (backgroundProcessManager.ts).cleanup(ws, { failClosedAfterDrainTimeout: true }), which throws after 60 s (🤖 Background cleanup: surface failed kills and bound the pending-admission drain #5477). An unconfirmed stop therefore blocks the destructive step, as 🤖 fix: only a supervisor inside the group signals a background process group #5521 requires. Session disposal (no deadline) waits until the command's exit is confirmed. This only delays the disposal's tail steps, as an admitted spawn stuck in a runtime call already does.Model
formal/background-processes/BgCleanup.tla:MJoinTimeoutnow moves the migration to"stopping", whichMStoppedends only after the command exits.MigrationOwnedcounts"stopping"as owned.MC_cleanup_refused_join_timeoutandMC_cleanup_failed_join_timeout(the forced admitted-then-failed path) now hold (EXPECT""). A new mutant constantJoinUntracked(the code before this PR) isFALSEin every other config. Its configMC_mut_cleanup_join_untrackedstill violatesNoLiveAfterDeleteandMigrationOwned, so the model still sees the defect class.Validation
Test-first. On main before this PR (🤖 tests: cover #5465 background-process cases in the TLA+ model #5575), both repros ("a refused migration ..." and "a failed migration ...") fail at their target assertion,
expect(early === "finished").toBe(false), withExpected: false, Received: true: the removal's cleanup has already finished while the command is alive. The first version of the plain test, run without the fix, also failed:With the fix, both pass as plain tests. A third test, "a migration whose exit observation fails stays pending", uses a runtime whose
exitCoderejects after the kill. Without the rejection handling it fails (Expected: "waiting", Received: "finished"). Each also releases the held kill and checks that the cleanup then finishes and the PID is gone.formal/background-processes/check.sh(full) andFORMAL_FAST=1exit 0.Bun 1.3.12:
backgroundProcessesFormalRepro,bash*,backgroundProcessManager,workspaceService.archiveandworkspaceService.removetests pass (403 tests).make static-checkpasses.Risks
Low. The change only affects the path where a migration was refused or failed, and only lengthens how long cleanup waits for that command. A command that never exits now makes removal or archive fail after 60 s instead of deleting the checkout under it. That is the intended fail-closed behavior.
Generated with
xum• Model:anthropic:claude-opus-5-5• Thinking:high• Cost:$32.91