Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions internal/state/state.go
Original file line number Diff line number Diff line change
Expand Up @@ -215,8 +215,9 @@ func (ts *ThreadState) RequestSafeStateChange(nextState State) bool {
}
ts.mu.Unlock()

// wait for the state to change to a stable state
ts.WaitFor(Ready, Inactive, Reserved)
// Done too: a thread that ends on its own goes ShuttingDown then Done without
// being stable again, and only Done means its C side is gone
ts.WaitFor(Ready, Inactive, Reserved, Done)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is a good idea. "Done" is kind of a transitionary state, meaning another thread probably is waiting for "Done" to set it to "Reserved".

Is this even necessary if #2662 is merged instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still needed with #2662: that one runs once RequestSafeStateChange() has returned, this one is what lets it return. A thread still booting (TransitionComplete) that then fails on its own goes ShuttingDown → Done and never reaches a state the old wait listens to, so the caller hangs before #2662's code runs. On main that's forceReboot() from RestartWorkers() or opcache_reset() during startup; on #2617 it's the boot timeout, which returns from initWorkers() while the thread is still booting.

On Done being transitional: the waiter only reads it and refuses, it never writes. Reserved was already in the set, so whichever of the two it sees the answer is the same, and whoever moves Done to Reserved isn't affected.

@nicolas-grekas nicolas-grekas Sep 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e6dc03c adds a test that hangs on main with it in. A worker calls opcache_reset() while booting, then gives up. The reboot holds scalingMu and waits in forceReboot() for the thread, which ends on its own at Done, and Shutdown() queues behind scalingMu: Init() never returns. With this PR it returns the boot error in 0.1s.

On main alone that's a narrow race; the stronger driver is #2617, whose boot timeout returns from initWorkers() while the thread is still booting.


return ts.RequestSafeStateChange(nextState)
}
Expand Down
19 changes: 19 additions & 0 deletions internal/state/state_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,25 @@ func TestWaitForStateWithTimeoutGivesUpAndDropsItsSubscriber(t *testing.T) {
})
}

func TestRequestSafeStateChangeRefusesAThreadThatEndedOnItsOwn(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
threadState := &ThreadState{currentState: TransitionComplete}

refused := make(chan bool, 1)
go func() {
refused <- threadState.RequestSafeStateChange(ShuttingDown)
}()

// once the request is parked, the thread ends by itself without passing
// through a stable state, the way a worker that fails to boot does
synctest.Wait()
threadState.Set(ShuttingDown)
threadState.Set(Done)

assert.False(t, <-refused, "a thread that is already done cannot be asked to shut down")
})
}

func assertNumberOfSubscribers(t *testing.T, threadState *ThreadState, expected int) {
t.Helper()

Expand Down
7 changes: 7 additions & 0 deletions testdata/worker-reset-then-fail.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<?php

// asks for a reboot of every thread while booting, then fails before
// reaching frankenphp_handle_request()
opcache_reset();

exit(1);
19 changes: 19 additions & 0 deletions worker_internal_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,22 @@ func TestInitJoinsAThreadStuckInStartupTeardown(t *testing.T) {
assert.Error(t, <-initDone, "a worker failing to boot must fail Init")
assert.True(t, failedThread.state.Is(state.Reserved), "the thread must have exited and been reclaimed before Init returned, got: "+failedThread.state.Name())
}

// a reboot that meets a booting worker waits for its thread to settle, and a
// worker that then gives up only reaches Done: Init() must still return
func TestInitReturnsWhenARebootWaitsOnAWorkerThatGivesUp(t *testing.T) {
initDone := make(chan error, 1)
go func() {
initDone <- Init(
WithWorkers("reset-then-fail", testDataPath+"/worker-reset-then-fail.php", 1, WithWorkerMaxFailures(2)),
WithNumThreads(1),
)
}()

select {
case err := <-initDone:
require.ErrorContains(t, err, "too many consecutive failures")
case <-time.After(30 * time.Second):
t.Fatal("Init() hung behind a reboot waiting for a worker thread that ended on its own")
}
}
Loading