Skip to content

feat: declared background workers + frankenphp_get_worker_handle() #2617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nicolas-grekas wants to merge 39 commits into
base: main
Choose a base branch
from
+3,497 −493
Open

feat: declared background workers + frankenphp_get_worker_handle() #2617

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
531a5ab
fix: a thread that ended on its own can still be asked to shut down
nicolas-grekas Sep 21, 2026
2c34750
feat: declared background workers + frankenphp_get_worker_handle()
nicolas-grekas Sep 6, 2026
d66b65e
fix: a blocking receive on the handle marks a background worker ready
nicolas-grekas Sep 11, 2026
407bba8
fix: never inherit the reserved worker variables
nicolas-grekas Sep 11, 2026
77fe5a5
fix: cap the restart backoff, and EOF a handle a forked child still h…
nicolas-grekas Sep 11, 2026
0a341c0
fix: resolve max_threads auto on the HTTP budget
nicolas-grekas Sep 11, 2026
32645d0
chore: simplifications and wording from the review
nicolas-grekas Sep 11, 2026
ba0249b
test: cover the parked run and the handle cache
nicolas-grekas Sep 11, 2026
502c060
docs: flag frankenphp_get_worker_handle() experimental in the stub
nicolas-grekas Sep 13, 2026
f94625d
feat: frankenphp_worker_tick(), the explicit ready point of backgroun…
nicolas-grekas Sep 13, 2026
cf0cc7d
feat: FRANKENPHP_WORKER_BACKGROUND holds the name, HTTP workers untou…
nicolas-grekas Sep 14, 2026
5c894d1
feat: wake a background worker once at start
nicolas-grekas Sep 14, 2026
376818e
docs: event loop example for background workers
nicolas-grekas Sep 14, 2026
5ea2489
refactor: one lifecycle abstraction instead of two
nicolas-grekas Sep 14, 2026
4c83649
fix: never block or race on the startup failure channel
nicolas-grekas Sep 14, 2026
769cc4d
feat: max_execution_time bounds the bootstrap of a background worker
nicolas-grekas Sep 14, 2026
2b706d0
docs: stop the whole loop on drain in the Revolt example
nicolas-grekas Sep 14, 2026
ce8d5d5
test: only assert the bounded bootstrap where the timers are known to…
nicolas-grekas Sep 14, 2026
bf63e1a
feat: num defaults to one thread for background workers
nicolas-grekas Sep 14, 2026
d5c485f
chore: review polish
nicolas-grekas Sep 14, 2026
cf5ff26
test: the tick leaves the handle quiet
nicolas-grekas Sep 14, 2026
9659857
fix: keep generating unique names for workers that share a script
nicolas-grekas Sep 17, 2026
cdf6760
feat: report the server of a worker as its own metric label
nicolas-grekas Sep 17, 2026
15f9587
fix: address the review of the background worker lifecycle
alexandre-daubois Sep 17, 2026
b21e9fa
fix: keep the declared path as the default worker name
nicolas-grekas Sep 18, 2026
e412a28
fix: pace only the background runs that end too fast
nicolas-grekas Sep 18, 2026
85ed2af
refactor: declare the labels of a worker instead of widening the metrics
nicolas-grekas Sep 18, 2026
9c3777a
feat: report the server of a worker through an optional ServerMetrics…
alexandre-daubois Sep 18, 2026
1516c46
feat: the handle of a background worker becomes an object
nicolas-grekas Sep 18, 2026
4b4c510
refactor: keep the stream of a handle on the handle, not on the thread
nicolas-grekas Sep 20, 2026
437e9da
docs: lead with the poll loop, the stream is for the libraries that t…
nicolas-grekas Sep 20, 2026
fe7cb3d
fix: a handle built behind the constructor must not reach the Go side
nicolas-grekas Sep 21, 2026
8e135b6
fix: pace a crash loop whatever the run lasted
nicolas-grekas Sep 21, 2026
c7b4575
chore: check the server the same way in SendRequest and SendMessage
nicolas-grekas Sep 21, 2026
77c7e69
fix: form the Windows socket pair on loopback, and only with ourselves
nicolas-grekas Sep 21, 2026
6cfb168
feat: bound the wait for a background worker to become ready
nicolas-grekas Sep 21, 2026
3e77906
chore: fewer comments, the ones left say why
nicolas-grekas Sep 21, 2026
47cf687
fix: Validate() refuses a worker scoped to a server it was not given
nicolas-grekas Sep 30, 2026
ff61aba
chore: refresh the stub hash of the arginfo
nicolas-grekas Sep 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
783 changes: 783 additions & 0 deletions bgworker_test.go
View file
Open in desktop

Large diffs are not rendered by default.

51 changes: 11 additions & 40 deletions caddy/app.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ import (
"github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/dunglas/frankenphp"
"github.com/dunglas/frankenphp/internal/fastabs"
)

var (
Expand Down Expand Up @@ -141,20 +140,17 @@ func (f *FrankenPHPApp) collectOptions(repl *caddy.Replacer, keep bool) ([]frank
frankenphp.WithMaxRequests(f.MaxRequests),
)

usedWorkerNames := make(map[string]bool, len(f.Workers))

// register global workers
for _, w := range f.Workers {
w.FileName = repl.ReplaceKnown(w.FileName, "")
Comment thread
alexandre-daubois marked this conversation as resolved.
w.Name = createUniqueWorkerName(usedWorkerNames, w, "")
workerOptions, err := w.toWorkerOptions()
if err != nil {
return nil, err
}
opts = append(opts, frankenphp.WithWorkers(w.Name, w.FileName, w.Num, workerOptions...))
}

moduleOpts, err := f.collectModuleOptions(repl, usedWorkerNames, keep)
moduleOpts, err := f.collectModuleOptions(repl, keep)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -213,7 +209,7 @@ func (f *FrankenPHPApp) Stop() error {
}

// register workers and servers for "php" and "php_server" modules
func (f *FrankenPHPApp) collectModuleOptions(repl *caddy.Replacer, usedWorkerNames map[string]bool, keep bool) ([]frankenphp.Option, error) {
func (f *FrankenPHPApp) collectModuleOptions(repl *caddy.Replacer, keep bool) ([]frankenphp.Option, error) {
opts := make([]frankenphp.Option, 0, len(f.modules))
serversByIndex := make(map[int]*frankenphp.Server, len(f.modules))

Expand All @@ -231,7 +227,7 @@ func (f *FrankenPHPApp) collectModuleOptions(repl *caddy.Replacer, usedWorkerNam
}
}

server, moduleOpts, err := f.collectModule(repl, module, usedWorkerNames)
server, moduleOpts, err := f.collectModule(repl, module)
if err != nil {
return nil, err
}
Expand All @@ -248,7 +244,7 @@ func (f *FrankenPHPApp) collectModuleOptions(repl *caddy.Replacer, usedWorkerNam
return opts, nil
}

func (f *FrankenPHPApp) collectModule(repl *caddy.Replacer, module *FrankenPHPModule, usedWorkerNames map[string]bool) (*frankenphp.Server, []frankenphp.Option, error) {
func (f *FrankenPHPApp) collectModule(repl *caddy.Replacer, module *FrankenPHPModule) (*frankenphp.Server, []frankenphp.Option, error) {
serverName := f.resolveServerName(module)
server, err := frankenphp.NewServer(
module.resolvedDocumentRoot,
Expand All @@ -265,7 +261,6 @@ func (f *FrankenPHPApp) collectModule(repl *caddy.Replacer, module *FrankenPHPMo

for _, w := range module.Workers {
w.FileName = repl.ReplaceKnown(w.FileName, "")
w.Name = createUniqueWorkerName(usedWorkerNames, w, serverName)
workerOptions, err := w.toWorkerOptions()
if err != nil {
return nil, nil, err
Expand All @@ -277,33 +272,6 @@ func (f *FrankenPHPApp) collectModule(repl *caddy.Replacer, module *FrankenPHPMo
return server, opts, nil
}

// avoid name collisions for workers
// on collision, a name is first qualified with the server name
// ("<serverName>:<name>") before falling back to a numeric postfix
func createUniqueWorkerName(usedWorkerNames map[string]bool, wc workerConfig, serverName string) string {
if wc.Name == "" {
wc.Name, _ = fastabs.FastAbs(wc.FileName)
}

name := wc.Name
suffix := 0
for {
if _, ok := usedWorkerNames[name]; !ok {
usedWorkerNames[name] = true
break
}
if serverName != "" {
name = serverName + ":" + wc.Name
serverName = ""
continue
}
suffix++
name = fmt.Sprintf("%s_%d", wc.Name, suffix)
}

return name
}

// UnmarshalCaddyfile implements caddyfile.Unmarshaler.
func (f *FrankenPHPApp) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
for d.Next() {
Expand Down Expand Up @@ -418,10 +386,13 @@ func (f *FrankenPHPApp) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
if len(wc.MatchPath) != 0 {
return d.Errf(`"match" can only be used in a php_server worker block, not in a global one: %q`, wc.FileName)
}
// check for duplicate workers
for _, existingWorker := range f.Workers {
if existingWorker.FileName == wc.FileName {
return d.Errf("global workers must not have duplicate filenames: %q", wc.FileName)
// check for duplicate workers; background workers are keyed
// by name, several may share a script
if !wc.Background {
for _, existingWorker := range f.Workers {
if !existingWorker.Background && existingWorker.FileName == wc.FileName {
return d.Errf("global workers must not have duplicate filenames: %q", wc.FileName)
}
}
}

Expand Down
103 changes: 71 additions & 32 deletions caddy/caddy_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -759,6 +759,45 @@ func TestMetrics(t *testing.T) {
require.NoError(t, testutil.GatherAndCompare(ctx.GetMetricsRegistry(), strings.NewReader(expectedMetrics), "frankenphp_total_threads", "frankenphp_busy_threads"))
}

// TestBackgroundWorkerFromCaddyfile starts a background worker from a
// Caddyfile and checks it runs: the sentinel its script touches appears
func TestBackgroundWorkerFromCaddyfile(t *testing.T) {
sentinel := filepath.ToSlash(filepath.Join(t.TempDir(), "bg.sentinel"))
tester := caddytest.NewTester(t)
initServer(t, tester, `
{
skip_install_trust
admin localhost:2999
http_port `+testPort+`
https_port 9443

frankenphp {
worker {
file ../testdata/bgworker/basic.php
num 1
name bg-caddy
background
env BG_SENTINEL `+sentinel+`
}
}
}

localhost:`+testPort+` {
route {
php {
root ../testdata
}
}
}
`, "caddyfile")

require.Eventually(t, func() bool {
_, err := os.Stat(sentinel)

return err == nil
}, 5*time.Second, 25*time.Millisecond, "the background worker declared in the Caddyfile did not run")
}

func TestWorkerMetrics(t *testing.T) {
var wg sync.WaitGroup
tester := caddytest.NewTester(t)
Expand Down Expand Up @@ -831,19 +870,19 @@ func TestWorkerMetrics(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="` + workerName + `"} 0
frankenphp_busy_workers{server="",worker="` + workerName + `"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="` + workerName + `"} 2
frankenphp_total_workers{server="",worker="` + workerName + `"} 2

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="` + workerName + `"} 10
frankenphp_worker_request_count{server="",worker="` + workerName + `"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="` + workerName + `"} 2
frankenphp_ready_workers{server="",worker="` + workerName + `"} 2
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -906,7 +945,7 @@ func TestPhpServerWorkerMatchPoolCount(t *testing.T) {

var pools []string
for line := range strings.SplitSeq(metrics.String(), "\n") {
if !strings.HasPrefix(line, "frankenphp_total_workers{worker=") {
if !strings.HasPrefix(line, "frankenphp_total_workers{") {
continue
}
if !strings.Contains(line, "dedup-match-worker.php") && !strings.Contains(line, "dedup-plain-worker.php") {
Expand Down Expand Up @@ -988,19 +1027,19 @@ func TestNamedWorkerMetrics(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="my_app"} 0
frankenphp_busy_workers{server="",worker="my_app"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="my_app"} 2
frankenphp_total_workers{server="",worker="my_app"} 2

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="my_app"} 10
frankenphp_worker_request_count{server="",worker="my_app"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="my_app"} 2
frankenphp_ready_workers{server="",worker="my_app"} 2
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1084,19 +1123,19 @@ func TestAutoWorkerConfig(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="` + workerName + `"} 0
frankenphp_busy_workers{server="",worker="` + workerName + `"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="` + workerName + `"} ` + workers + `
frankenphp_total_workers{server="",worker="` + workerName + `"} ` + workers + `

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="` + workerName + `"} 10
frankenphp_worker_request_count{server="",worker="` + workerName + `"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="` + workerName + `"} ` + workers + `
frankenphp_ready_workers{server="",worker="` + workerName + `"} ` + workers + `
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1357,7 +1396,7 @@ func TestMaxWaitTimeWorker(t *testing.T) {

expectedMetrics := `
# TYPE frankenphp_worker_queue_depth gauge
frankenphp_worker_queue_depth{worker="service"} 0
frankenphp_worker_queue_depth{server="",worker="service"} 0
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1458,21 +1497,21 @@ func TestMultiWorkersMetrics(t *testing.T) {

# HELP frankenphp_busy_workers Number of busy PHP workers for this worker
# TYPE frankenphp_busy_workers gauge
frankenphp_busy_workers{worker="service1"} 0
frankenphp_busy_workers{server="",worker="service1"} 0

# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="service1"} 2
frankenphp_total_workers{worker="service2"} 3
frankenphp_total_workers{server="",worker="service1"} 2
frankenphp_total_workers{server="",worker="service2"} 3

# HELP frankenphp_worker_request_count
# TYPE frankenphp_worker_request_count counter
frankenphp_worker_request_count{worker="service1"} 10
frankenphp_worker_request_count{server="",worker="service1"} 10

# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="service1"} 2
frankenphp_ready_workers{worker="service2"} 3
frankenphp_ready_workers{server="",worker="service1"} 2
frankenphp_ready_workers{server="",worker="service2"} 3
`

ctx := caddy.ActiveContext()
Expand Down Expand Up @@ -1623,12 +1662,12 @@ func TestWorkerRestart(t *testing.T) {

// Check metrics
expectedMetrics := `
# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="service"} 1
frankenphp_ready_workers{server="",worker="service"} 1
# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="service"} 1
frankenphp_total_workers{server="",worker="service"} 1
`

require.NoError(t,
Expand All @@ -1651,15 +1690,15 @@ func TestWorkerRestart(t *testing.T) {

// frankenphp_ready_workers should be back to 1 even after worker restarts
expectedMetrics = `
# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, WorkerHandle::tick() for background workers
# TYPE frankenphp_ready_workers gauge
frankenphp_ready_workers{worker="service"} 1
frankenphp_ready_workers{server="",worker="service"} 1
# HELP frankenphp_total_workers Total number of PHP workers for this worker
# TYPE frankenphp_total_workers gauge
frankenphp_total_workers{worker="service"} 1
frankenphp_total_workers{server="",worker="service"} 1
# HELP frankenphp_worker_restarts Number of PHP worker restarts for this worker
# TYPE frankenphp_worker_restarts counter
frankenphp_worker_restarts{worker="service"} 3
frankenphp_worker_restarts{server="",worker="service"} 3
`

require.NoError(t,
Expand Down Expand Up @@ -2122,7 +2161,7 @@ func TestSymlinkWorkerBehavior(t *testing.T) {

// Accessing the worker script without worker configuration MUST fail
// The script checks $_SERVER['FRANKENPHP_WORKER'] and dies if not set
tester.AssertGetResponse("http://localhost:"+testPort+"/index.php", http.StatusOK, "Error: This script must be run in worker mode (FRANKENPHP_WORKER not set to '1')\n")
tester.AssertGetResponse("http://localhost:"+testPort+"/index.php", http.StatusOK, "Error: This script must be run in worker mode (FRANKENPHP_WORKER not set)\n")
})

t.Run("MultipleRequests", func(t *testing.T) {
Expand Down
Loading
Loading