Skip to content

Accept FIOASYNC ioctl and F_SETOWN/F_GETOWN fcntl#81

Open
Max042004 wants to merge 1 commit into
sysprog21:mainfrom
Max042004:ioctl-fioasync
Open

Accept FIOASYNC ioctl and F_SETOWN/F_GETOWN fcntl#81
Max042004 wants to merge 1 commit into
sysprog21:mainfrom
Max042004:ioctl-fioasync

Conversation

@Max042004
Copy link
Copy Markdown
Collaborator

@Max042004 Max042004 commented Jun 6, 2026

Problem

Running nginx (aarch64 Linux, e.g. nginx:alpine) under elfuse in its default master/worker mode never serves requests: the listen socket binds, the host kernel completes TCP handshakes, but every request hangs and times out. With master_process off; (single process, no fork) nginx serves correctly, which masks the real issue.

Root cause

nginx's ngx_spawn_process arms the master→worker channel socket right before fork() with:

ioctl(channel[0], FIOASYNC, &on);     /* enable SIGIO-driven I/O   */
fcntl(channel[0], F_SETOWN, ngx_pid); /* who receives that SIGIO   */

and treats failure of either as fatal — it logs an alert, ngx_close_channel(), and return NGX_INVALID_PID without ever calling fork(). elfuse answered FIOASYNC with ENOTTY and F_SETOWN with EINVAL, so the master silently ended up with zero workers. A verbose trace shows no clone(220), no accept4(242), no epoll_pwait(22) after the FIOASYNC failure — the master just spins in rt_sigsuspend, and the accepted connections are never serviced.

Fix

elfuse does not forward host SIGIO into the guest, and nginx workers receive both client I/O and channel commands via epoll rather than SIGIO, so both calls are safe to accept as no-ops:

  • sys_ioctl: FIOASYNC reads the int arg (for EFAULT parity) and returns success without arming host O_ASYNC.
  • sys_fcntl: F_SETOWN / F_SETOWN_EX accept and track no owner; F_GETOWN / F_GETOWN_EX report "no owner". glibc implements fcntl(F_GETOWN) on top of F_GETOWN_EX, so the EX form writes a struct f_owner_ex{type=F_OWNER_PID, pid=0} to stay coherent.

With this, nginx -g "daemon off;" forks its worker and serves HTTP on the stock config: GET/HEAD/404, keep-alive, and 100-way concurrency all return 200 with byte-identical bodies.

Testing

  • New tests/test-ioctl-fioasync.c (registered in tests/manifest.txt) replays nginx's pre-fork channel arming on a socketpair and a TCP socket, asserting FIOASYNC, F_SETOWN, and F_GETOWN all succeed.
  • make check: the full guest driver suite passes (the only failure observed was the pre-existing, unrelated flaky test-fork — "unexpected exit reason 0x3" — which is racy on main and untouched by this change, which only adds switch cases in sys_ioctl/sys_fcntl).

Summary by cubic

Accepts FIOASYNC ioctl and F_SETOWN/F_GETOWN (and EX variants) fcntl as safe no-ops so nginx master/worker mode forks workers and serves requests under elfuse. Fixes the hang where requests timed out because no workers were spawned.

  • Bug Fixes
    • sys_ioctl: handle LINUX_FIOASYNC as a no-op (reads int arg; no host SIGIO). Adds constant in abi.h.
    • sys_fcntl: accept F_SETOWN/F_SETOWN_EX as no-ops; F_GETOWN returns 0; F_GETOWN_EX writes {type=F_OWNER_PID, pid=0}.
    • Tests: add tests/test-ioctl-fioasync.c and manifest entry; checks socketpair and TCP socket paths.

Written for commit b6260b6. Summary will update on new commits.

Review in cubic

nginx's ngx_spawn_process arms the master->worker channel socket with
ioctl(FIOASYNC) immediately followed by fcntl(F_SETOWN), right before
fork(), and treats failure of either as fatal: it logs an alert, closes
the channel, and returns NGX_INVALID_PID without ever forking the worker.

elfuse answered FIOASYNC with ENOTTY and F_SETOWN with EINVAL, so nginx
in its default master/worker mode silently ended up with zero workers:
the listen socket still accepted connections at the host kernel, but no
guest worker ever accept()ed them, so every request hung. (With
"master_process off" nginx served fine, which masked the issue.)

elfuse does not forward host SIGIO into the guest, and nginx workers
receive both client I/O and channel commands via epoll rather than
SIGIO, so both calls are safe to accept as no-ops:

  - sys_ioctl: FIOASYNC reads the int arg (for EFAULT parity) and
    returns success without arming host O_ASYNC.
  - sys_fcntl: F_SETOWN / F_SETOWN_EX accept and track no owner;
    F_GETOWN / F_GETOWN_EX report "no owner". glibc implements
    fcntl(F_GETOWN) on top of F_GETOWN_EX, so the EX form writes a
    struct f_owner_ex{type=F_OWNER_PID, pid=0} to stay coherent.

With this, "nginx -g 'daemon off;'" forks its worker and serves HTTP
(GET/HEAD/404, keep-alive, concurrency) on the default config.

Add tests/test-ioctl-fioasync.c, which replays nginx's pre-fork channel
arming on a socketpair and a TCP socket.
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/syscall/fs.c">

<violation number="1" location="src/syscall/fs.c:804">
P2: New F_SETOWN/F_GETOWN handlers can return success for closed FDs because they skip closed-descriptor validation.</violation>

<violation number="2" location="src/syscall/fs.c:805">
P2: F_SETOWN_EX no-op path skips user-pointer read/validation, so bad guest pointers incorrectly succeed.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/syscall/fs.c
return 0;
}
case 8: /* F_SETOWN */
case 15: /* F_SETOWN_EX */
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: F_SETOWN_EX no-op path skips user-pointer read/validation, so bad guest pointers incorrectly succeed.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/syscall/fs.c, line 805:

<comment>F_SETOWN_EX no-op path skips user-pointer read/validation, so bad guest pointers incorrectly succeed.</comment>

<file context>
@@ -801,6 +801,29 @@ int64_t sys_fcntl(guest_t *g, int fd, int cmd, uint64_t arg)
         return 0;
     }
+    case 8:  /* F_SETOWN */
+    case 15: /* F_SETOWN_EX */
+        /* SIGIO/SIGURG delivery owner. nginx's ngx_spawn_process pairs
+         * ioctl(FIOASYNC) with fcntl(F_SETOWN) on the channel socket before
</file context>

Comment thread src/syscall/fs.c
host_fd_ref_close(&host_ref);
return 0;
}
case 8: /* F_SETOWN */
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: New F_SETOWN/F_GETOWN handlers can return success for closed FDs because they skip closed-descriptor validation.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/syscall/fs.c, line 804:

<comment>New F_SETOWN/F_GETOWN handlers can return success for closed FDs because they skip closed-descriptor validation.</comment>

<file context>
@@ -801,6 +801,29 @@ int64_t sys_fcntl(guest_t *g, int fd, int cmd, uint64_t arg)
         host_fd_ref_close(&host_ref);
         return 0;
     }
+    case 8:  /* F_SETOWN */
+    case 15: /* F_SETOWN_EX */
+        /* SIGIO/SIGURG delivery owner. nginx's ngx_spawn_process pairs
</file context>

Copy link
Copy Markdown
Contributor

@jserv jserv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase latest 'main` branch and resolve conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants