Skip to content

Implement epoll APIs in the JS filesystem - #27207

Merged
sbc100 merged 9 commits into
emscripten-core:mainfrom
guybedford:epoll
Aug 18, 2026
Merged

Implement epoll APIs in the JS filesystem#27207
sbc100 merged 9 commits into
emscripten-core:mainfrom
guybedford:epoll

Conversation

@guybedford

@guybedford guybedford commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Adds epoll_create1, epoll_ctl, epoll_wait and epoll_pwait on a single fd readiness model shared with poll().

Builds off of the existing event-driven readiness model in the JS FS system with the integration point as the per-inode wait-queue, having each FS node carrying a listeners set and producers calling notifyNodeListeners(node, flags) on ready transitions. There is no separate or parallel readiness machinery - it integrates directly with the existing model. pollOne(fd, events) is reused on the same readiness definition.

Per standard epoll semantics - epoll_ctl ADD installs a new listener on the watched node. If items are already ready they are added to the ready list. That listener then appends the registration to the epoll's ready list for waking. The epoll_wait consumes the ready list, re-checking each item against its current mask via pollOne.

  • EPOLLONESHOT clears listeners to avoid unnecessary callback firing. EPOLL_CTL_MOD can then re-arm them again.
  • EPOLLET is implemented correctly to avoid refiring items that remain ready
  • EPOLLEXCLUSIVE is passed for listeners allowing only one wake for multiple epoll listeners to avoid the "thundering herd".
  • When exceeding maxevents, draining follows Linux-like semantics in supporting round-robin ready calling. To achieve this without losing performance, a doubly-linked list is used for the registrations. A simpler set / array with copying could be used alternatively if we don't want to use this approach.
  • Registrations key on the open file description (the dup-shared stream state): closing a watched fd and reusing its number for a different open does not resurrect the registration onto the new fd (matching Linux).
  • dup(2) of an epoll fd yields another reference to the same epoll instance (registrations and ready list shared, per Linux eventpoll semantics); only the last close reclaims it.

Most of the diff is tests, covering these semantics in depth including error handling, level versus edge reporting, nesting and ELOOP, fd-close auto-removal, dup instance sharing, JSPI and pthreads, real sockets, deregistration, deterministic round-robin fairness, and multi-threaded waits on a shared epoll fd (per-edge exactly-once wakeup under EPOLLET, herd wakeup under level triggering, and EPOLLONESHOT disarm/re-arm and exactly-once delivery against racing waiter threads).

Minor semantic divergences to note:

  • epoll_pwait ignores sigmask
  • epoll_create1 accepts and ignores EPOLL_CLOEXEC (unknown flags are rejected with EINVAL)
  • closing a watched fd evicts its registration even when a dup of the open description survives; Linux keeps the registration alive through the file and continues delivering events (the classic epoll footgun)
  • nesting is capped at 5 levels
  • epoll_event under Wasm in Musl is laid out as aligned 16 rather than x86-64's packed 12 bytes.

This PR originally also included emscripten_epoll_set_callback, a non-blocking JS-callback readiness variant (usable without ASYNCIFY/JSPI), which was split out into the follow-on PR #27547.

Fixes: #5033, #10556

PR made with AI assistance, under my review

@sbc100 sbc100 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this like this direction.

I've not had time to look at all the details yet, but it seems like a great idea to unify the node events like this.

Comment thread system/include/emscripten/emscripten.h Outdated
Comment thread ChangeLog.md Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libsockfs.js Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libsockfs_node.js Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libpipefs.js
Comment thread src/lib/libpipefs.js Outdated
Comment thread system/include/emscripten/emscripten.h Outdated
Comment thread src/lib/libsyscall.js
@sbc100 sbc100 changed the title epoll implementation for the JS filesystem Implement epoll APIs in the JS filesystem Jun 27, 2026
@guybedford
guybedford force-pushed the epoll branch 2 times, most recently from 0e64f2b to 9b46f69 Compare June 29, 2026 23:46
Comment thread src/modules.mjs Outdated
Comment thread system/include/emscripten/epoll.h Outdated
Comment thread src/lib/libsyscall.js Outdated
Comment thread src/lib/libepoll.js Outdated
@guybedford
guybedford force-pushed the epoll branch 3 times, most recently from 97a4580 to d33958b Compare June 30, 2026 23:04
@sbc100

sbc100 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

I'm liking the direction of this commit but still a little overwhelmed by the size of it.

Can you think of any more ways to split it up? I'm not sure myself... for example, could we land the internal refactoring of the poll/select to use the new notification system before we land the rest of epoll? Maybe not practically separable?

Comment thread src/lib/libepoll.js Outdated
Comment thread test/codesize/test_codesize_cxx_ctors1.json Outdated
Comment thread test/core/test_epoll_noderawfs.out Outdated
Comment thread test/test_core.py Outdated
Comment thread test/test_core.py Outdated
@guybedford

Copy link
Copy Markdown
Collaborator Author

Can you think of any more ways to split it up? I'm not sure myself... for example, could we land the internal refactoring of the poll/select to use the new notification system before we land the rest of epoll? Maybe not practically separable?

I've refactored out the JS notification changes into #27226 if that helps?

@sbc100

sbc100 commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

Is there some existing test for standalone.c where it verifies that it can at least link against all these symbol stubs? If not, we can add one, either as part of this PR or as a followup.

@guybedford

Copy link
Copy Markdown
Collaborator Author

Is there some existing test for standalone.c where it verifies that it can at least link against all these symbol stubs? If not, we can add one, either as part of this PR or as a followup.

other.test_standalone_syscalls is now extended to verify the syscalls.

sbc100 pushed a commit that referenced this pull request Aug 17, 2026
This splits the `sockets` test suite into `sockets_node` and
`sockets_browser`, as suggested by @sbc100 in #27207. A pure test
refactor with no test content changes, so that the node socket tests no
longer run under a browser-based suite.

* `test/sockets_common.py`: shared server harnesses and helpers
(websockify/compiled harnesses, echo server processes, IPv6 loopback
probe, skip decorators)
* `test/test_sockets_node.py`: `sockets_node`/`sockets_node64` on
`RunnerCore` (previously these ran under `BrowserCore`) - the nodejs
echo, connect-failure, subprotocol, and NODERAWSOCKETS tests
* `test/test_sockets_browser.py`: `sockets_browser`/`sockets_browser64`
on `BrowserCore` - all btest-based tests, unchanged
* `test/runner.py`, CircleCI config, and the test-suite docs updated;
the chrome CI job now runs both suites, preserving existing coverage

Verified that every test method in the old file appears exactly once
across the new files.
Comment thread test/test_sockets_node.py Outdated
Comment thread test/test_other.py
Comment thread test/sockets/test_epoll_socket_blocking.c Outdated
Comment thread test/sockets/test_epoll_socket_blocking.c Outdated
Comment thread test/other/test_standalone_syscalls.cpp Outdated
Comment thread test/other/test_epoll_fairness.c Outdated
@sbc100

sbc100 commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Looks like all the tests pass. ChangeLog.md needs a rebase, but I'll land without waiting for anther round of CI if you only change that.

Add epoll_create1/epoll_ctl/epoll_wait/epoll_pwait on the legacy (non-WASMFS)
JS syscall layer, built on the per-inode readiness wait-queue: level- and
edge-triggered modes, EPOLLONESHOT, EPOLLEXCLUSIVE, EPOLLRDHUP, nesting,
dup-shared epoll instances, and blocking waits under PROXY_TO_PTHREAD,
ASYNCIFY, and JSPI.
@sbc100
sbc100 enabled auto-merge (squash) August 18, 2026 03:01
@sbc100
sbc100 merged commit ca200ba into emscripten-core:main Aug 18, 2026
42 checks passed
@guybedford
guybedford deleted the epoll branch August 18, 2026 05:09
guybedford added a commit to guybedford/libc that referenced this pull request Aug 18, 2026
Emscripten restored the `sys/epoll.h` header in 6.0.2 and implements
`epoll_create`, `epoll_create1`, `epoll_ctl`, `epoll_wait` and
`epoll_pwait` in the JS filesystem as of 6.0.8.

Version detection in libc-test now also tracks the tiny version
component so that point releases can be distinguished, and parses the
uppercase `__EMSCRIPTEN_MAJOR__` macro forms used since Emscripten
5.0.1, where the lowercase names became non-integer aliases.

Link: https://github.com/emscripten-core/emscripten/blob/77dc595f295427f8a010fb8de8d67925c254d91d/system/lib/libc/musl/include/sys/epoll.h
Link: emscripten-core/emscripten#27206
Link: emscripten-core/emscripten#27207
guybedford added a commit to guybedford/libc that referenced this pull request Aug 18, 2026
Emscripten restored the `sys/epoll.h` header in 6.0.2 and implements
`epoll_create`, `epoll_create1`, `epoll_ctl`, `epoll_wait` and
`epoll_pwait` in the JS filesystem as of 6.0.8.

Version detection in libc-test now also tracks the tiny version
component so that point releases can be distinguished, and parses the
uppercase `__EMSCRIPTEN_MAJOR__` macro forms used since Emscripten
5.0.1, where the lowercase names became non-integer aliases.

Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h
Link: emscripten-core/emscripten#27206
Link: emscripten-core/emscripten#27207
guybedford added a commit to guybedford/libc that referenced this pull request Aug 18, 2026
Emscripten restored the `sys/epoll.h` header in 6.0.2 and implements
`epoll_create`, `epoll_create1`, `epoll_ctl`, `epoll_wait` and
`epoll_pwait` in the JS filesystem as of 6.0.8.

Version detection in libc-test now also tracks the tiny version
component so that point releases can be distinguished, and parses the
uppercase `__EMSCRIPTEN_MAJOR__` macro forms used since Emscripten
5.0.1, where the lowercase names became non-integer aliases.

Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L17
Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L70-L74
Link: emscripten-core/emscripten#27206
Link: emscripten-core/emscripten#27207
guybedford added a commit to guybedford/libc that referenced this pull request Aug 18, 2026
Emscripten restored the `sys/epoll.h` header in 6.0.2 and implements
`epoll_create`, `epoll_create1`, `epoll_ctl`, `epoll_wait` and
`epoll_pwait` in the JS filesystem as of 6.0.8.

Version detection in libc-test now also tracks the tiny version
component so that point releases can be distinguished, and parses the
uppercase `__EMSCRIPTEN_MAJOR__` macro forms used since Emscripten
5.0.1, where the lowercase names became non-integer aliases.

Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L17
Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L70-L74
Link: emscripten-core/emscripten#27206
Link: emscripten-core/emscripten#27207
guybedford added a commit to guybedford/libc that referenced this pull request Aug 19, 2026
Emscripten restored the `sys/epoll.h` header in 6.0.2 and implements
`epoll_create`, `epoll_create1`, `epoll_ctl`, `epoll_wait` and
`epoll_pwait` in the JS filesystem as of 6.0.8.

Version detection in libc-test now also tracks the tiny version
component so that point releases can be distinguished, and parses the
uppercase `__EMSCRIPTEN_MAJOR__` macro forms used since Emscripten
5.0.1, where the lowercase names became non-integer aliases.

Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L17
Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L70-L74
Link: emscripten-core/emscripten#27206
Link: emscripten-core/emscripten#27207
guybedford added a commit to guybedford/libc that referenced this pull request Aug 19, 2026
Emscripten restored the `sys/epoll.h` header in 6.0.2 and implements
`epoll_create`, `epoll_create1`, `epoll_ctl`, `epoll_wait` and
`epoll_pwait` in the JS filesystem as of 6.0.8.

Version detection in libc-test now also tracks the tiny version
component so that point releases can be distinguished, and parses the
uppercase `__EMSCRIPTEN_MAJOR__` macro forms used since Emscripten
5.0.1, where the lowercase names became non-integer aliases.

Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L17
Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L70-L74
Link: emscripten-core/emscripten#27206
Link: emscripten-core/emscripten#27207
renovate-bot pushed a commit to renovate-bot/rust-lang-_-libc that referenced this pull request Aug 20, 2026
Emscripten restored the `sys/epoll.h` header in 6.0.2 and implements
`epoll_create`, `epoll_create1`, `epoll_ctl`, `epoll_wait` and
`epoll_pwait` in the JS filesystem as of 6.0.8.

Version detection in libc-test now also tracks the tiny version
component so that point releases can be distinguished, and parses the
uppercase `__EMSCRIPTEN_MAJOR__` macro forms used since Emscripten
5.0.1, where the lowercase names became non-integer aliases.

Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L17
Link: https://github.com/emscripten-core/emscripten/blob/4483d70a78098ed5d860dff2dc21f3025b2da2ee/system/lib/libc/musl/include/sys/epoll.h#L70-L74
Link: emscripten-core/emscripten#27206
Link: emscripten-core/emscripten#27207
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

epoll support

2 participants