feat(webrtc): WebRTC-Direct v2 flow (libp2p/specs#715) β listener accepts v2, dialer opt-in - #1459
Conversation
β¦rity by DTLS role go-libp2p (go-msgio pbio fallback) writes the uvarint prefix and the protobuf body as two SCTP messages, so every py<->go WebRTC-Direct connection died at Noise msg#1 with "malformed handshake frame length". New stream._decode_frames pops complete frames off an accumulating buffer (frames may be split or batched across SCTP messages, bounded by MAX_MESSAGE_SIZE); DataChannelReadWriter and WebRTCStream.on_data both use it. A malformed frame now resets the stream instead of dropping one message and desynchronising. With the handshake passing, a py listener then hit aiortc's `assert stream_id not in self._data_channels`: aiortc picks DCEP stream id parity from the ICE role, which is inverted for a WebRTC-Direct listener (ICE-controlled, DTLS server), so it used even ids like go's dialer. _create_channel now passes an explicit id by DTLS role (RFC 8832: client even, server odd). Verified live against go-libp2p v0.49.0: go->py and py->go, v1 and v2. Refs libp2p#1437
f53ceb2 to
f42dae9
Compare
- keep aiortc allocating (and recycling) SCTP stream ids; only re-seed the parity from the DTLS role (explicit ids tied the 16-bit space to our never-recycled counter) - on_data: deliver frames decoded before a malformed one, reset the peer (RESET + cleanup) instead of a local-only reset, ignore bytes after reset, and apply a whole SCTP message in one trio hop - _decode_frames: derive the max prefix length from MAX_MESSAGE_SIZE Refs libp2p#1437
The test performs two full dials and two bounded peer-connection closes; on Windows each close can take ~5 s when a datagram write is in flight, so 30 s was flaky (seen once on CI, no socket errors logged). Refs libp2p#1437
f42dae9 to
a7bc836
Compare
β¦ collides Harness mode binds TCP on the same port number as the OS-chosen UDP mux port; Windows reserves port ranges per protocol, so that TCP bind can fail with WinError 10013 (seen on CI in the harness loopback test). Retry up to five times with a new UDP port before giving up; explicit ports still fail loudly. Refs libp2p#1437
a7bc836 to
466ca00
Compare
β¦epts v2, dialer opt-in Listener: stop dropping libp2p+webrtc+v2/ first contacts. The server ufrag minus the prefix is the dialer's ICE pwd (must be a valid one, 22..256 ice-chars, else rejected in parse_direct_username); the inferred offer uses client_ufrag + that pwd, and our local ufrag/pwd stay server_ufrag verbatim as in v1. _accept_v1 -> _accept(client_pwd=...). Dialer: new WebRTCTransportConfig.webrtc_direct_dial_version (default 1). v2 keeps the aioice ufrag/pwd (no munging) and sets the synthetic answer's ufrag == pwd to "libp2p+webrtc+v2/" + local_password. Default stays v1 while specs#715 is unmerged; unknown values raise WebRTCConnectionError. Tests: v2 parse/reject cases, SDP shape, v2 loopback echo, no-munge check, short-suffix rejection, mixed v1+v2 dialers on one listener. Refs libp2p#1437
- validate webrtc_direct_dial_version in the config (was only checked mid-dial after the PC was built) - parse_direct_username returns the validated client pwd; make_v2_credential; one pwd check instead of two version branches - inline the v1 credential writes; trim the dial() comment Refs libp2p#1437
466ca00 to
167a507
Compare
|
@seetadev @acul71 Ready for review β stacked on #1460, CI 37/37 green, rebased on current main (re-checked against the new varint error-type changes from #1462; |
acul71
left a comment
There was a problem hiding this comment.
π€ AI-Assisted Maintainer Review (acul71)
Reviewed on branch
feat/webrtc-direct-v2(0 behind / 10 ahead ofmain).
Checks:make lintβ Β·make typecheckβ Β·make testβ οΈ 3 pre-existing Kademlia flakes, 3459 passed Β·make linux-docsβ Β· merge conflicts: none.
Overall
Good, spec-faithful implementation of WebRTC-Direct v2 on top of #1449. The credential derivation, no-munge dialer path, _decode_frames byte-stream accumulation, and SCTP stream-id parity fix are all correct and well-commented. The 3 test failures are pre-existing Kademlia flakiness unrelated to this PR.
Quality: Good Β· Security impact: Low Β· Merge readiness: Needs minor fixes
π΄ Required before merge
[M1] _decode_frames β head[consumed - 1] wraps silently if consumed == 0
libp2p/transport/webrtc/stream.py, inside _decode_frames:
length, consumed = decode_varint_with_size(bytes(head))
if head[consumed - 1] & 0x80: # β wraps to head[-1] if consumed == 0decode_varint_with_size on a non-empty buf should always return consumed >= 1, but that assumption is not enforced here. Add a guard:
if consumed == 0:
raise ValueError("empty varint prefix")[M2] _apply_batch_on_trio_thread β processing continues after a RESET frame mid-batch
libp2p/transport/webrtc/stream.py:
for payload, flag in items:
if payload:
self._read_send.send_nowait(payload) # called after RESET
...
elif flag == Message.RESET:
self._reset_locally()
# loop continues β subsequent items still delivered to a reset streamAfter _reset_locally() sets StreamState.RESET, the inner loop keeps running. A malformed or adversarial peer could craft a batch with RESET followed by more data and still have it enqueued. Fix:
for payload, flag in items:
if self._state == StreamState.RESET:
break
...π‘ Should fix before merge
[m2] assert hasattr(dtls, "_role") is stripped by -O
libp2p/transport/webrtc/_aiortc_helpers.py line 358:
assert hasattr(dtls, "_role"), "RTCDtlsTransport has no attribute '_role'"assert is a no-op with PYTHONOPTIMIZE=1. If _role disappears in a future aiortc version the parity re-seed silently falls through and stream-id collisions reappear. Replace with:
if not hasattr(dtls, "_role"):
logger.warning(
"channel %d: RTCDtlsTransport has no '_role'; using aiortc default parity",
channel_id,
)
else:
role = dtls._role[m5] README needs an "experimental / spec pending" callout for webrtc_direct_dial_version=2
The README documents WebRTCTransportConfig(webrtc_direct_dial_version=2) without a visible note that specs#715 is still unmerged and v2 interop is not yet guaranteed. The config.py comment says this, but README readers won't see it. Add a short caveat next to the v2 example.
π’ Minor / optional
- [m1]
newsfragments/1437.{bugfix,feature}.rstare now very long single-line strings.towncrierreflows them fine, but they're hard to diff. Consider wrapping at ~80 chars. - [m3]
_data_channel_idis re-seeded before everycreateDataChannelcall. A comment explaining why repeating this on subsequent channels is safe (aiortc bumps past in-use ids) would help future maintainers. - [m4]
test_ids_follow_dtls_role_and_are_recycledusesasyncio.run(self._run())instead of@pytest.mark.trio. Not a blocker, but inconsistent with project convention.
Questions for @yashksaini-coder
- Stacking on #1460 β the current branch already contains all of #1460's commits (framing + stream-id parity). Is the plan to merge this PR and close #1460 as superseded, or to merge #1460 first and rebase?
- Mid-batch RESET (M2) β can a valid peer ever send a RESET frame followed by more data in the same batch? If not, the
breakguard is unconditionally safe to add. read(n=None)spin (minor) β thewhile not self._buffer and await self._fill(): passloop inDataChannelReadWritercould spin on a stream of flag-only frames with no payload bytes. Is there a Noise-handshake timeout that would bound this in practice?- Go/JS interop follow-up β is there an issue or planned PR for the interop tests mentioned in the PR description?
β¦ RESET guards - _create_channel: a missing RTCDtlsTransport._role now degrades to aiortc's default stream-id parity with a warning instead of asserting (an assert in a live connection path, also stripped under -O) - _decode_frames: reject consumed == 0 from the varint decoder rather than silently indexing head[-1] - _reset_locally is idempotent; _apply_batch_on_trio_thread stops delivering a batch's remaining frames once the stream is RESET - note in DataChannelReadWriter._fill that frames after FIN/RESET are dropped by design; wrap the newsfragment Refs libp2p#1437
Review follow-up on libp2p#1459: README readers should see that libp2p/specs#715 is unmerged and cross-implementation v2 interop is not yet guaranteed (config.py already said so). Refs libp2p#1437
|
@acul71 All items in via
Q1 (stacking): merge #1460 first and I'll rebase this, or merge this and close #1460 as superseded β your call, both are ready. Q3 (flag-only spin): each |
What
The v2 flow from libp2p/specs#715, on top of the v1 listener that landed in #1449.
libp2p+webrtc+v2/first contacts alongside v1: recoversclient_pwdby stripping the prefix (rejected unless a valid ice-pwd, 22β256), infers the offer withice-ufrag=client_ufrag/ice-pwd=client_pwd, and sets its own ufrag and pwd toserver_ufragverbatim. Unknown/missing prefix still rejected; v1 unchanged (pwd = client half).WebRTCTransportConfig(webrtc_direct_dial_version=2): no munging β it reads its aioice-generated pwd and encodes it in the synthetic answer's ufrag/pwd aslibp2p+webrtc+v2/<client_pwd>. Default stays v1 until specs#715 merges.Verified
Stacked on #1460. Refs #1437 (v2 item); go/js interop tests come next.