sdk: dial routers over WebSocket, so both SDKs join sam-one - #515
Conversation
6d574cd to
714f45e
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces WebSocket transport support to the JS SDK, enabling browser-compatible multi-agent communication over single-port WebSocket listeners. Key changes include refactoring cryptographic operations to use noble curves, replacing Node-specific Buffer with Uint8Array, dynamically importing Node.js built-ins to improve portability, and adding comprehensive integration and UI tests. The feedback highlights several critical improvements: in host.ts, you should check if the abort signal is already aborted in sendClose to prevent hanging promises, add support for the dnsaddr protocol, and remove a redundant type cast. Additionally, across biscuit.ts and mesh.ts, you should use variable indirection for dynamic imports of Node.js built-ins to prevent static analysis and build failures in browser bundlers.
714f45e to
bb63465
Compare
bb63465 to
16f44b9
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request adds WebSocket transport support to both the JS and Python SDKs, enabling them to connect to routers (such as sam-one) that listen on WebSockets. It also adds integration tests verifying SDK compatibility with sam-one, including scenarios behind a TLS-terminating edge. The review feedback suggests several improvements: ensuring case-insensitive comparisons for HTTP Host headers and TLS SNI server names in the test TLS edge using strings.EqualFold, explicitly passing loop-dependent variables to goroutines in Go tests, adding a defensive check before monkeypatching WebsocketTransport in Python, and using nested try...finally blocks in JS tests to prevent port leaks if host creation fails.
| }} | ||
| server := &http.Server{ | ||
| Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
| if h, _, err := net.SplitHostPort(r.Host); (err == nil && h != host) || (err != nil && r.Host != host) { |
There was a problem hiding this comment.
HTTP Host headers are case-insensitive per RFC 7230. Using a case-sensitive comparison (h != host) can lead to unexpected 403 Forbidden errors if a client sends the host header with different casing (e.g., Localhost). Use strings.EqualFold for a case-insensitive comparison instead.
| if h, _, err := net.SplitHostPort(r.Host); (err == nil && h != host) || (err != nil && r.Host != host) { | |
| if h, _, err := net.SplitHostPort(r.Host); (err == nil && !strings.EqualFold(h, host)) || (err != nil && !strings.EqualFold(r.Host, host)) { |
There was a problem hiding this comment.
Applied in 154df54 (also the SNI comparison and the nested try/finally in host.test.ts).
| TLSConfig: &tls.Config{ | ||
| MinVersion: tls.VersionTLS12, | ||
| GetCertificate: func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) { | ||
| if hello.ServerName != host { |
There was a problem hiding this comment.
TLS SNI server names are case-insensitive per RFC 6066. A case-sensitive comparison (hello.ServerName != host) can cause handshake failures if a client sends the SNI with mixed casing. Use strings.EqualFold to perform a case-insensitive comparison.
| if hello.ServerName != host { | |
| if !strings.EqualFold(hello.ServerName, host) { |
| go func(i int) { | ||
| defer wg.Done() | ||
| agents[i], errs[i] = startExampleAgent(t, launchers[i].name, cmd) | ||
| }(i) |
There was a problem hiding this comment.
Passing cmd as a parameter to the goroutine avoids relying on Go 1.22's loop block scoping behavior and prevents potential data races or accidental sharing if the loop is refactored in the future. It is safer and more idiomatic to pass all loop-dependent variables explicitly to the concurrent goroutine.
| go func(i int) { | |
| defer wg.Done() | |
| agents[i], errs[i] = startExampleAgent(t, launchers[i].name, cmd) | |
| }(i) | |
| go func(i int, cmd *exec.Cmd) { | |
| defer wg.Done() | |
| agents[i], errs[i] = startExampleAgent(t, launchers[i].name, cmd) | |
| }(i, cmd) |
There was a problem hiding this comment.
cmd is not the loop variable; it is declared inside the loop body (cmd, _ := l.cmd(...)), so every iteration has its own cmd in every Go version, and the closure captures that iteration's. i is the loop variable and is passed as a parameter. Kept as is.
| def _dial_websockets_by_name(transport: WebsocketTransport) -> None: | ||
| """py-libp2p 0.7 resolves a `/dns4/<host>/tcp/443/wss` address to its IP | ||
| before dialing and then names the IP in the TLS SNI and the Host header, | ||
| which a TLS-terminating edge (sam-one behind a tunnel) answers with 403. | ||
| The transport's own dial of an unresolved address keeps the name; the | ||
| dial here goes straight to it, as go-libp2p and js-libp2p do.""" | ||
|
|
||
| async def dial(maddr: multiaddr.Multiaddr): # noqa: ANN202 - py-libp2p's RawConnection | ||
| return await transport._dial_resolved(maddr) # noqa: SLF001 | ||
|
|
||
| transport.dial = dial # type: ignore[method-assign] |
There was a problem hiding this comment.
The monkeypatch relies on the private method _dial_resolved of WebsocketTransport. If a future version of py-libp2p modifies or removes this private method, it will raise an AttributeError at runtime. Adding a defensive check using hasattr ensures the host can still start up (falling back to default dial behavior) even if the internal implementation changes.
| def _dial_websockets_by_name(transport: WebsocketTransport) -> None: | |
| """py-libp2p 0.7 resolves a `/dns4/<host>/tcp/443/wss` address to its IP | |
| before dialing and then names the IP in the TLS SNI and the Host header, | |
| which a TLS-terminating edge (sam-one behind a tunnel) answers with 403. | |
| The transport's own dial of an unresolved address keeps the name; the | |
| dial here goes straight to it, as go-libp2p and js-libp2p do.""" | |
| async def dial(maddr: multiaddr.Multiaddr): # noqa: ANN202 - py-libp2p's RawConnection | |
| return await transport._dial_resolved(maddr) # noqa: SLF001 | |
| transport.dial = dial # type: ignore[method-assign] | |
| def _dial_websockets_by_name(transport: WebsocketTransport) -> None: | |
| """py-libp2p 0.7 resolves a `/dns4/<host>/tcp/443/wss` address to its IP | |
| before dialing and then names the IP in the TLS SNI and the Host header, | |
| which a TLS-terminating edge (sam-one behind a tunnel) answers with 403. | |
| The transport's own dial of an unresolved address keeps the name; the | |
| dial here goes straight to it, as go-libp2p and js-libp2p do.""" | |
| if not hasattr(transport, "_dial_resolved"): | |
| return | |
| async def dial(maddr: multiaddr.Multiaddr): # noqa: ANN202 - py-libp2p's RawConnection | |
| return await transport._dial_resolved(maddr) # noqa: SLF001 | |
| transport.dial = dial # type: ignore[method-assign] |
There was a problem hiding this comment.
A silent fallback would be the wrong failure: the default dial is what returns 403 behind a TLS edge, so a py-libp2p that renamed _dial_resolved would make the host "start up" and then fail every wss dial far from the cause. libp2p is pinned >=0.7,<0.8, so the private method is stable for what installs; for anything else, 154df54 binds transport._dial_resolved at host construction, so a rename raises there, once, with the attribute named. The fix belongs upstream and is being sent to py-libp2p.
| const client = await createMeshHost(Identity.generate()); | ||
| try { | ||
| const addr = multiaddr(`/dns4/localhost/tcp/${port}/ws/p2p/${Identity.generate().peerId}`); | ||
| await assert.rejects(client.dial(addr, { signal: AbortSignal.timeout(5_000) })); | ||
| const head = (await request).split("\r\n").map((l) => l.toLowerCase()); | ||
| assert.ok(head.includes(`host: localhost:${port}`), `upgrade request:\n${head.join("\n")}`); | ||
| } finally { | ||
| await client.stop(); | ||
| server.close(); | ||
| } |
There was a problem hiding this comment.
If createMeshHost throws an error, the try block is never entered, which means the finally block is skipped and the listening server is leaked. Wrapping the host creation and test execution in nested try...finally blocks ensures that the TCP server is always closed properly, preventing port leaks in tests.
try {
const client = await createMeshHost(Identity.generate());
try {
const addr = multiaddr(`/dns4/localhost/tcp/${port}/ws/p2p/${Identity.generate().peerId}`);
await assert.rejects(client.dial(addr, { signal: AbortSignal.timeout(5_000) }));
const head = (await request).split("\r\n").map((l) => l.toLowerCase());
assert.ok(head.includes(`host: localhost:${port}`), `upgrade request:\n${head.join("\n")}`);
} finally {
await client.stop();
}
} finally {
server.close();
}f121859 to
154df54
Compare
sam-one's embedded router listens on WebSocket alone, on the single port that also serves its control plane, and both SDKs dialed TCP only, so the "Run your own" path of the Native SDKs guide admitted neither. The JS host adds @libp2p/websockets and the Python host py-libp2p's WebSocket transport (enable_websocket); the Python address filter lets /ws and /wss through. Both handle /wss and /tls/ws, the form sam-one advertises behind a tunnel. Behind `sam-one --tunnel cloudflare` the router is /dns4/<host>.trycloudflare.com/tcp/443/wss/p2p/<id>, and the edge selects the origin by the TLS server name and the Host header. py-libp2p 0.7 resolves the name to its IP before dialing and then names the IP in both, which Cloudflare answers with 403; the Python host dials a WebSocket address by its name, as go-libp2p and js-libp2p do. Verified against a Cloudflare quick tunnel in both directions. TestStandaloneSDKAgents starts sam-one, runs each language's A2A agent example against it and calls it with the other language's A2A caller; TestStandaloneSDKAgentsBehindTLSEdge does the same behind a TLS-terminating edge in the test that, like Cloudflare, serves only a client naming it in SNI and Host. Without either transport the agent of that language reports that no router admitted it; without the name-preserving dial the Python agent fails the edge's TLS handshake. host.test.ts and test_host.py dial a /ws listener and pin the Host header of a /dns4 dial in each SDK. runExample takes the repository root so a test without an sdkMesh can use it. The browser stays out of scope: routers and nodes accept libp2p TLS alone, which a browser cannot speak, and the SDK's state and HTTP on streams use Node's modules; the docs say so.
154df54 to
e6192e5
Compare
Both SDKs now join
sam-one, directly and behind--tunnel cloudflare, and tests hold them to both.Why
sam-one's embedded router listens on WebSocket alone, on the single port that also serves its control plane (/ws, or/wssbehind--tunnel). Both SDKs dialed TCP only, so the "Run your own" path of the Native SDKs guide admitted neither of them. Nothing tested that path.Change
sdk/js/src/host.ts):@libp2p/websocketsjoinstcp()in the host's transports. It is the maintained transport (backpressure,/ws,/wss,/tls/sni/…/ws, listeners), declared inpackage.json; the lockfile gains it and its four dependencies.sdk/python/src/agent_mesh/host.py):new_host(enable_websocket=True)adds py-libp2p 0.7's WebSocket transport; the address filter no longer drops/wsand/wss.sam-one --tunnel cloudflarethe router is/dns4/<host>.trycloudflare.com/tcp/443/wss/p2p/<id>, and Cloudflare selects the origin by TLS SNI and theHostheader. py-libp2p 0.7 resolves the name to its IP before dialing and then puts the IP in both, which Cloudflare answers with403 Forbidden. The Python host now dials a WebSocket address by its name (what go-libp2p and js-libp2p do), anddial_addrsno longer pre-resolves/dns4on WebSocket addresses. This is a py-libp2p bug, present on itsmaintoo; to be reported upstream.wsswithverify_mode=CERT_NONEunless given a TLS client context. The host passesssl.create_default_context(), the system roots, as the JS (ws→ Node TLS) and Go hosts verify.Verified against a real tunnel
bin/sam-one --tunnel cloudflare --tunnel-install, then throughhttps://<x>.trycloudflare.com: JS agent ← Python caller (2.4 s) and Python agent ← JS caller (3.0 s), both answering with the verified caller. The first Python attempt was the 403 above; the fix is what made it pass.Tests
TestStandaloneSDKAgents(~4.5 s): startssam-one, runs each language's A2A agent example against it and calls it with the other language's A2A caller (js-calls-python,python-calls-js). Mutation-checked: with the JS transport removed the JS agent fails with "no router admitted this member"; with the Python transport removed the Python agent fails with "no transport found for …/ws/…".TestStandaloneSDKAgentsBehindTLSEdge(~4.5 s): the same pair behind an in-test TLS-terminating reverse proxy reached ashttps://localhost:<port>that, like Cloudflare, fails the handshake for any other SNI and answers any otherHostwith 403;sam-onegets it asExternalURLand advertises/dns4/localhost/…/wss. Both SDKs trust the edge certificate throughNODE_EXTRA_CA_CERTS/SSL_CERT_FILE. Mutation-checked: without the name-preserving dial the Python agent fails the edge's TLS handshake ("unknown server name"); withoutSSL_CERT_FILEit fails withCERTIFICATE_VERIFY_FAILED.host.test.ts: a mesh host dials/wsand/tcplisteners with TLS+yamux; a/dns4WebSocket dial carries the name inHost.test_host.py:/wsand/tls/wsaddresses are dialable; a host reaches a/wslistener; a/dns4WebSocket dial carries the name inHost(fails with127.0.0.1without the fix); wss is dialed withCERT_REQUIREDand hostname checking (fails without the context).runExampletakes the repository root instead of ansdkMesh.Docs
sdk/README.mdand the Native SDKs guide say the SDKs dial TCP and WebSocket and whatsam-onelistens on. The browser stays out of scope, stated precisely: routers and nodes accept libp2p TLS alone, which a browser cannot speak (that needs Noise on the Go side), and the SDK keeps its state and speaks HTTP on streams with Node's modules.Not in this PR (was in the first version)
@libp2p/websockets.Buffer/node:crypto/node:fsportability refactor: it leftlibp2p-http.tsonnode:http, so it did not make the package browser-loadable, and it imported@noble/curvesand@multiformats/multiaddr-matcherwithout declaring them.tests/ui/agents.spec.jsand the runner split: the sessions ran in the Playwright Node process, not in the browser, and the console coverage exists inconsole.spec.js.session.serve, which the SDK no longer has.Validation:
npm run typecheck,npm test(70),pytest(82),TestStandaloneSDKAgents*|TestStandaloneNodeJoin|TestNativeSDKsMesh|TestNativeSDKA2A|TestNativeSDKExamples|TestSDKCanaryScript,hack/verify-sdk-generated.sh,hack/verify-secrets.sh, golangci-lint v2.14.0 ontests/integration.