Skip to content

fix(request-response): debug_assert panic on last connection close - #6601

Open
procdump wants to merge 1 commit into
libp2p:masterfrom
procdump:fix/panic-request-response-connection-tracking-assert_main
Open

fix(request-response): debug_assert panic on last connection close#6601
procdump wants to merge 1 commit into
libp2p:masterfrom
procdump:fix/panic-request-response-connection-tracking-assert_main

Conversation

@procdump

@procdump procdump commented Sep 3, 2026

Copy link
Copy Markdown

Description

The problem

I've been experiencing panics with debug builds in some tests around the
rayls blockchain, where rust-libp2p v0.56.0 is used.

After some investigation it turns out request-response records a connection in self.connected
from handle_established_inbound_connection / handle_established_outbound_connection. Those run at
handler-creation time, before the swarm has committed to the connection. In a composed
NetworkBehaviour the derive calls each field's handle_established_* in order and ?-propagates
the result, so a sibling ordered after request-response returning ConnectionDenied aborts the
connection once we have already recorded it. Neither ConnectionEstablished nor ConnectionClosed
ever follows for that id, and the entry is left behind as a phantom.

That has two consequences:

  • self.connected desynchronises from remaining_established, so closing the peer's last real
    connection reports remaining_established == 0 against a non-empty list and trips the
    debug_assert_eq! in on_connection_closed. This is the panic I was seeing.
  • For as long as the phantom is there the peer looks connected, so send_request notifies a
    connection_id that has no handler behind it instead of dialing the peer. This one is silent and
    happens in release builds too.

What's changed

A self.connected entry is now created on FromSwarm::ConnectionEstablished, the first point at
which the swarm has committed to the connection, so connected mirrors the swarm's established set
by construction. handle_established_* become pure handler factories with no side effects.

Queued requests can no longer be preloaded into the handler at construction, so they are emitted as
ToSwarm::NotifyHandler { handler: NotifyHandler::One(connection_id), .. } instead. Same queue, same
connection — they are just dispatched on the next poll rather than baked into the handler.

AI Assistance Disclosure

Tools used (required — write none if no AI was used): Claude Code

Claude Code wrote the two regression tests and helped explore alternative fixes, including checking
whether the change could affect consumers negatively. I reviewed and verified the result.

Attestation (required):

  • I have read every line of this diff, understand what it does, and can explain it in review.

Notes & open questions

Reproducing

Keep the two new tests and restore the old recording logic:

git checkout master -- protocols/request-response/src/lib.rs
cargo test -p libp2p-request-response --test connection_tracking

Further steps

I also have this backported to 0.56.0 here where I no longer observe the panic after a few nights of testing. In this regard if this PR gets merged is it possible that 0.56.1 gets published with this fix in?

I'd be happy to have this resolved so share your feedback.

Recording at handle_established_* left a phantom entry when a sibling
behaviour denied the connection, tripping the on_connection_closed
debug_assert and mis-routing requests to a dead connection. Record on
FromSwarm::ConnectionEstablished instead, so `connected` mirrors the
swarm's committed set.
@procdump
procdump marked this pull request as draft September 3, 2026 12:00
@procdump
procdump marked this pull request as ready for review September 3, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant