Add allow_ddm_traffic to PortConfig - #11091
Open
andrewjstone wants to merge 17 commits into
Open
Conversation
Add a new multirack join service tokio task that runs inside the sled-agent on demand. It's run in the same manner as RSS, shares the same context, and is mutually exclusive with RSS. Two new bootstrap-agent-lockstep APIs were added in order to trigger a multirack join and get its ongoing status. Unlike RSS, a `MultirackJoinRequest` can be resent in the case of a typo, misconfiguration, or runtime error. In many cases this will allow the join process to correct itself without requiring a clean slate of the rack. This is all managed via input and output watch channels and therefore obviates the need for worrying about channel bounds. As this PR started to get large, I have only implemented the first part of the multirack join service which sets up the trust quorum. This was enough to validate the starting of the service, the watch channel plumbing, and the ability to correct mistakes in the `MultirackJoinRequest` that triggers the behavior of the service. In order to test this new API, I used [voxel](oxidecomputer/voxel#13) with a 2 rack configuration of 3 sleds each. On the second, non-RSS sled, I logged into `g3` and ran curl requests against the bootstrap-agent-lockstep server. In order to help this testing, I created a one off tool to generate example JSON output from a hardcoded configuration. I expect this to be useful for the remainder of the implementation and then removed once the implementation is complete. Another large chunk of this code is just integrating with the existing bootstrap agent, wicket, and wicketd functionality. This should remain static for the remainder of the implementation.
andrewjstone
requested review from
cfzimmerman,
jgallagher,
nicolaskagami and
taspelund
August 14, 2026 18:07
andrewjstone
force-pushed
the
multirack-join-service-part-2
branch
from
August 16, 2026 15:15
6cc81ce to
238e6b3
Compare
andrewjstone
force-pushed
the
multirack-join-service-part-3
branch
from
August 16, 2026 15:44
50dd23a to
e71b23d
Compare
This builds upon #10894 and adds support for starting sled agents after trust quorum completes. Combined, the two PRs implement 2/3 of the requirements for the first cut of the multirack join service described in #10637. The remaining part is to bring the front ports on line for serving DDM traffic and and announcing the new rack prefix over DDM.
andrewjstone
force-pushed
the
multirack-join-service-part-2
branch
from
August 17, 2026 22:00
238e6b3 to
8ba5f1c
Compare
andrewjstone
force-pushed
the
multirack-join-service-part-3
branch
from
August 17, 2026 22:02
e71b23d to
5cea605
Compare
andrewjstone
force-pushed
the
multirack-join-service-part-2
branch
from
August 18, 2026 20:36
8ba5f1c to
8f9463f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is necessary to both enable front ports for DDM and disable front ports as uplink ports in dpd. Both are necessary for multirack.
It's important to note that this is a temporary fix to get multirack moving. We expect to overhaul the structure of
RackNetworkConfigandPortConfigsoon. BgpPeer info should likely live outside the port itself and ports should only contain link-level information.