Skip to content

A gateway node failure blackholes the tenant egress it was holding #407

Description

@ecv

What needs to happen

A tenant that turns on egress gets assigned one gateway node, once. Every compute node holding that tenant then installs a default route pointing at that node, and all of the tenant's outbound traffic encapsulates toward it.

Nothing recomputes that assignment. If the assigned node goes down, is drained, or loses its uplink, the default route keeps pointing at it. The tenant loses all outbound connectivity and stays that way until an operator notices and deletes the policy so it can be reassigned.

Desired outcome: detect that the assigned gateway node is no longer serving, pick another one, and have the compute nodes follow the new assignment.

Why this matters

Ingress already survives this. Every gateway node advertises every rule it serves, and traffic moves to whichever one is still up. Egress has no equivalent, so the same node failure that ingress rides out takes egress down completely for whoever was assigned to it.

The blast radius is the whole tenant rather than one connection, and the recovery is manual.

Status

This is a known and stated gap in the design, listed as a phase one non goal, not something that slipped through. Filed so it is tracked somewhere other than a plan document before the feature is offered to anyone.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions