feat(gateway): load balancing - #370
Conversation
|
💬 Discussion in Slack: #pr-review-cli-370-feat-gateway-report-active-connection-count-for-pool-load-bala Posted by Review Police — reviews, comments, new commits, and CI failures will stream into this channel. |
|
| Filename | Overview |
|---|---|
| packages/api/api.go | Adds the v2 gateway load-report endpoint wrapper; its synchronous request inherits the shared client's lack of a timeout. |
| packages/api/model.go | Adds the activeChannels request DTO with the expected JSON field. |
| packages/gateway-v2/gateway.go | Adds atomic channel accounting and periodic reporting, with a non-blocking concern around indefinitely stalled HTTP requests. |
Reviews (1): Last reviewed commit: "feat(gateway): report active channel cou..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
bernie-g
changed the title
Counts channels in handleIncomingChannel, the single point every inbound channel passes through whatever opened it, and reports the total every 10s so the platform can route new work to the least busy member of a pool. Kept off the heartbeat deliberately: a heartbeat makes the platform dial back through the relay and write to its database, which is far too costly at the cadence selection needs. A missed report only costs accuracy, so failures are logged at debug and the platform falls back to its own view.
The report ran with no deadline, so an endpoint that accepts the connection and then stalls would block the reporting loop indefinitely: no further reports, and no reaction to cancellation either, so shutdown would hang behind it. Each report now runs on a context with a 5s deadline derived from the gateway's own context, which keeps a stall from reaching the next tick and lets cancellation abort a request already in flight.
c8f7b62 to
1d66d56
Compare
Clears GO-2026-6163, a panic on a malformed XOR-MAPPED-ADDRESS attribute. Reached through pion/turn in the gateway relay path. v3.1.5 is the first fixed release; later patches pull in dtls and transport bumps this does not need.
The e2e module replaces the root module, so its go.sum needs the same versions or every e2e job fails with "updates to go.mod needed".
| } | ||
| defer channel.Close() | ||
|
|
||
| g.activeChannels.Add(1) |
There was a problem hiding this comment.
Claude:
This counter only goes down via the defer, so a leaked goroutine inflates it permanently and the gateway stops getting work. Could we reset it to 0 on each new relay connection?
| // Still republish an unchanged count so the platform can tell a quiet gateway from | ||
| // one that has stopped reporting. | ||
| if err := g.sendLoadReport(ctx, count); err != nil { | ||
| log.Debug().Msgf("Load report failed: %v", err) |
There was a problem hiding this comment.
An older self hosted platform has no metrics endpoint, so this retries every 10s forever. Maybe we should have some back-off mechanism that stops the report after X failures?
| } | ||
| } | ||
|
|
||
| // reportLoad publishes the gateway's active channel count so the platform can route new work to the |
There was a problem hiding this comment.
The comment mentions "reportLoad" but this is the "sendMetricsReport" function
There was a problem hiding this comment.
Also this comment may be for "reportMetrics" instead of "sendMetricsReport"? Not 100% sure
There was a problem hiding this comment.
Also maybe we should rename "reportMetrics" to "startMetricsReport" to be more clear
3 participants
Description 📣
The gateway now keeps a count of how many connections it is currently handling and reports that number to Infisical every 10 seconds. This lets gateway pools send new work to whichever gateway is least busy, including work the platform never sees directly such as PAM sessions opened from the CLI.
Pairs with the platform change in Infisical/infisical#7703. Older gateways simply don't report, and the platform falls back to its own view for those.
Type ✨
Tests 🛠️
Ran two gateways in a pool and watched the reported count while holding connections open. It rises with each connection, holds while they are open, and returns to zero when they close. Verified this includes a PAM session opened with
infisical pam access, which the platform hands certificates to and never proxies itself.