perf: Deduplicate concurrent polling requests with a flight group - #798
Draft
keelerm84 wants to merge 5 commits into
Draft
perf: Deduplicate concurrent polling requests with a flight group#798keelerm84 wants to merge 5 commits into
keelerm84 wants to merge 5 commits into
Conversation
The streaming endpoints have always collapsed concurrent replay builds through a singleflight group, but every polling request snapshotted and serialized the store on its own. The two server-side full-payload polling endpoints (GET /sdk/poll and GET /sdk/flags) now share one payload build per environment through a flight group exposed by the EnvContext, keyed by the FDv2 basis where the payload depends on it. The per-context evaluation endpoints and the per-key PHP endpoints are unchanged: their results rarely collide, so there is little duplicate work to share. The store and serialize spans belong to the one request that executes the build; every request records relay.singleflight.shared on its request span so a trace without those child spans is explainable.
…time A request that received its payload from a flight another request was executing now records how long it waited as relay.singleflight.wait_ms on its request span. The executing request carries no wait attribute -- it did not wait, and its time is visible as the store and serialize child spans -- so the attribute's presence alone identifies a request that waited, and slow waits are queryable. Both polling handlers now resolve their flights through a shared runPollingFlight helper that owns the Do call and both span annotations. That indirection also breaks the taint chain behind the gosec G705 false positive on writeCacheableJSONResponse, so its nolint directive is removed.
The streaming repositories' replay flight groups now record the same information the polling endpoints do: relay.singleflight.shared on the subscribing request's span, plus relay.singleflight.wait_ms when a replay waited on a flight another subscriber was already executing. The annotation logic moves into tracing.SingleflightDo so both sides record identical information by construction. The flags-only repository never implemented ReplayWithContext, so it had no request context to annotate; it now advertises context support the same way the main server-side repository does.
Review feedback: ReplayWithContext's comment claimed the context was only used for telemetry, but cancellation-on-disconnect is the reason the eventsource RepositoryWithContext interface exists. The flags-only replay now behaves like the main server-side repository: it skips the payload build if the subscriber is already gone and abandons the send on disconnect instead of relying on the eventsource server draining the channel on its behalf. The flight functions' telemetry-only comments now explain why they do not cancel: a flight in progress may be shared with other waiting subscribers, so disconnect handling belongs to replay's send loop.
A request that waited on another request's payload build used to show that wait as unexplained empty space between its child spans, with only the relay.singleflight.wait_ms attribute to account for it. The wait is now also emitted as a relay.singleflight.wait child span covering exactly the waiting window, so trace timelines read directly. The span is back-dated: whether a caller waited (rather than executed) is only known once the flight resolves, so it cannot be opened beforehand without giving the executing caller a bogus wait span. It comes from the provider owning the surrounding request span, so it lands wherever that span is recorded.
keelerm84
force-pushed
the
mk/SDK-2877/singleflight-for-poll
branch
from
August 7, 2026 16:12
4bc9aaf to
c6da085
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The streaming endpoints have always collapsed concurrent replay builds
through a singleflight group, but every polling request snapshotted and
serialized the store on its own. The two server-side full-payload
polling endpoints (GET /sdk/poll and GET /sdk/flags) now share one
payload build per environment through a flight group exposed by the
EnvContext, keyed by the FDv2 basis where the payload depends on it.
The per-context evaluation endpoints and the per-key PHP endpoints are
unchanged: their results rarely collide, so there is little duplicate
work to share.
The store and serialize spans belong to the one request that executes
the build; every request records relay.singleflight.shared on its
request span so a trace without those child spans is explainable.