perf: Encode FDv2 polling payloads in a single jwriter pass - #797
Open
keelerm84 wants to merge 3 commits into
Open
perf: Encode FDv2 polling payloads in a single jwriter pass#797keelerm84 wants to merge 3 commits into
keelerm84 wants to merge 3 commits into
Conversation
Whole-handler benchmarks for pollHandlerV2 (/sdk/poll) and pollEvalHandlerV2Shared (/sdk/poll/eval) over a realistic data set (flags with rules and targets, plus segments) at 100 and 2000 flags. These establish the baseline for optimizing the payload serialization, which live traces show dominating /sdk/poll request time.
The FDv2 polling handlers built their response by marshaling every item into its own jwriter buffer, boxing each event into an any-typed struct, and then running encoding/json over the whole document -- which walks it by reflection and re-scans every embedded raw JSON object through its compact() validator before copying it into yet another buffer. Replace that with fdv2PayloadWriter, which writes the entire document in one jwriter pass; put-object bodies are marshaled directly into the output buffer. This is the same treatment the FDv2 SSE events received in the streaming path, and the encoder is pinned to encoding/json's output by equivalence tests (structurally equal; jwriter does not HTML-escape, which the streaming encoders already ship). Benchmarks (2000 flags): /sdk/poll 29.5ms -> 13.3ms per request (-55%) with allocations down from 15046 to 45 (-99.7%); /sdk/poll/eval 1.97ms -> 1.54ms (-22%) with allocations halved, the remainder being evaluation itself. The pollingPayload/payloadEvent types move into test code, where they still describe the wire format that responses are unmarshaled through.
The polling fdv2PayloadWriter and internal/streams' SSE encoders spell out the same protocol event shapes in separate jwriter code. These tests run the same inputs through both and require identical event names and data JSON, so a protocol-shape change in either implementation fails until both move. The streaming encoders' output shapes were otherwise almost unasserted. The comparison also covers the intent payloads: the streams package derives the up-to-date and full-transfer intents from the selector internally, so the payloads the polling handlers construct must match.
keelerm84
marked this pull request as ready for review
August 7, 2026 13:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Medium Risk
Touches the wire format and serialization path for server- and client-side FDv2 polling, but behavior is heavily pinned to prior JSON and streaming encoders; main risk is subtle protocol or escaping differences under load.
Overview
FDv2 polling responses (
pollHandlerV2and client-sidepollEvalHandlerV2Shared) no longer buildpollingPayloadstructs andjson.Marshalthem. They now stream the{"events":[...]}document through a newfdv2PayloadWriter, writingserver-intent,put-object(with flag/segment/eval bodies inlined viabeginPutObject/endPutObject), andpayload-transferredin one buffer. That cuts per-item intermediate bytes andencoding/jsonwork on large snapshots.Test helpers
pollingPayload/payloadEventmove toendpoint_tests_base_test.gowith a note that handlers use the writer. New tests assert the writer matches priorencoding/jsonoutput andinternal/streamsSSE event shapes; benchmarks exercise the poll handlers at 100 and 2000 flags.Reviewed by Cursor Bugbot for commit 0976340. Bugbot is set up for automated code reviews on this repo. Configure here.