Skip to content

feat(eap): Support collecting distinct values into an array on TraceItemTable - #8256

Merged
pbhandari merged 4 commits into
masterfrom
feat/aggregate_on_arrays
Aug 10, 2026
Merged

feat(eap): Support collecting distinct values into an array on TraceItemTable#8256
pbhandari merged 4 commits into
masterfrom
feat/aggregate_on_arrays

Conversation

@pbhandari

@pbhandari pbhandari commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds support for FUNCTION_COLLECT_UNIQUE in the TraceItemTable endpoint. Unlike every other aggregation — which reduces a group to a single scalar — this one gathers each group's distinct scalar values into an array, returned as a val_array in the response. It maps to ClickHouse's groupUniqArray.

Related

Changes

Aggregation expression (aggregation.py)

  • Map FUNCTION_COLLECT_UNIQUE to groupUniqArrayIf(max_array_size), in both the plain and FUNCTION_ANY-style non-rounded branches (the result is an array, so round() must not be applied).
  • Reject extrapolation for FUNCTION_COLLECT_UNIQUE: scaling a set of values by a sampling weight is meaningless, so get_extrapolated_function raises BadSnubaRPCRequestException rather than silently producing a nonsensical result.
  • Refactored get_extrapolated_function to hoist the shared condition and rounded_sampling_weight sub-expressions out of the per-function map, removing the repeated inline and_cond(...) / cast boilerplate.

Response conversion (trace_item_table.py)

  • New _get_aggregate_converter centralizes the aggregate-result → AttributeValue mapping: FUNCTION_COLLECT_UNIQUE → array converter, FUNCTION_ANY → key-typed converter, everything else → double. Replaces the duplicated FUNCTION_ANY special-casing across the aggregation and conditional_aggregation branches.

Config (schema.json)

  • New snuba_query_max_array_size option (default 1000). Bounds how many distinct values groupUniqArray accumulates per group. Unlike uniq, groupUniqArray holds every distinct value on the aggregating node, so this caps its memory footprint; a group exceeding the limit is truncated rather than allowed to grow unbounded.

Tests

  • test_aggregation.py: FUNCTION_COLLECT_UNIQUE is rejected under every extrapolation mode, and produces a groupUniqArrayIf expression when extrapolation is off.
  • test_endpoint_trace_item_table.py: end-to-end coverage for the aggregation (returns val_array per group) and for the conditional-aggregation variant (collects distinct values of one column filtered on another).

🤖 Generated with Claude Code

@pbhandari
pbhandari requested review from a team as code owners August 3, 2026 14:57

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5904a20. Configure here.

Comment thread snuba/web/rpc/v1/resolvers/common/aggregation.py

@wmak wmak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As Discussed, would be good to know if this is a function we can expose to customers publicly via the general Events Query Syntax, or if this is something we want to keep private for use only on the specific AI conversations endpoint.

assert [v.WhichOneof("value") for v in collected] == ["val_array"]
# Only red rows contribute; "mobile" appears once despite two red occurrences,
# and "frontend" (blue) is filtered out. groupUniqArray gives no element ordering.
assert sorted(e.val_str for e in collected[0].val_array.values) == ["backend", "mobile"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for my sanity, would it be possible to add one more test that checks more than two elements? maybe 3-4?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@pbhandari
pbhandari force-pushed the feat/aggregate_on_arrays branch from 5904a20 to 1ebdf84 Compare August 6, 2026 16:40
Co-Authored-By: Prajjwal Bhandari <prajjwal.bhandari@sentry.io>
@pbhandari

Copy link
Copy Markdown
Contributor Author

@wmak I think this is probably best left internal only for now.

@pbhandari
pbhandari enabled auto-merge (squash) August 10, 2026 14:58
@pbhandari
pbhandari merged commit df6b5b4 into master Aug 10, 2026
69 checks passed
@pbhandari
pbhandari deleted the feat/aggregate_on_arrays branch August 10, 2026 15:15
@linear-code

linear-code Bot commented Aug 10, 2026

Copy link
Copy Markdown

EAP-623

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants