feat: add cross-shard HAVING support for grouped aggregates by murex971 · Pull Request #1841 · pgdogdev/pgdog

murex971 · 2026-10-16T16:26:33Z

closes #695

codecov · 2026-10-16T16:30:29Z

Codecov Report

❌ Patch coverage is 97.73414% with 15 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...frontend/router/parser/rewrite/statement/having.rs	95.31%	9 Missing ⚠️
...tend/router/parser/rewrite/statement/projection.rs	97.91%	3 Missing ⚠️
...end/client/query_engine/test/rewrite_projection.rs	96.42%	2 Missing ⚠️
pgdog/src/backend/pool/connection/buffer.rs	98.38%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

github-actions · 2026-10-16T16:35:58Z

Bencher Report

Project	PgDog
Branch	murex-having-support
Testbed	Intel v1

Click to view all benchmark results

Benchmark	Throughput	Benchmark Result operations / second (ops/s) x 1e3 (Result Δ%)	Lower Boundary operations / second (ops/s) x 1e3 (Limit %)	cpu-per-txn	Benchmark Result Measure (units) (Result Δ%)	Upper Boundary Measure (units) (Limit %)
lb-v3	📈 view plot 🚷 view threshold	15.99 ops/s x 1e3 (+0.00%) Baseline: 15.99 ops/s x 1e3	15.83 ops/s x 1e3 (99.00%)	📈 view plot 🚷 view threshold	42.02 units (+0.97%) Baseline: 41.62 units	42.03 units (99.97%)

🐰 View full continuous benchmarking report in Bencher

github-actions · 2026-10-16T16:36:17Z

Bencher Report

Project	PgDog
Branch	murex-having-support
Testbed	Intel v1

⚠️ WARNING: Truncated view!
The full continuous benchmarking report exceeds the maximum length allowed on this platform.

🐰 View full continuous benchmarking report in Bencher

github-actions · 2026-10-16T16:39:37Z

Bencher Report

Project	PgDog
Branch	murex-having-support
Testbed	Intel v1

🚨 1 Alert

Benchmark	Measure Units	View	Benchmark Result (Result Δ%)	Upper Boundary (Limit %)
pooler-v3	cpu-per-txn Measure (units)	📈 plot 🚷 threshold 🚨 alert (🔔)	40.95 units (+1.50%) Baseline: 40.35 units	40.75 units (100.49%)

Click to view all benchmark results

Benchmark	Throughput	Benchmark Result operations / second (ops/s) x 1e3 (Result Δ%)	Lower Boundary operations / second (ops/s) x 1e3 (Limit %)	cpu-per-txn	Benchmark Result Measure (units) (Result Δ%)	Upper Boundary Measure (units) (Limit %)
pooler-v3	📈 view plot 🚷 view threshold	16.99 ops/s x 1e3 (-0.00%) Baseline: 16.99 ops/s x 1e3	16.82 ops/s x 1e3 (99.00%)	📈 view plot 🚷 view threshold 🚨 view alert (🔔)	40.95 units (+1.50%) Baseline: 40.35 units	40.75 units (100.49%)

🐰 View full continuous benchmarking report in Bencher

jkaczman · 2026-10-16T16:40:18Z

Working on adjusting Bencher, please ignore those alerts for now

jkaczman

I also recommend adding HAVING to

pgdog/pgdog/src/frontend/client/query_engine/test/unsupported_aggregation.rs

Lines 99 to 103 in eda06e8

    
                   }) 
        
                   .await 
        
                   .unwrap_or_else(|_| panic!("unsupported aggregation or subsequent query hung: {begin:?}")); 
        
               } 
        
           }

(in ref. to #1802 (fix for #1795)), this code seems to add a case where it could propagate UnsupportedHaving off of

pgdog/pgdog/src/frontend/client/query_engine/route_query.rs

Lines 156 to 164 in eda06e8

    
           projection::finalize_after_route( 
        
               client_request, 
        
               &cluster.schema(), 
        
               rewrite_result.and_then(RewriteResult::offset_plan), 
        
           )?; 
        
           if let Some(rewrite_result) = rewrite_result { 
        
               rewrite_result.apply_after_route(client_request)?;

and disconnect the client

murex971 · 2026-10-19T19:01:05Z

@jkaczman addressed

github-actions · 2026-10-21T15:06:25Z

Bencher Report

Project	PgDog
Branch	murex-having-support
Testbed	Intel v1

Click to view all benchmark results

Benchmark	Throughput	Benchmark Result operations / second (ops/s) x 1e3 (Result Δ%)	Lower Boundary operations / second (ops/s) x 1e3 (Limit %)	cpu-per-txn	Benchmark Result Measure (units) (Result Δ%)	Upper Boundary Measure (units) (Limit %)	ctx-switches-per-txn	Benchmark Result Measure (units) (Result Δ%)	Upper Boundary Measure (units) (Limit %)
sharding-w2-v3	📈 view plot 🚷 view threshold	10.00 ops/s x 1e3 (+0.00%) Baseline: 10.00 ops/s x 1e3	9.90 ops/s x 1e3 (99.00%)	📈 view plot 🚷 view threshold	62.59 units (-0.07%) Baseline: 62.63 units	63.26 units (98.94%)	📈 view plot 🚷 view threshold	1.27 units (+0.71%) Baseline: 1.26 units	1.30 units (97.77%)

🐰 View full continuous benchmarking report in Bencher

github-actions · 2026-10-22T21:53:43Z

Metal bench failed. 3 of 150 measures outside bounds.

suite	throughput	cpu/txn	user cycles/txn	user instr/txn	kernel cycles/txn	ctx/txn
pooler		27.78 (-1.5% ± 0.8%)	27,006 (-1.9% ± 1.0%)	35,060 (+0.0% ± 0.02%)	38,726 (-0.8% ± 1.3%)
lb		28.8 (-1.3% ± 1.0%)	28,436 (-1.4% ± 1.4%)	35,919 (+0.0% ± 0.02%)	39,750 (-0.8% ± 1.4%)
sharding-w2		52.49 (+3.1%), bound +3%	65,970 (+3.1%), bound +3%	72,282 (+0.0%)	61,054 (+3.3%)	1.419 (-0.6%)
sharding-w8	156,928 (-2.4%)	46.46 (+2.7%)	66,663 (+3.8%), bound +3%	56,105 (+0.0%)	53,008 (+1.6%)	0.1495 (-2.1%)

sharding-w2: user cycles +3.1%, instructions flat. Unexplained.
sharding-w8: user cycles +3.8%, instructions flat. Unexplained.

Counters per txn

suite	frontend stall	l1i refill	backend stall
pooler	13,026 (-3.6% ± 1.9%)	3,103 (-1.0% ± 0.7%)	2,635 (-0.1% ± 2.2%)
lb	13,861 (-2.7% ± 2.9%)	3,185 (-0.8% ± 1.2%)	2,918 (+0.9% ± 2.2%)
sharding-w2	37,174 (+6.0%)	5,243 (+1.6%)	7,133 (-1.6%)
sharding-w8	37,663 (+8.9%)	4,972 (+2.5%)	9,466 (-6.6%)

Microbench: all 135 measures within bounds.

lb flamegraph vs 692b9d9

Open interactive for hover, zoom and search.

Open interactive.

microbench flamegraph vs 692b9d9

Open interactive for hover, zoom and search.

Open interactive.

pooler flamegraph vs 692b9d9

Open interactive for hover, zoom and search.

Open interactive.

sharding-w2 flamegraph vs 692b9d9

Open interactive for hover, zoom and search.

Open interactive.

sharding-w8 flamegraph vs 692b9d9

Open interactive for hover, zoom and search.

Open interactive.

Logs

lb, microbench, pooler, sharding-w2, sharding-w8, verdict.json

Compared d9348d5 with 692b9d9.

jkaczman · 2026-10-27T03:59:02Z

What are your thoughts on splitting this into multiple stacked PRs? I think it'd help address some of the things I mentioned above as well as make it less likely for us to miss something.

murex971 · 2026-10-27T08:33:06Z

What are your thoughts on splitting this into multiple stacked PRs? I think it'd help address some of the things I mentioned above as well as make it less likely for us to miss something.

I am not sure if we can split the having feature itself into multiple PRs, but I can definitely separate out some refactoring that was done in this PR around ProjectRewrite

…urex-having-support

jkaczman · 2026-10-27T14:56:13Z

Hmm.. how about this? Ideally each of these is less than 500 LOC.

refactor: shared projection rewrite plan as Option #1902
Reject cross-shard HAVING with an explicit error
Main infra: support one condition per HAVING, e.g., HAVING count(*) > 3 or HAVING count(*) > $1
Extends on 3 by handling AND/OR/NOT and NULLs, e.g., HAVING count(*) > 3 AND max(shipped_at) IS NULL
Extends on 4 by handling decimals and floats, e.g., HAVING sum(price) > 99.99 or HAVING count(*)::numeric > 3

…-support

jkaczman

I suggest splitting this up again due to it being over >1000 LOC, perhaps like this, although feel free to devise your own split if you have a better idea!

Execution (first)

apply HAVING after merging shards' results
don't wire in yet, continue to error from the prev stacked PR (re #1906)
code:
- parser/having.rs
- buffer/having.rs
- Buffer::having() in buffer.rs
- AggregateError variants in aggregate/error.rs
- is_unsupported() in backend/error.rs
- ProjectionRewritePlan.having in projection.rs
explicit unit tests to exercise all functionality via hand-built HAVING plan (extended upon with integration tests in planning)

Planning (second)

rewrite the query sent to the shards
wire in Buffer::having(), remove error from prev stacked PR
code:
- rewrite/statement/having.rs
- build() and finalize_after_route() in projection.rs
- Buffer::having() call in multi_shard/mod.rs
- Route::set_aggregate() in route.rs
- AggregateFunction::from_builtin_name() in parser/aggregate.rs
rewriter, query engine and integration tests

murex971 mentioned this pull request Oct 16, 2026

feat: add support for having clause #1165

Closed

murex971 changed the title ~~feat: add support for Having clause~~ feat: add cross-shard HAVING support for grouped aggregates Oct 16, 2026

jkaczman force-pushed the murex-having-support branch from 0242c2a to d39bcef Compare October 16, 2026 16:48

jkaczman reviewed Oct 16, 2026

View reviewed changes

Comment thread pgdog/src/frontend/router/parser/rewrite/statement/projection.rs Outdated

jkaczman self-requested a review October 17, 2026 00:02

jkaczman requested changes Oct 17, 2026

View reviewed changes

jkaczman reviewed Oct 17, 2026

View reviewed changes

PranavV205 mentioned this pull request Oct 17, 2026

Cross-shard GROUP BY merges rows on selected columns only (distinct groups summed); ORDER BY sum(...) without alias ignored #1821

Open

murex971 requested a review from jkaczman October 19, 2026 18:57

jkaczman force-pushed the murex-having-support branch from eaf189d to b0566fb Compare October 21, 2026 13:45

murex971 added 8 commits October 22, 2026 14:24

init

5363d1e

improvements

98291d0

use arc for having rewrite plan

718ff44

store rewritten aggreate in post route plan

9b68185

nits

f24c437

tests

467946d

use option

611347a

redo argh

7f92f25

jkaczman force-pushed the murex-having-support branch 2 times, most recently from 4ca2dcc to 7f92f25 Compare October 22, 2026 21:25

wrap option

9c6fea4

murex971 changed the base branch from main to project-rewrite-refactor October 27, 2026 08:47

Merge remote-tracking branch 'origin/project-rewrite-refactor' into m…

dbc9596

…urex-having-support

murex971 added this pull request to stack #1903 October 27, 2026 08:57

murex971 added 4 commits October 27, 2026 20:45

arc clone

9ac875e

remove agg

773bb04

fix: reject cross-shard HAVING with an explicit error

516b41c

Merge branch 'having-reject' into murex-having-support

9c658b3

murex971 removed this pull request from stack #1903 October 27, 2026 15:50

murex971 changed the base branch from project-rewrite-refactor to having-reject October 27, 2026 15:51

murex971 added this pull request to stack #1907 October 27, 2026 15:52

murex971 added 4 commits October 27, 2026 21:32

cleanup

f442f6e

Merge remote-tracking branch 'origin/main' into having-reject

5af57ae

Merge remote-tracking branch 'origin/having-reject' into murex-having…

05ae025

…-support

trim

d9348d5

Base automatically changed from having-reject to main October 27, 2026 19:26

Merge remote-tracking branch 'origin/main' into murex-having-support

ea091cc

jkaczman reviewed Oct 27, 2026

View reviewed changes

	})
	.await
	.unwrap_or_else(\|_\| panic!("unsupported aggregation or subsequent query hung: {begin:?}"));
	}
	}


	projection::finalize_after_route(
	client_request,
	&cluster.schema(),
	rewrite_result.and_then(RewriteResult::offset_plan),
	)?;

	if let Some(rewrite_result) = rewrite_result {
	rewrite_result.apply_after_route(client_request)?;

Conversation

murex971 commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bencher Report

Uh oh!

github-actions Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bencher Report

⚠️ WARNING: Truncated view!

Uh oh!

github-actions Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bencher Report

🚨 1 Alert

Uh oh!

jkaczman commented Oct 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jkaczman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

murex971 commented Oct 19, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bencher Report

Uh oh!

github-actions Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jkaczman commented Oct 27, 2026

Uh oh!

murex971 commented Oct 27, 2026

Uh oh!

jkaczman commented Oct 27, 2026

Uh oh!

jkaczman left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

murex971 commented Oct 16, 2026 •

edited

Loading

codecov Bot commented Oct 16, 2026 •

edited

Loading

github-actions Bot commented Oct 16, 2026 •

edited

Loading

github-actions Bot commented Oct 16, 2026 •

edited

Loading

github-actions Bot commented Oct 16, 2026 •

edited

Loading

github-actions Bot commented Oct 21, 2026 •

edited

Loading

github-actions Bot commented Oct 22, 2026 •

edited

Loading