[feat](cloud) Expose approximate rowset metric by jayakasadev · Pull Request #67502 · apache/doris

jayakasadev · 2026-09-03T14:58:29Z

What problem does this PR solve?

Issue Number: close #67458

Related PR: N/A

Problem Summary:

writes are rejected with TOO_MANY_VERSION errors when a tablet has too many versions. The code checks _approximate_num_rowsets, but its not emitted as a metric.
tablet_version_num_distribution is similar, but it updates less frequently and skips tablets that are currently being written to
added tablet_approximate_num_rowsets_distribution which uses _approximate_num_rowsets and includes active tablets
- allows users to create alerts before getting TOO_MANY_VERSION errors

Release note

Add tablet_approximate_num_rowsets_distribution for cloud-mode tablet version-limit monitoring

Check List (For Author)

Test
- Unit Test
  - Added CloudTabletMgrTest.TestApproximateRowsetsMetricIncludesActiveTablets.
- Manual test
  - build-support/clang-format.sh
  - build-support/check-format.sh
  - build-support/check-build-hygiene.sh
  - git diff --check
Behavior changed:
- Yes.
  - Cloud BEs expose the new tablet_approximate_num_rowsets_distribution Prometheus histogram.
Does this need documentation?
- No.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

### What problem does this PR solve? Issue Number: close apache#67458 Related PR: N/A Problem Summary: Cloud write admission checks each tablet's _approximate_num_rowsets, but Prometheus exposed only the materialized version-count distribution and omitted recently accessed tablets. Register a separate histogram from the exact admission counter before the inactive-tablet filter so active tablets are included. A focused unit test verifies an active tablet is excluded from the existing report yet included in the new metric. ### Release note Add tablet_approximate_num_rowsets_distribution for cloud-mode tablet version-limit monitoring. ### Check List (For Author) - Test: Header hygiene and formatting checks passed. Focused BE unit test was attempted but local configuration cannot proceed without third-party artifacts; Docker build image extraction also fails on this host. - Behavior changed: Yes. Cloud BEs expose a new Prometheus histogram. - Does this need documentation: No.

hello-stephen · 2026-09-03T14:58:38Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

jayakasadev · 2026-09-03T14:59:53Z

run buildall

Set a nonzero inactive-report window in the approximate-rowsets metric test so the test tablet takes the active-tablet path. Restore the global after the assertion to avoid changing later tests.

jayakasadev · 2026-09-03T18:40:21Z

run buildall

hello-stephen · 2026-09-04T00:52:58Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.33% (34666/45415)
Line Coverage	61.37% (391109/637341)
Region Coverage	57.46% (328310/571408)
Branch Coverage	58.36% (149882/256814)

jayakasadev · 2026-09-04T15:07:11Z

run buildall

hello-stephen · 2026-09-04T16:33:18Z

TPC-H: Total hot run time: 17068 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 71830b09e71993737350e6dda31e021b1e7b1068, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17614	3113	3119	3113
q2	2077	247	223	223
q3	10264	900	518	518
q4	4668	251	210	210
q5	7664	576	384	384
q6	137	117	96	96
q7	531	506	389	389
q8	9233	915	985	915
q9	3427	2407	2399	2399
q10	6546	845	720	720
q11	407	212	181	181
q12	617	260	205	205
q13	18103	1533	1165	1165
q14	159	151	140	140
q15	q16	442	407	373	373
q17	1393	904	803	803
q18	3209	2319	2255	2255
q19	1289	885	833	833
q20	378	284	205	205
q21	5597	1714	1930	1714
q22	337	268	227	227
Total cold run time: 94092 ms
Total hot run time: 17068 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3491	3432	3457	3432
q2	511	401	389	389
q3	2226	2626	2205	2205
q4	1191	1172	906	906
q5	2196	2115	2137	2115
q6	165	126	85	85
q7	1039	926	856	856
q8	1626	1436	1436	1436
q9	3161	3118	3123	3118
q10	1864	1812	1625	1625
q11	370	278	251	251
q12	458	425	362	362
q13	1495	1527	1161	1161
q14	182	166	166	166
q15	q16	395	401	358	358
q17	3618	3234	3280	3234
q18	4836	4468	4958	4468
q19	930	859	846	846
q20	1004	965	814	814
q21	3807	3068	3192	3068
q22	398	346	325	325
Total cold run time: 34963 ms
Total hot run time: 31220 ms

hello-stephen · 2026-09-04T16:39:47Z

TPC-DS: Total hot run time: 82639 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 71830b09e71993737350e6dda31e021b1e7b1068, data reload: false

query5	4265	410	336	336
query6	376	139	123	123
query7	4956	430	238	238
query8	287	126	118	118
query9	8666	2909	2894	2894
query10	411	222	189	189
query11	5363	1077	908	908
query12	109	70	72	70
query13	1186	443	327	327
query14	6130	2222	2118	2118
query14_1	1991	1989	1990	1989
query15	174	130	115	115
query16	924	383	347	347
query17	802	465	389	389
query18	2329	337	244	244
query19	170	142	110	110
query20	76	74	74	74
query21	200	102	88	88
query22	5550	5372	5327	5327
query23	6645	6287	6109	6109
query23_1	6080	6097	6055	6055
query24	7326	1092	768	768
query24_1	782	774	785	774
query25	440	309	261	261
query26	1228	256	130	130
query27	2767	417	266	266
query28	4673	1497	1516	1497
query29	941	438	364	364
query30	258	159	131	131
query31	831	399	333	333
query32	127	74	74	74
query33	464	239	184	184
query34	995	831	492	492
query35	411	411	354	354
query36	577	591	606	591
query37	116	80	73	73
query38	1028	852	830	830
query39	514	486	474	474
query39_1	488	475	454	454
query40	204	93	108	93
query41	53	51	51	51
query42	73	72	70	70
query43	242	243	214	214
query44	1018	552	535	535
query45	115	104	98	98
query46	795	842	558	558
query47	785	778	716	716
query48	295	313	226	226
query49	547	234	187	187
query50	717	263	191	191
query51	8324	8389	8421	8389
query52	66	69	59	59
query53	196	200	150	150
query54	227	166	152	152
query55	77	58	55	55
query56	213	172	167	167
query57	717	673	627	627
query58	183	171	165	165
query59	1241	1247	1152	1152
query60	229	203	172	172
query61	115	118	112	112
query62	353	213	176	176
query63	175	144	141	141
query64	2763	696	585	585
query65	1613	1654	1589	1589
query66	1833	254	203	203
query67	10108	9748	9753	9748
query68	2748	1247	792	792
query69	345	216	181	181
query70	669	641	608	608
query71	256	172	160	160
query72	2302	1732	1164	1164
query73	661	568	341	341
query74	1573	1223	1160	1160
query75	1168	1105	994	994
query76	2277	746	549	549
query77	248	261	205	205
query78	3734	3694	3205	3205
query79	2314	826	605	605
query80	1556	337	279	279
query81	474	155	135	135
query82	617	119	99	99
query83	268	217	192	192
query84	284	116	91	91
query85	792	353	291	291
query86	391	174	174	174
query87	1039	979	900	900
query88	2793	2108	2087	2087
query89	286	195	177	177
query90	1972	129	125	125
query91	132	125	102	102
query92	76	72	69	69
query93	1349	1055	726	726
query94	627	260	214	214
query95	541	320	223	223
query96	823	570	283	283
query97	1065	1033	999	999
query98	149	138	138	138
query99	418	352	316	316
Total cold run time: 177291 ms
Total hot run time: 82639 ms

hello-stephen · 2026-09-04T16:44:10Z

ClickBench: Total hot run time: 14.75 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 71830b09e71993737350e6dda31e021b1e7b1068, data reload: false

query1	0.00	0.01	0.01
query2	0.07	0.04	0.04
query3	0.24	0.11	0.11
query4	1.59	0.09	0.09
query5	0.18	0.17	0.16
query6	1.26	0.68	0.70
query7	0.03	0.01	0.00
query8	0.05	0.03	0.03
query9	0.28	0.22	0.21
query10	0.35	0.36	0.33
query11	0.16	0.13	0.12
query12	0.14	0.12	0.12
query13	0.31	0.31	0.30
query14	0.47	0.45	0.44
query15	0.36	0.35	0.35
query16	0.23	0.22	0.25
query17	0.70	0.72	0.68
query18	0.18	0.16	0.17
query19	1.19	1.20	1.08
query20	0.01	0.00	0.01
query21	15.43	0.16	0.12
query22	5.06	0.05	0.05
query23	16.19	0.26	0.09
query24	3.03	0.33	0.26
query25	0.10	0.04	0.04
query26	0.81	0.16	0.12
query27	0.04	0.02	0.04
query28	3.66	0.56	0.27
query29	12.43	3.16	2.57
query30	0.27	0.12	0.12
query31	2.75	0.37	0.17
query32	3.51	0.33	0.23
query33	1.48	1.54	1.53
query34	15.38	2.22	1.78
query35	1.80	1.74	1.74
query36	0.46	0.29	0.29
query37	0.06	0.04	0.04
query38	0.04	0.03	0.03
query39	0.03	0.03	0.02
query40	0.11	0.07	0.08
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 90.58 s
Total hot run time: 14.75 s

hello-stephen · 2026-09-04T20:52:58Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.30% (34660/45425)
Line Coverage	61.33% (390918/637442)
Region Coverage	57.49% (328504/571459)
Branch Coverage	58.35% (149859/256842)

jayakasadev requested a review from gavinchou as a code owner September 3, 2026 14:58

Merge branch 'master' into jaya/approximate-rowsets-metric

44554e8

jayakasadev changed the title ~~[feature](cloud) Expose approximate rowset metric~~ [feat](cloud) Expose approximate rowset metric Sep 3, 2026

[test](cloud) Set active tablet report window

0d04296

Set a nonzero inactive-report window in the approximate-rowsets metric test so the test tablet takes the active-tablet path. Restore the global after the assertion to avoid changing later tests.

Merge branch 'master' into jaya/approximate-rowsets-metric

71830b0

Navigation Menu

Uh oh!

[feat](cloud) Expose approximate rowset metric - #67502

[feat](cloud) Expose approximate rowset metric#67502
jayakasadev wants to merge 4 commits into
apache:masterfrom
jayakasadev:jaya/approximate-rowsets-metric

jayakasadev commented Sep 3, 2026

Uh oh!

hello-stephen commented Sep 3, 2026

Uh oh!

jayakasadev commented Sep 3, 2026

Uh oh!

jayakasadev commented Sep 3, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

jayakasadev commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

jayakasadev commented Sep 3, 2026

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Sep 3, 2026

Uh oh!

jayakasadev commented Sep 3, 2026

Uh oh!

jayakasadev commented Sep 3, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

BE Regression && UT Coverage Report

Uh oh!

jayakasadev commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

Uh oh!

hello-stephen commented Sep 4, 2026

BE Regression && UT Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants