Skip to content

[refine](exec) replace std::shared_mutex/std::shared_lock with annotated wrappers for thread safety analysis#63109

Open
Mryange wants to merge 1 commit intoapache:masterfrom
Mryange:AnnotatedSharedMutex
Open

[refine](exec) replace std::shared_mutex/std::shared_lock with annotated wrappers for thread safety analysis#63109
Mryange wants to merge 1 commit intoapache:masterfrom
Mryange:AnnotatedSharedMutex

Conversation

@Mryange
Copy link
Copy Markdown
Contributor

@Mryange Mryange commented May 9, 2026

What problem does this PR solve?

Issue Number: N/A

Problem Summary:

After #63070 replaced std::mutex/std::lock_guard with annotated wrappers (AnnotatedMutex/LockGuard), shared mutex usage still relied on raw std::shared_mutex and std::shared_lock/std::unique_lock, which are invisible to Clang's thread safety analysis. This leaves shared-lock sites unverified — GUARDED_BY, REQUIRES_SHARED, and other annotations cannot be enforced.

  • Add AnnotatedSharedMutex (wrapping std::shared_mutex with CAPABILITY/ACQUIRE/RELEASE/ACQUIRE_SHARED/RELEASE_SHARED annotations) and SharedLockGuard (RAII SCOPED_CAPABILITY with ACQUIRE_SHARED/RELEASE) in thread_safety_annotations.h.
  • Migrate VDataStreamMgr and RuntimeFilterMergeControllerEntity from std::shared_mutex to AnnotatedSharedMutex, and from std::unique_lock/std::shared_lock to LockGuard/SharedLockGuard.
  • Add GUARDED_BY annotations to the protected maps.
  • Extract _find_recvr as a private helper annotated with REQUIRES_SHARED(_lock), eliminating the bool acquire_lock parameter that previously bypassed lock tracking.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented May 9, 2026

/review

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated code review summary:\n\nNo blocking issues found in the actual PR diff. The change is focused on adding annotated shared-mutex/shared-lock wrappers and applying them to VDataStreamMgr and RuntimeFilterMergeControllerEntity without changing the effective locking order or heavy-work placement.\n\nCritical checkpoints:\n- Goal/test: The PR goal appears to be enabling Clang thread-safety analysis for shared-mutex users; the code accomplishes this for the touched classes. No test changes are included, but this is a static-analysis annotation/refactor with no intended runtime behavior change.\n- Scope: The modification is small and focused.\n- Concurrency: Existing shared/exclusive lock protection is preserved. VDataStreamMgr keeps receiver-map/set operations under the same mutex, and cancel still copies receivers before doing heavier cancellation. RuntimeFilterMergeControllerEntity preserves the map mutex plus per-filter mutex pattern.\n- Lifecycle/static initialization: No new lifecycle-sensitive globals or static initialization dependencies were introduced.\n- Configuration/compatibility/persistence/data writes: Not applicable.\n- Parallel paths: The two touched shared-mutex users are updated consistently in their declarations and call sites.\n- Tests/results: No runtime test result changes. A compile/static-analysis run in CI should validate the annotation intent.\n- Observability/performance: No new logs or metrics needed; runtime lock behavior remains equivalent to std::shared_mutex/std::shared_lock.\n- User focus: No additional user-provided review focus was specified.

@Mryange Mryange changed the title refine](exec) replace std::shared_mutex/std::shared_lock with annotated wrappers for thread safety analysis [refine](exec) replace std::shared_mutex/std::shared_lock with annotated wrappers for thread safety analysis May 9, 2026
@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented May 9, 2026

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 29710 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit de25d0859bda847da02dff3760136983d63fd7e6, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17650	3965	3885	3885
q2	q3	10700	873	610	610
q4	4661	460	342	342
q5	7455	1343	1158	1158
q6	189	167	139	139
q7	901	949	760	760
q8	9308	1407	1289	1289
q9	5603	5423	5377	5377
q10	6259	2084	1824	1824
q11	490	273	257	257
q12	622	412	290	290
q13	18100	3366	2747	2747
q14	288	287	263	263
q15	q16	864	874	792	792
q17	979	993	681	681
q18	6451	5779	5631	5631
q19	1176	1271	1073	1073
q20	524	393	260	260
q21	4961	2447	1997	1997
q22	471	424	335	335
Total cold run time: 97652 ms
Total hot run time: 29710 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4770	4755	4826	4755
q2	q3	4758	4819	4199	4199
q4	2141	2177	1394	1394
q5	5018	5045	5324	5045
q6	200	184	144	144
q7	2057	1790	1645	1645
q8	3392	3145	3120	3120
q9	8449	8500	8427	8427
q10	4524	4521	4252	4252
q11	605	436	403	403
q12	733	786	550	550
q13	3342	3655	2953	2953
q14	315	307	272	272
q15	q16	771	921	692	692
q17	1352	1292	1242	1242
q18	8031	7134	7172	7134
q19	1151	1183	1150	1150
q20	2221	2232	1953	1953
q21	6213	5405	4861	4861
q22	548	495	416	416
Total cold run time: 60591 ms
Total hot run time: 54607 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 170673 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de25d0859bda847da02dff3760136983d63fd7e6, data reload: false

query5	4298	657	527	527
query6	331	218	199	199
query7	4219	592	301	301
query8	325	243	229	229
query9	8826	3996	4031	3996
query10	451	340	291	291
query11	5768	2421	2197	2197
query12	181	129	123	123
query13	1264	595	415	415
query14	5947	5360	5024	5024
query14_1	4318	4346	4306	4306
query15	209	207	182	182
query16	1023	478	417	417
query17	1006	741	627	627
query18	2460	479	367	367
query19	227	206	177	177
query20	143	136	136	136
query21	214	139	123	123
query22	13584	13905	14578	13905
query23	17351	16561	16268	16268
query23_1	16502	16303	16277	16277
query24	7518	1754	1364	1364
query24_1	1372	1345	1357	1345
query25	606	525	453	453
query26	1301	315	170	170
query27	2736	618	338	338
query28	4462	1996	1935	1935
query29	1008	665	543	543
query30	302	239	202	202
query31	1104	1068	950	950
query32	92	74	72	72
query33	531	363	309	309
query34	1180	1157	671	671
query35	774	785	680	680
query36	1336	1363	1197	1197
query37	154	101	90	90
query38	3191	3114	2983	2983
query39	919	969	906	906
query39_1	893	878	871	871
query40	231	156	134	134
query41	65	60	60	60
query42	110	107	103	103
query43	319	326	283	283
query44	
query45	209	201	192	192
query46	1067	1209	715	715
query47	2371	2348	2176	2176
query48	406	416	305	305
query49	651	529	428	428
query50	690	285	221	221
query51	4322	4235	4247	4235
query52	104	106	93	93
query53	240	282	206	206
query54	314	280	260	260
query55	89	90	85	85
query56	299	293	304	293
query57	1435	1396	1305	1305
query58	297	271	262	262
query59	1631	1642	1453	1453
query60	344	341	331	331
query61	160	159	161	159
query62	673	622	563	563
query63	237	207	204	204
query64	2531	830	691	691
query65	
query66	1768	503	394	394
query67	30052	30056	29799	29799
query68	
query69	460	340	305	305
query70	1034	995	980	980
query71	307	292	275	275
query72	2986	2764	2476	2476
query73	860	793	416	416
query74	5078	4890	4708	4708
query75	2801	2653	2330	2330
query76	2298	1112	725	725
query77	410	409	333	333
query78	12904	12937	12223	12223
query79	1507	1029	736	736
query80	1021	583	489	489
query81	489	279	238	238
query82	1329	162	126	126
query83	351	281	257	257
query84	304	139	114	114
query85	934	535	449	449
query86	436	365	314	314
query87	3414	3346	3206	3206
query88	3546	2716	2635	2635
query89	439	384	338	338
query90	1807	188	186	186
query91	180	173	139	139
query92	85	73	71	71
query93	950	947	555	555
query94	634	369	317	317
query95	661	463	364	364
query96	1024	767	358	358
query97	2666	2684	2593	2593
query98	233	229	230	229
query99	1134	1119	995	995
Total cold run time: 254467 ms
Total hot run time: 170673 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants