Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

branch-3.0: [fix](scan) Fix scan with limit #46035 #46090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Dec 31, 2024
Merged

branch-3.0: [fix](scan) Fix scan with limit #46035 #46090

merged 1 commit into from
Dec 31, 2024

Conversation

zhiqiang-hhhh
Copy link
Contributor

cherry pick from #46035

@zhiqiang-hhhh
[fix](scan) Fix scan with limit (apache#46035)
952facf 
Related PR: apache#39927

apache#39927 introduced a new session var
`adaptive_pipeline_task_serial_read_on_limit`, its implementation has
flaw, reduce scanner parallelism should only happen when there is not
conjunction. Since we can not estimate the filtration efficiency.
@zhiqiang-hhhh
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

TPC-H: Total hot run time: 40798 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 952facff5623cedb41caa97517a19782b4d7fee7, data reload: false

------ Round 1 ----------------------------------
q1	17626	7533	7176	7176
q2	2045	192	166	166
q3	10679	1050	1190	1050
q4	10577	715	729	715
q5	7762	2873	2832	2832
q6	235	146	141	141
q7	983	613	595	595
q8	9357	1948	1964	1948
q9	6647	6425	6439	6425
q10	7047	2278	2334	2278
q11	467	254	262	254
q12	407	214	214	214
q13	18033	2972	3047	2972
q14	251	234	225	225
q15	559	522	517	517
q16	719	607	610	607
q17	981	552	589	552
q18	7327	6806	6690	6690
q19	1382	1065	1007	1007
q20	472	206	195	195
q21	4183	3314	3237	3237
q22	1116	1002	1021	1002
Total cold run time: 108855 ms
Total hot run time: 40798 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7255	7164	7183	7164
q2	319	226	227	226
q3	2908	2944	2966	2944
q4	2065	1899	1792	1792
q5	5723	5696	5764	5696
q6	225	137	141	137
q7	2251	1806	1851	1806
q8	3356	3540	3429	3429
q9	8880	8929	8824	8824
q10	3569	3543	3550	3543
q11	604	508	494	494
q12	852	666	619	619
q13	10238	3219	3138	3138
q14	310	277	272	272
q15	582	526	520	520
q16	747	662	669	662
q17	1832	1612	1583	1583
q18	8218	7906	7473	7473
q19	1688	1576	1561	1561
q20	2089	1895	1853	1853
q21	5591	5459	5410	5410
q22	1141	1088	1049	1049
Total cold run time: 70443 ms
Total hot run time: 60195 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197156 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 952facff5623cedb41caa97517a19782b4d7fee7, data reload: false

query1	1239	917	911	911
query2	6219	2125	2084	2084
query3	11003	4361	4118	4118
query4	66612	28808	23395	23395
query5	4961	453	458	453
query6	432	172	175	172
query7	5500	312	317	312
query8	320	237	231	231
query9	8439	2689	2688	2688
query10	441	264	255	255
query11	17092	15190	15803	15190
query12	155	102	101	101
query13	1484	448	445	445
query14	9768	7377	7181	7181
query15	213	182	191	182
query16	7198	509	520	509
query17	1093	572	574	572
query18	1857	327	331	327
query19	205	167	155	155
query20	118	111	114	111
query21	58	45	45	45
query22	5108	4826	4843	4826
query23	34644	34394	34051	34051
query24	6247	2876	2878	2876
query25	502	412	392	392
query26	659	158	160	158
query27	1953	299	312	299
query28	4415	2497	2461	2461
query29	699	446	420	420
query30	246	170	163	163
query31	1005	815	844	815
query32	69	55	56	55
query33	469	275	290	275
query34	926	499	486	486
query35	827	730	740	730
query36	1112	942	977	942
query37	117	72	70	70
query38	4154	4026	4022	4022
query39	1498	1466	1505	1466
query40	135	78	82	78
query41	49	48	46	46
query42	114	101	95	95
query43	545	502	503	502
query44	1179	824	850	824
query45	186	166	168	166
query46	1143	747	722	722
query47	2049	1960	1920	1920
query48	472	381	387	381
query49	721	375	378	375
query50	850	416	416	416
query51	7385	7331	7058	7058
query52	92	83	81	81
query53	249	175	178	175
query54	546	436	430	430
query55	75	78	75	75
query56	267	261	234	234
query57	1277	1099	1079	1079
query58	207	203	213	203
query59	3150	2921	3104	2921
query60	282	261	244	244
query61	108	107	110	107
query62	791	652	655	652
query63	211	180	183	180
query64	1373	667	611	611
query65	3255	3168	3184	3168
query66	702	297	296	296
query67	16001	15675	15624	15624
query68	3635	603	589	589
query69	420	263	270	263
query70	1163	1113	1036	1036
query71	345	245	253	245
query72	6283	3986	3960	3960
query73	745	347	336	336
query74	9985	9020	9028	9020
query75	3362	2654	2631	2631
query76	1815	1023	1071	1023
query77	504	289	284	284
query78	10509	9672	9630	9630
query79	1459	587	605	587
query80	886	461	446	446
query81	521	246	243	243
query82	1263	119	119	119
query83	179	154	144	144
query84	283	87	81	81
query85	892	353	360	353
query86	324	306	302	302
query87	4419	4363	4263	4263
query88	3792	2359	2336	2336
query89	413	286	291	286
query90	1996	185	181	181
query91	178	148	146	146
query92	62	47	45	45
query93	1864	544	535	535
query94	782	294	288	288
query95	352	246	248	246
query96	610	269	278	269
query97	3363	3203	3189	3189
query98	208	202	196	196
query99	1578	1286	1277	1277
Total cold run time: 316661 ms
Total hot run time: 197156 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.87 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 952facff5623cedb41caa97517a19782b4d7fee7, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.53	0.51	0.51
query6	1.14	0.72	0.74
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.51	0.49
query10	0.53	0.54	0.56
query11	0.14	0.10	0.10
query12	0.13	0.11	0.12
query13	0.60	0.59	0.60
query14	3.05	2.96	2.92
query15	0.90	0.82	0.81
query16	0.39	0.40	0.38
query17	1.03	1.03	0.98
query18	0.25	0.22	0.23
query19	1.83	1.87	1.91
query20	0.01	0.01	0.01
query21	15.35	0.57	0.58
query22	2.54	3.13	1.82
query23	17.02	0.99	0.81
query24	2.51	0.96	1.96
query25	0.18	0.08	0.09
query26	0.61	0.15	0.13
query27	0.05	0.03	0.05
query28	10.28	1.09	1.08
query29	12.54	3.22	3.22
query30	0.24	0.06	0.05
query31	2.86	0.39	0.38
query32	3.27	0.45	0.47
query33	2.98	3.00	3.02
query34	17.19	4.50	4.53
query35	4.60	4.49	4.49
query36	0.67	0.48	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.5 s
Total hot run time: 32.87 s

@yiguolei yiguolei merged commit 395b469 into apache:branch-3.0 Dec 31, 2024
20 of 21 checks passed
@zhiqiang-hhhh zhiqiang-hhhh deleted the pick_46035_to_upstream_branch-3.0 branch December 31, 2024 17:40
@gavinchou gavinchou mentioned this pull request Feb 18, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zhiqiang-hhhh @hello-stephen @doris-robot @yiguolei
Morty Proxy This is a proxified and sanitized view of the page, visit original site.