Nikita tkachenko/test env be03ce5b82b7cc8ca9cf2fe3f8af015d39690559 by nikita-tkachenko-datadog · Pull Request #8494 · DataDog/dd-trace-java

nikita-tkachenko-datadog · Mar 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

…ed (#8458)

pr-commenter · Mar 4, 2025

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-env-be03ce5b82b7cc8ca9cf2fe3f8af015d39690559
git_commit_date	1741082792	1740763125
git_commit_sha	`8f945b0`	`be03ce5`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~be03ce5b82

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741091579	1741091579
ci_job_id	831518993	831518993
ci_pipeline_id	57626503	57626503
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner--t2f38de-project-304-concurrent-4-ye7zmoiz 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner--t2f38de-project-304-concurrent-4-ye7zmoiz 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 21 performance improvements and 7 performance regressions! Performance is the same for 29 metrics, 6 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent	better [-758.938ms; -569.381ms] or [-7.651%; -5.740%]	9.255s	9.919s
scenario:startup:insecure-bank:iast:Agent.start	better [-99.279ms; -82.745ms] or [-7.862%; -6.552%]	1.172s	1.263s
scenario:startup:insecure-bank:iast:Remote Config	worse [+82.610µs; +103.073µs] or [+16.264%; +20.293%]	600.765µs	507.924µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent	better [-682.116ms; -595.276ms] or [-6.924%; -6.043%]	9.212s	9.851s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start	better [-93.826ms; -83.922ms] or [-7.446%; -6.660%]	1.171s	1.260s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Remote Config	worse [+79.913µs; +118.662µs] or [+15.637%; +23.219%]	610.346µs	511.058µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent	better [-692.590ms; -579.041ms] or [-7.022%; -5.871%]	9.227s	9.863s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start	better [-86.634ms; -76.903ms] or [-6.914%; -6.137%]	1.171s	1.253s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config	worse [+71.912µs; +118.637µs] or [+13.755%; +22.692%]	618.080µs	522.806µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry	better [-3.719ms; -3.425ms] or [-30.476%; -28.069%]	8.630ms	12.202ms
scenario:startup:insecure-bank:tracing:Agent	better [-707.344ms; -571.334ms] or [-7.609%; -6.146%]	8.657s	9.296s
scenario:startup:insecure-bank:tracing:Agent.start	better [-103.938ms; -90.629ms] or [-9.140%; -7.970%]	1.040s	1.137s
scenario:startup:insecure-bank:tracing:Remote Config	worse [+94.585µs; +146.769µs] or [+16.246%; +25.209%]	702.878µs	582.201µs
scenario:startup:petclinic:appsec:Agent	better [-998.012ms; -795.140ms] or [-8.560%; -6.820%]	10.763s	11.660s
scenario:startup:petclinic:appsec:Agent.start	better [-112.411ms; -82.512ms] or [-8.765%; -6.433%]	1.185s	1.283s
scenario:startup:petclinic:appsec:Telemetry	better [-4.406ms; -3.949ms] or [-35.271%; -31.609%]	8.315ms	12.492ms
scenario:startup:petclinic:iast:Agent	better [-1086.636ms; -906.375ms] or [-9.069%; -7.565%]	10.985s	11.981s
scenario:startup:petclinic:iast:Agent.start	better [-95.891ms; -61.962ms] or [-7.597%; -4.909%]	1.183s	1.262s
scenario:startup:petclinic:iast:Remote Config	worse [+89.944µs; +137.002µs] or [+17.265%; +26.297%]	634.448µs	520.975µs
scenario:startup:petclinic:profiling:Agent	better [-1083.205ms; -805.475ms] or [-9.197%; -6.839%]	10.833s	11.778s
scenario:startup:petclinic:profiling:Agent.start	better [-145.217ms; -107.667ms] or [-10.464%; -7.758%]	1.261s	1.388s
scenario:startup:petclinic:profiling:ProfilingAgent	better [-29.421ms; -25.620ms] or [-23.743%; -20.675%]	96.395ms	123.915ms
scenario:startup:petclinic:profiling:Remote Config	worse [+24.476µs; +56.687µs] or [+3.973%; +9.202%]	656.619µs	616.037µs
scenario:startup:petclinic:profiling:Telemetry	better [-3.988ms; -3.451ms] or [-31.576%; -27.325%]	8.911ms	12.631ms
scenario:startup:petclinic:profiling:Profiling	better [-29.418ms; -25.615ms] or [-23.737%; -20.668%]	96.419ms	123.936ms
scenario:startup:petclinic:tracing:Agent	better [-1.173s; -0.951s] or [-10.208%; -8.276%]	10.432s	11.494s
scenario:startup:petclinic:tracing:Agent.start	better [-102.060ms; -92.034ms] or [-8.989%; -8.106%]	1.038s	1.135s
scenario:startup:petclinic:tracing:Remote Config	worse [+95.696µs; +126.610µs] or [+16.554%; +21.902%]	689.232µs	578.079µs

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.137 s) : 0, 1137128
Total [baseline] (9.296 s) : 0, 9296081
Agent [candidate] (1.04 s) : 0, 1039844
Total [candidate] (8.657 s) : 0, 8656742
section iast
Agent [baseline] (1.263 s) : 0, 1262823
Total [baseline] (9.919 s) : 0, 9919024
Agent [candidate] (1.172 s) : 0, 1171811
Total [candidate] (9.255 s) : 0, 9254864
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.26 s) : 0, 1260127
Total [baseline] (9.851 s) : 0, 9851090
Agent [candidate] (1.171 s) : 0, 1171253
Total [candidate] (9.212 s) : 0, 9212395
section iast_TELEMETRY_OFF
Agent [baseline] (1.253 s) : 0, 1253104
Total [baseline] (9.863 s) : 0, 9862698
Agent [candidate] (1.171 s) : 0, 1171336
Total [candidate] (9.227 s) : 0, 9226882

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.137 s	-
Agent	iast	1.263 s	125.695 ms (11.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.26 s	122.999 ms (10.8%)
Agent	iast_TELEMETRY_OFF	1.253 s	115.976 ms (10.2%)
Total	tracing	9.296 s	-
Total	iast	9.919 s	622.943 ms (6.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.851 s	555.009 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.863 s	566.616 ms (6.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	iast	1.172 s	131.967 ms (12.7%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.171 s	131.409 ms (12.6%)
Agent	iast_TELEMETRY_OFF	1.171 s	131.491 ms (12.6%)
Total	tracing	8.657 s	-
Total	iast	9.255 s	598.121 ms (6.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.212 s	555.652 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.227 s	570.139 ms (6.6%)

gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.54 ms) : 0, 717540
BytebuddyAgent [candidate] (717.72 ms) : 0, 717720
GlobalTracer [baseline] (240.835 ms) : 0, 240835
GlobalTracer [candidate] (239.346 ms) : 0, 239346
AppSec [baseline] (55.885 ms) : 0, 55885
AppSec [candidate] (55.714 ms) : 0, 55714
Remote Config [baseline] (582.201 µs) : 0, 582
Remote Config [candidate] (702.878 µs) : 0, 703
Telemetry [baseline] (12.334 ms) : 0, 12334
Telemetry [candidate] (11.498 ms) : 0, 11498
Debugger [baseline] (94.944 ms) : 0, 94944
section iast
BytebuddyAgent [baseline] (837.997 ms) : 0, 837997
BytebuddyAgent [candidate] (837.459 ms) : 0, 837459
GlobalTracer [baseline] (231.759 ms) : 0, 231759
GlobalTracer [candidate] (230.026 ms) : 0, 230026
AppSec [baseline] (57.168 ms) : 0, 57168
AppSec [candidate] (57.344 ms) : 0, 57344
Remote Config [baseline] (507.924 µs) : 0, 508
Remote Config [candidate] (600.765 µs) : 0, 601
Telemetry [baseline] (14.209 ms) : 0, 14209
Telemetry [candidate] (8.693 ms) : 0, 8693
Debugger [baseline] (83.241 ms) : 0, 83241
IAST [baseline] (22.986 ms) : 0, 22986
IAST [candidate] (22.791 ms) : 0, 22791
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (837.333 ms) : 0, 837333
BytebuddyAgent [candidate] (836.661 ms) : 0, 836661
GlobalTracer [baseline] (231.048 ms) : 0, 231048
GlobalTracer [candidate] (230.253 ms) : 0, 230253
AppSec [baseline] (56.956 ms) : 0, 56956
AppSec [candidate] (57.163 ms) : 0, 57163
Remote Config [baseline] (511.058 µs) : 0, 511
Remote Config [candidate] (610.346 µs) : 0, 610
Telemetry [baseline] (12.911 ms) : 0, 12911
Telemetry [candidate] (8.732 ms) : 0, 8732
Debugger [baseline] (83.608 ms) : 0, 83608
IAST [baseline] (22.733 ms) : 0, 22733
IAST [candidate] (22.863 ms) : 0, 22863
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (832.858 ms) : 0, 832858
BytebuddyAgent [candidate] (836.724 ms) : 0, 836724
GlobalTracer [baseline] (230.14 ms) : 0, 230140
GlobalTracer [candidate] (230.682 ms) : 0, 230682
AppSec [baseline] (57.049 ms) : 0, 57049
AppSec [candidate] (53.306 ms) : 0, 53306
Remote Config [baseline] (522.806 µs) : 0, 523
Remote Config [candidate] (618.08 µs) : 0, 618
Telemetry [baseline] (12.202 ms) : 0, 12202
Telemetry [candidate] (8.63 ms) : 0, 8630
Debugger [baseline] (82.911 ms) : 0, 82911
IAST [baseline] (22.458 ms) : 0, 22458
IAST [candidate] (26.372 ms) : 0, 26372

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.135 s) : 0, 1135381
Total [baseline] (11.494 s) : 0, 11494001
Agent [candidate] (1.038 s) : 0, 1038335
Total [candidate] (10.432 s) : 0, 10431726
section appsec
Agent [baseline] (1.283 s) : 0, 1282566
Total [baseline] (11.66 s) : 0, 11659617
Agent [candidate] (1.185 s) : 0, 1185105
Total [candidate] (10.763 s) : 0, 10763042
section iast
Agent [baseline] (1.262 s) : 0, 1262196
Total [baseline] (11.981 s) : 0, 11981455
Agent [candidate] (1.183 s) : 0, 1183270
Total [candidate] (10.985 s) : 0, 10984950
section profiling
Agent [baseline] (1.388 s) : 0, 1387743
Total [baseline] (11.778 s) : 0, 11777754
Agent [candidate] (1.261 s) : 0, 1261301
Total [candidate] (10.833 s) : 0, 10833414

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.135 s	-
Agent	appsec	1.283 s	147.185 ms (13.0%)
Agent	iast	1.262 s	126.815 ms (11.2%)
Agent	profiling	1.388 s	252.361 ms (22.2%)
Total	tracing	11.494 s	-
Total	appsec	11.66 s	165.617 ms (1.4%)
Total	iast	11.981 s	487.454 ms (4.2%)
Total	profiling	11.778 s	283.753 ms (2.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.038 s	-
Agent	appsec	1.185 s	146.77 ms (14.1%)
Agent	iast	1.183 s	144.935 ms (14.0%)
Agent	profiling	1.261 s	222.966 ms (21.5%)
Total	tracing	10.432 s	-
Total	appsec	10.763 s	331.316 ms (3.2%)
Total	iast	10.985 s	553.224 ms (5.3%)
Total	profiling	10.833 s	401.688 ms (3.9%)

gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.845 ms) : 0, 716845
BytebuddyAgent [candidate] (716.109 ms) : 0, 716109
GlobalTracer [baseline] (239.946 ms) : 0, 239946
GlobalTracer [candidate] (238.572 ms) : 0, 238572
AppSec [baseline] (55.84 ms) : 0, 55840
AppSec [candidate] (55.843 ms) : 0, 55843
Remote Config [baseline] (578.079 µs) : 0, 578
Remote Config [candidate] (689.232 µs) : 0, 689
Telemetry [baseline] (12.344 ms) : 0, 12344
Telemetry [candidate] (12.199 ms) : 0, 12199
Debugger [baseline] (94.829 ms) : 0, 94829
section appsec
BytebuddyAgent [baseline] (741.104 ms) : 0, 741104
BytebuddyAgent [candidate] (736.403 ms) : 0, 736403
GlobalTracer [baseline] (239.143 ms) : 0, 239143
GlobalTracer [candidate] (236.264 ms) : 0, 236264
AppSec [baseline] (178.004 ms) : 0, 178004
AppSec [candidate] (177.676 ms) : 0, 177676
Remote Config [baseline] (633.743 µs) : 0, 634
Remote Config [candidate] (665.387 µs) : 0, 665
Telemetry [baseline] (12.492 ms) : 0, 12492
Telemetry [candidate] (8.315 ms) : 0, 8315
Debugger [baseline] (85.123 ms) : 0, 85123
IAST [baseline] (21.989 ms) : 0, 21989
IAST [candidate] (21.595 ms) : 0, 21595
section iast
BytebuddyAgent [baseline] (837.334 ms) : 0, 837334
BytebuddyAgent [candidate] (845.468 ms) : 0, 845468
GlobalTracer [baseline] (231.056 ms) : 0, 231056
GlobalTracer [candidate] (231.646 ms) : 0, 231646
AppSec [baseline] (56.998 ms) : 0, 56998
AppSec [candidate] (58.228 ms) : 0, 58228
Remote Config [baseline] (520.975 µs) : 0, 521
Remote Config [candidate] (634.448 µs) : 0, 634
Telemetry [baseline] (15.087 ms) : 0, 15087
Telemetry [candidate] (8.978 ms) : 0, 8978
Debugger [baseline] (83.381 ms) : 0, 83381
IAST [baseline] (22.837 ms) : 0, 22837
IAST [candidate] (23.264 ms) : 0, 23264
section profiling
ProfilingAgent [baseline] (123.915 ms) : 0, 123915
ProfilingAgent [candidate] (96.395 ms) : 0, 96395
BytebuddyAgent [baseline] (714.09 ms) : 0, 714090
BytebuddyAgent [candidate] (710.573 ms) : 0, 710573
GlobalTracer [baseline] (351.878 ms) : 0, 351878
GlobalTracer [candidate] (349.752 ms) : 0, 349752
AppSec [baseline] (55.37 ms) : 0, 55370
AppSec [candidate] (54.655 ms) : 0, 54655
Remote Config [baseline] (616.037 µs) : 0, 616
Remote Config [candidate] (656.619 µs) : 0, 657
Telemetry [baseline] (12.631 ms) : 0, 12631
Telemetry [candidate] (8.911 ms) : 0, 8911
Profiling [baseline] (123.936 ms) : 0, 123936
Profiling [candidate] (96.419 ms) : 0, 96419
Debugger [baseline] (88.533 ms) : 0, 88533

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-04T12:04:26	2025-03-04T12:12:18
git_branch	master	nikita-tkachenko/test-env-be03ce5b82b7cc8ca9cf2fe3f8af015d39690559
git_commit_date	1741082792	1740763125
git_commit_sha	`8f945b0`	`be03ce5`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~be03ce5b82
start_time	2025-03-04T12:04:12	2025-03-04T12:12:04

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741090735	1741090735
ci_job_id	831518994	831518994
ci_pipeline_id	57626503	57626503
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8ksbllmc-project-304-concurrent-2-huvdry9y 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8ksbllmc-project-304-concurrent-2-huvdry9y 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1332, 1372
.   : milestone, 1352,
appsec (1.736 ms) : 1712, 1760
.   : milestone, 1736,
appsec_no_iast (1.711 ms) : 1687, 1734
.   : milestone, 1711,
code_origins (1.685 ms) : 1653, 1717
.   : milestone, 1685,
iast (1.511 ms) : 1485, 1536
.   : milestone, 1511,
profiling (1.516 ms) : 1491, 1542
.   : milestone, 1516,
tracing (1.498 ms) : 1473, 1523
.   : milestone, 1498,
section candidate
no_agent (1.367 ms) : 1348, 1386
.   : milestone, 1367,
appsec (1.723 ms) : 1700, 1747
.   : milestone, 1723,
appsec_no_iast (1.738 ms) : 1714, 1762
.   : milestone, 1738,
code_origins (1.691 ms) : 1658, 1723
.   : milestone, 1691,
iast (1.505 ms) : 1481, 1529
.   : milestone, 1505,
profiling (1.516 ms) : 1492, 1540
.   : milestone, 1516,
tracing (1.505 ms) : 1481, 1530
.   : milestone, 1505,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.352 ms [1.332 ms, 1.372 ms]	-
appsec	1.736 ms [1.712 ms, 1.76 ms]	384.459 µs (28.4%)
appsec_no_iast	1.711 ms [1.687 ms, 1.734 ms]	358.688 µs (26.5%)
code_origins	1.685 ms [1.653 ms, 1.717 ms]	332.846 µs (24.6%)
iast	1.511 ms [1.485 ms, 1.536 ms]	158.773 µs (11.7%)
profiling	1.516 ms [1.491 ms, 1.542 ms]	164.415 µs (12.2%)
tracing	1.498 ms [1.473 ms, 1.523 ms]	145.993 µs (10.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.367 ms [1.348 ms, 1.386 ms]	-
appsec	1.723 ms [1.7 ms, 1.747 ms]	356.341 µs (26.1%)
appsec_no_iast	1.738 ms [1.714 ms, 1.762 ms]	371.047 µs (27.1%)
code_origins	1.691 ms [1.658 ms, 1.723 ms]	323.675 µs (23.7%)
iast	1.505 ms [1.481 ms, 1.529 ms]	137.817 µs (10.1%)
profiling	1.516 ms [1.492 ms, 1.54 ms]	148.934 µs (10.9%)
tracing	1.505 ms [1.481 ms, 1.53 ms]	138.385 µs (10.1%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (379.087 µs) : 359, 399
.   : milestone, 379,
iast (511.173 µs) : 489, 533
.   : milestone, 511,
iast_FULL (730.438 µs) : 708, 753
.   : milestone, 730,
iast_GLOBAL (557.059 µs) : 535, 579
.   : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (507.495 µs) : 486, 529
.   : milestone, 507,
iast_INACTIVE (456.246 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (501.603 µs) : 479, 524
.   : milestone, 502,
tracing (455.758 µs) : 432, 479
.   : milestone, 456,
section candidate
no_agent (381.244 µs) : 360, 403
.   : milestone, 381,
iast (506.75 µs) : 484, 529
.   : milestone, 507,
iast_FULL (726.905 µs) : 705, 749
.   : milestone, 727,
iast_GLOBAL (556.346 µs) : 535, 578
.   : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (510.459 µs) : 488, 533
.   : milestone, 510,
iast_INACTIVE (463.966 µs) : 442, 486
.   : milestone, 464,
iast_TELEMETRY_OFF (493.233 µs) : 471, 515
.   : milestone, 493,
tracing (453.344 µs) : 433, 474
.   : milestone, 453,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	379.087 µs [358.791 µs, 399.382 µs]	-
iast	511.173 µs [489.14 µs, 533.206 µs]	132.086 µs (34.8%)
iast_FULL	730.438 µs [708.281 µs, 752.595 µs]	351.351 µs (92.7%)
iast_GLOBAL	557.059 µs [534.785 µs, 579.333 µs]	177.972 µs (46.9%)
iast_HARDCODED_SECRET_DISABLED	507.495 µs [485.823 µs, 529.168 µs]	128.409 µs (33.9%)
iast_INACTIVE	456.246 µs [435.286 µs, 477.205 µs]	77.159 µs (20.4%)
iast_TELEMETRY_OFF	501.603 µs [479.356 µs, 523.85 µs]	122.516 µs (32.3%)
tracing	455.758 µs [432.384 µs, 479.133 µs]	76.672 µs (20.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.244 µs [359.643 µs, 402.846 µs]	-
iast	506.75 µs [484.411 µs, 529.088 µs]	125.505 µs (32.9%)
iast_FULL	726.905 µs [705.119 µs, 748.692 µs]	345.661 µs (90.7%)
iast_GLOBAL	556.346 µs [534.594 µs, 578.097 µs]	175.101 µs (45.9%)
iast_HARDCODED_SECRET_DISABLED	510.459 µs [487.695 µs, 533.222 µs]	129.214 µs (33.9%)
iast_INACTIVE	463.966 µs [442.297 µs, 485.636 µs]	82.722 µs (21.7%)
iast_TELEMETRY_OFF	493.233 µs [471.497 µs, 514.97 µs]	111.989 µs (29.4%)
tracing	453.344 µs [432.654 µs, 474.033 µs]	72.1 µs (18.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-env-be03ce5b82b7cc8ca9cf2fe3f8af015d39690559
git_commit_date	1741082792	1740763125
git_commit_sha	`8f945b0`	`be03ce5`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~be03ce5b82

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1741091217	1741091217
ci_job_id	831518995	831518995
ci_pipeline_id	57626503	57626503
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner--t2f38de-project-304-concurrent-5-mii1hi44 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner--t2f38de-project-304-concurrent-5-mii1hi44 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.775 s) : 14775000, 14775000
.   : milestone, 14775000,
appsec (14.741 s) : 14741000, 14741000
.   : milestone, 14741000,
iast (18.54 s) : 18540000, 18540000
.   : milestone, 18540000,
iast_GLOBAL (17.83 s) : 17830000, 17830000
.   : milestone, 17830000,
profiling (14.927 s) : 14927000, 14927000
.   : milestone, 14927000,
tracing (14.868 s) : 14868000, 14868000
.   : milestone, 14868000,
section candidate
no_agent (15.04 s) : 15040000, 15040000
.   : milestone, 15040000,
appsec (14.985 s) : 14985000, 14985000
.   : milestone, 14985000,
iast (18.426 s) : 18426000, 18426000
.   : milestone, 18426000,
iast_GLOBAL (17.682 s) : 17682000, 17682000
.   : milestone, 17682000,
profiling (15.044 s) : 15044000, 15044000
.   : milestone, 15044000,
tracing (15.03 s) : 15030000, 15030000
.   : milestone, 15030000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.775 s [14.775 s, 14.775 s]	-
appsec	14.741 s [14.741 s, 14.741 s]	-34.0 ms (-0.2%)
iast	18.54 s [18.54 s, 18.54 s]	3.765 s (25.5%)
iast_GLOBAL	17.83 s [17.83 s, 17.83 s]	3.055 s (20.7%)
profiling	14.927 s [14.927 s, 14.927 s]	152.0 ms (1.0%)
tracing	14.868 s [14.868 s, 14.868 s]	93.0 ms (0.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.04 s [15.04 s, 15.04 s]	-
appsec	14.985 s [14.985 s, 14.985 s]	-55.0 ms (-0.4%)
iast	18.426 s [18.426 s, 18.426 s]	3.386 s (22.5%)
iast_GLOBAL	17.682 s [17.682 s, 17.682 s]	2.642 s (17.6%)
profiling	15.044 s [15.044 s, 15.044 s]	4.0 ms (0.0%)
tracing	15.03 s [15.03 s, 15.03 s]	-10.0 ms (-0.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~be03ce5b82, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1457, 1481
.   : milestone, 1469,
appsec (2.336 ms) : 2292, 2380
.   : milestone, 2336,
iast (2.115 ms) : 2060, 2170
.   : milestone, 2115,
iast_GLOBAL (2.151 ms) : 2096, 2207
.   : milestone, 2151,
profiling (1.959 ms) : 1915, 2002
.   : milestone, 1959,
tracing (1.946 ms) : 1903, 1989
.   : milestone, 1946,
section candidate
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.341 ms) : 2297, 2385
.   : milestone, 2341,
iast (2.109 ms) : 2054, 2164
.   : milestone, 2109,
iast_GLOBAL (2.147 ms) : 2091, 2202
.   : milestone, 2147,
profiling (1.961 ms) : 1917, 2005
.   : milestone, 1961,
tracing (1.951 ms) : 1909, 1993
.   : milestone, 1951,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.457 ms, 1.481 ms]	-
appsec	2.336 ms [2.292 ms, 2.38 ms]	866.824 µs (59.0%)
iast	2.115 ms [2.06 ms, 2.17 ms]	645.859 µs (44.0%)
iast_GLOBAL	2.151 ms [2.096 ms, 2.207 ms]	682.241 µs (46.4%)
profiling	1.959 ms [1.915 ms, 2.002 ms]	489.638 µs (33.3%)
tracing	1.946 ms [1.903 ms, 1.989 ms]	477.165 µs (32.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.458 ms, 1.481 ms]	-
appsec	2.341 ms [2.297 ms, 2.385 ms]	871.991 µs (59.3%)
iast	2.109 ms [2.054 ms, 2.164 ms]	639.466 µs (43.5%)
iast_GLOBAL	2.147 ms [2.091 ms, 2.202 ms]	677.278 µs (46.1%)
profiling	1.961 ms [1.917 ms, 2.005 ms]	491.827 µs (33.5%)
tracing	1.951 ms [1.909 ms, 1.993 ms]	481.798 µs (32.8%)

mcculls and others added 2 commits February 28, 2025 17:17

Reduce use of activeScope() in instrumentation (#8470)

ba51027

Don't instrument trigger probes if the distributed debugger is disabl…

be03ce5

…ed (#8458)

nikita-tkachenko-datadog added the tag: do not merge Do not merge changes label Mar 4, 2025

nikita-tkachenko-datadog closed this Mar 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Nikita tkachenko/test env be03ce5b82b7cc8ca9cf2fe3f8af015d39690559#8494

nikita-tkachenko-datadog commented Mar 4, 2025

Uh oh!

pr-commenter Bot commented Mar 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Search code, repositories, users, issues, pull requests...

Uh oh!

Conversation

nikita-tkachenko-datadog commented Mar 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter Bot commented Mar 4, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants