List iteration benchmark by dougqh · Pull Request #10888 · DataDog/dd-trace-java

dougqh · 2026-03-18T14:00:21Z

What Does This Do

Benchmark examining different ways to iterate over Lists: Collections.emptyList, Collections.singletonList, and ArrayList of different sizes

Covers multiple iteration options...

c-style for ( int i = 0; i < list.size(); ++i ) ...
iterator
enhanced for ( x : list ) (sugar for iterator)
List.forEach
List.stream
List.parallelStream

Each benchmark has a variation with and without the loop body fully inlined

Motivation

Illustrate overhead and make recommendations about the preferred way to iterate over lists in dd-java-agent.

Additional Notes

Recommends are...

prefer enhanced for -- except in the uncommon situation where escape analysis fails -- in that case, forEach or c-style are the best altenatives
avoid streams - always incur extra allocation - generally unnecessary for the task at hand

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

pr-commenter · 2026-03-18T14:47:03Z

Benchmarks

⚠️ Warning: Baseline build not found for merge-base commit. Comparing against the latest commit on master instead.

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/list-iteration-benchmark
git_commit_date	1782158081	1782171008
git_commit_sha	`ea827ba`	`b816fde`
release_version	1.64.0-SNAPSHOT~ea827ba0c4	1.61.0-SNAPSHOT~b816fdeed7

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1782173150	1782173150
ci_job_id	1794044753	1794044753
ci_pipeline_id	120371156	120371156
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-oyjdvy3d 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-oyjdvy3d 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 8 performance improvements and 1 performance regressions! Performance is the same for 48 metrics, 14 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:BytebuddyAgent	better [-32.818ms; -19.041ms] or [-3.969%; -2.303%]	800.992ms	826.921ms
scenario:startup:petclinic:appsec:BytebuddyAgent	better [-19.483ms; -13.711ms] or [-2.869%; -2.019%]	662.427ms	679.024ms
scenario:startup:petclinic:appsec:AppSec	better [-5.755ms; -4.441ms] or [-3.042%; -2.347%]	184.125ms	189.223ms
scenario:startup:petclinic:appsec:Telemetry	worse [+346.258µs; +791.768µs] or [+4.357%; +9.964%]	8.516ms	7.947ms
scenario:startup:petclinic:appsec:Flare Poller	better [-5.904ms; -5.541ms] or [-63.776%; -59.855%]	3.535ms	9.258ms
scenario:startup:petclinic:profiling:Agent.start	better [-152.079ms; -102.303ms] or [-11.456%; -7.706%]	1.200s	1.328s
scenario:startup:petclinic:profiling:Remote Config	better [-69.886µs; -19.354µs] or [-11.263%; -3.119%]	575.852µs	620.472µs
scenario:startup:petclinic:tracing:AgentMeter	better [-1081.498µs; -652.191µs] or [-3.588%; -2.164%]	29.277ms	30.144ms
scenario:startup:petclinic:tracing:Telemetry	better [-519.928µs; -318.758µs] or [-6.133%; -3.760%]	8.058ms	8.477ms

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1071462
Total [baseline] (10.964 s) : 0, 10963509
Agent [candidate] (1.059 s) : 0, 1058573
Total [candidate] (11.025 s) : 0, 11024502
section appsec
Agent [baseline] (1.274 s) : 0, 1273545
Total [baseline] (10.951 s) : 0, 10950504
Agent [candidate] (1.248 s) : 0, 1248302
Total [candidate] (11.196 s) : 0, 11195889
section iast
Agent [baseline] (1.256 s) : 0, 1256403
Total [baseline] (11.352 s) : 0, 11352370
Agent [candidate] (1.231 s) : 0, 1231317
Total [candidate] (11.294 s) : 0, 11294026
section profiling
Agent [baseline] (1.328 s) : 0, 1327526
Total [baseline] (11.145 s) : 0, 11144568
Agent [candidate] (1.2 s) : 0, 1200335
Total [candidate] (11.019 s) : 0, 11018783

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.071 s	-
Agent	appsec	1.274 s	202.083 ms (18.9%)
Agent	iast	1.256 s	184.941 ms (17.3%)
Agent	profiling	1.328 s	256.063 ms (23.9%)
Total	tracing	10.964 s	-
Total	appsec	10.951 s	-13.005 ms (-0.1%)
Total	iast	11.352 s	388.861 ms (3.5%)
Total	profiling	11.145 s	181.059 ms (1.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.248 s	189.729 ms (17.9%)
Agent	iast	1.231 s	172.744 ms (16.3%)
Agent	profiling	1.2 s	141.762 ms (13.4%)
Total	tracing	11.025 s	-
Total	appsec	11.196 s	171.387 ms (1.6%)
Total	iast	11.294 s	269.524 ms (2.4%)
Total	profiling	11.019 s	-5.718 ms (-0.1%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.22 ms) : 0, 1220
crashtracking [candidate] (1.229 ms) : 0, 1229
BytebuddyAgent [baseline] (641.003 ms) : 0, 641003
BytebuddyAgent [candidate] (632.476 ms) : 0, 632476
AgentMeter [baseline] (30.144 ms) : 0, 30144
AgentMeter [candidate] (29.277 ms) : 0, 29277
GlobalTracer [baseline] (249.378 ms) : 0, 249378
GlobalTracer [candidate] (248.848 ms) : 0, 248848
AppSec [baseline] (32.293 ms) : 0, 32293
AppSec [candidate] (32.083 ms) : 0, 32083
Debugger [baseline] (60.37 ms) : 0, 60370
Debugger [candidate] (59.862 ms) : 0, 59862
Remote Config [baseline] (589.811 µs) : 0, 590
Remote Config [candidate] (600.01 µs) : 0, 600
Telemetry [baseline] (8.477 ms) : 0, 8477
Telemetry [candidate] (8.058 ms) : 0, 8058
Flare Poller [baseline] (11.208 ms) : 0, 11208
Flare Poller [candidate] (9.887 ms) : 0, 9887
section appsec
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.233 ms) : 0, 1233
BytebuddyAgent [baseline] (679.024 ms) : 0, 679024
BytebuddyAgent [candidate] (662.427 ms) : 0, 662427
AgentMeter [baseline] (12.143 ms) : 0, 12143
AgentMeter [candidate] (12.006 ms) : 0, 12006
GlobalTracer [baseline] (247.96 ms) : 0, 247960
GlobalTracer [candidate] (248.988 ms) : 0, 248988
AppSec [baseline] (189.223 ms) : 0, 189223
AppSec [candidate] (184.125 ms) : 0, 184125
Debugger [baseline] (65.148 ms) : 0, 65148
Debugger [candidate] (65.962 ms) : 0, 65962
Remote Config [baseline] (579.344 µs) : 0, 579
Remote Config [candidate] (599.113 µs) : 0, 599
Telemetry [baseline] (7.947 ms) : 0, 7947
Telemetry [candidate] (8.516 ms) : 0, 8516
Flare Poller [baseline] (9.258 ms) : 0, 9258
Flare Poller [candidate] (3.535 ms) : 0, 3535
IAST [baseline] (24.276 ms) : 0, 24276
IAST [candidate] (24.614 ms) : 0, 24614
section iast
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.236 ms) : 0, 1236
BytebuddyAgent [baseline] (834.191 ms) : 0, 834191
BytebuddyAgent [candidate] (806.659 ms) : 0, 806659
AgentMeter [baseline] (11.651 ms) : 0, 11651
AgentMeter [candidate] (11.512 ms) : 0, 11512
GlobalTracer [baseline] (243.944 ms) : 0, 243944
GlobalTracer [candidate] (240.19 ms) : 0, 240190
AppSec [baseline] (25.854 ms) : 0, 25854
AppSec [candidate] (31.689 ms) : 0, 31689
Debugger [baseline] (64.122 ms) : 0, 64122
Debugger [candidate] (61.991 ms) : 0, 61991
Remote Config [baseline] (521.402 µs) : 0, 521
Remote Config [candidate] (532.639 µs) : 0, 533
Telemetry [baseline] (7.879 ms) : 0, 7879
Telemetry [candidate] (11.667 ms) : 0, 11667
Flare Poller [baseline] (3.323 ms) : 0, 3323
Flare Poller [candidate] (3.463 ms) : 0, 3463
IAST [baseline] (26.943 ms) : 0, 26943
IAST [candidate] (25.878 ms) : 0, 25878
section profiling
crashtracking [baseline] (549.919 µs) : 0, 550
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (695.831 ms) : 0, 695831
BytebuddyAgent [candidate] (701.345 ms) : 0, 701345
AgentMeter [baseline] (9.943 ms) : 0, 9943
AgentMeter [candidate] (9.275 ms) : 0, 9275
GlobalTracer [baseline] (211.113 ms) : 0, 211113
GlobalTracer [candidate] (210.042 ms) : 0, 210042
AppSec [baseline] (32.742 ms) : 0, 32742
AppSec [candidate] (32.822 ms) : 0, 32822
Debugger [baseline] (66.641 ms) : 0, 66641
Debugger [candidate] (66.157 ms) : 0, 66157
Remote Config [baseline] (620.472 µs) : 0, 620
Remote Config [candidate] (575.852 µs) : 0, 576
Telemetry [baseline] (8.057 ms) : 0, 8057
Telemetry [candidate] (7.995 ms) : 0, 7995
Flare Poller [baseline] (3.597 ms) : 0, 3597
Flare Poller [candidate] (3.614 ms) : 0, 3614
ProfilingAgent [baseline] (96.023 ms) : 0, 96023
ProfilingAgent [candidate] (95.093 ms) : 0, 95093
Profiling [baseline] (96.583 ms) : 0, 96583
Profiling [candidate] (95.673 ms) : 0, 95673

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066231
Total [baseline] (8.871 s) : 0, 8871250
Agent [candidate] (1.067 s) : 0, 1067439
Total [candidate] (8.855 s) : 0, 8855192
section iast
Agent [baseline] (1.246 s) : 0, 1246472
Total [baseline] (9.491 s) : 0, 9490893
Agent [candidate] (1.225 s) : 0, 1224996
Total [candidate] (9.576 s) : 0, 9576358

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.066 s	-
Agent	iast	1.246 s	180.24 ms (16.9%)
Total	tracing	8.871 s	-
Total	iast	9.491 s	619.643 ms (7.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.067 s	-
Agent	iast	1.225 s	157.557 ms (14.8%)
Total	tracing	8.855 s	-
Total	iast	9.576 s	721.166 ms (8.1%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.208 ms) : 0, 1208
crashtracking [candidate] (1.237 ms) : 0, 1237
BytebuddyAgent [baseline] (638.433 ms) : 0, 638433
BytebuddyAgent [candidate] (639.455 ms) : 0, 639455
AgentMeter [baseline] (29.851 ms) : 0, 29851
AgentMeter [candidate] (29.77 ms) : 0, 29770
GlobalTracer [baseline] (247.817 ms) : 0, 247817
GlobalTracer [candidate] (251.511 ms) : 0, 251511
AppSec [baseline] (32.266 ms) : 0, 32266
AppSec [candidate] (32.381 ms) : 0, 32381
Debugger [baseline] (59.64 ms) : 0, 59640
Debugger [candidate] (59.711 ms) : 0, 59711
Remote Config [baseline] (601.099 µs) : 0, 601
Remote Config [candidate] (605.219 µs) : 0, 605
Telemetry [baseline] (8.466 ms) : 0, 8466
Telemetry [candidate] (8.224 ms) : 0, 8224
Flare Poller [baseline] (11.22 ms) : 0, 11220
Flare Poller [candidate] (8.257 ms) : 0, 8257
section iast
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (826.921 ms) : 0, 826921
BytebuddyAgent [candidate] (800.992 ms) : 0, 800992
AgentMeter [baseline] (11.483 ms) : 0, 11483
AgentMeter [candidate] (11.46 ms) : 0, 11460
GlobalTracer [baseline] (238.777 ms) : 0, 238777
GlobalTracer [candidate] (239.791 ms) : 0, 239791
AppSec [baseline] (28.063 ms) : 0, 28063
AppSec [candidate] (31.177 ms) : 0, 31177
Debugger [baseline] (63.063 ms) : 0, 63063
Debugger [candidate] (59.89 ms) : 0, 59890
Remote Config [baseline] (524.688 µs) : 0, 525
Remote Config [candidate] (1.145 ms) : 0, 1145
Telemetry [baseline] (7.753 ms) : 0, 7753
Telemetry [candidate] (13.668 ms) : 0, 13668
Flare Poller [baseline] (3.279 ms) : 0, 3279
Flare Poller [candidate] (3.656 ms) : 0, 3656
IAST [baseline] (28.679 ms) : 0, 28679
IAST [candidate] (25.851 ms) : 0, 25851

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/list-iteration-benchmark
git_commit_date	1782158081	1782171008
git_commit_sha	`ea827ba`	`b816fde`
release_version	1.64.0-SNAPSHOT~ea827ba0c4	1.61.0-SNAPSHOT~b816fdeed7

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1782173769	1782173769
ci_job_id	1794044754	1794044754
ci_pipeline_id	120371156	120371156
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-4z3ubgzq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-4z3ubgzq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load	worse [+331.345µs; +620.788µs] or [+6.179%; +11.576%]	worse [+0.675ms; +1.476ms] or [+5.263%; +11.508%]	unstable [-125.123op/s; +20.311op/s] or [-16.404%; +2.663%]	5.839ms	13.898ms	710.344op/s	5.363ms	12.823ms	762.750op/s
scenario:load:petclinic:profiling:high_load	worse [+1.848ms; +2.159ms] or [+10.353%; +12.098%]	worse [+1.745ms; +2.565ms] or [+5.935%; +8.725%]	unstable [-46.751op/s; -0.561op/s] or [-18.318%; -0.220%]	19.850ms	31.552ms	231.562op/s	17.847ms	29.398ms	255.219op/s
scenario:load:petclinic:iast:high_load	worse [+358.313µs; +1172.597µs] or [+2.028%; +6.638%]	worse [+0.912ms; +2.099ms] or [+3.208%; +7.381%]	unstable [-35.857op/s; +13.169op/s] or [-13.748%; +5.049%]	18.430ms	29.944ms	249.469op/s	17.664ms	28.438ms	260.812op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.278 ms) : 1266, 1291
.   : milestone, 1278,
iast (3.355 ms) : 3303, 3407
.   : milestone, 3355,
iast_FULL (6.063 ms) : 6001, 6125
.   : milestone, 6063,
iast_GLOBAL (3.583 ms) : 3536, 3630
.   : milestone, 3583,
profiling (2.161 ms) : 2141, 2181
.   : milestone, 2161,
tracing (1.929 ms) : 1912, 1945
.   : milestone, 1929,
section candidate
no_agent (1.276 ms) : 1263, 1288
.   : milestone, 1276,
iast (3.327 ms) : 3280, 3374
.   : milestone, 3327,
iast_FULL (6.516 ms) : 6447, 6584
.   : milestone, 6516,
iast_GLOBAL (3.631 ms) : 3571, 3691
.   : milestone, 3631,
profiling (2.348 ms) : 2324, 2373
.   : milestone, 2348,
tracing (1.935 ms) : 1918, 1953
.   : milestone, 1935,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.278 ms [1.266 ms, 1.291 ms]	-
iast	3.355 ms [3.303 ms, 3.407 ms]	2.077 ms (162.4%)
iast_FULL	6.063 ms [6.001 ms, 6.125 ms]	4.784 ms (374.2%)
iast_GLOBAL	3.583 ms [3.536 ms, 3.63 ms]	2.304 ms (180.2%)
profiling	2.161 ms [2.141 ms, 2.181 ms]	882.297 µs (69.0%)
tracing	1.929 ms [1.912 ms, 1.945 ms]	650.06 µs (50.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.276 ms [1.263 ms, 1.288 ms]	-
iast	3.327 ms [3.28 ms, 3.374 ms]	2.051 ms (160.8%)
iast_FULL	6.516 ms [6.447 ms, 6.584 ms]	5.24 ms (410.7%)
iast_GLOBAL	3.631 ms [3.571 ms, 3.691 ms]	2.355 ms (184.6%)
profiling	2.348 ms [2.324 ms, 2.373 ms]	1.073 ms (84.1%)
tracing	1.935 ms [1.918 ms, 1.953 ms]	659.403 µs (51.7%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.013 ms) : 17834, 18193
.   : milestone, 18013,
appsec (18.655 ms) : 18467, 18843
.   : milestone, 18655,
code_origins (18.848 ms) : 18659, 19038
.   : milestone, 18848,
iast (17.888 ms) : 17715, 18062
.   : milestone, 17888,
profiling (18.281 ms) : 18097, 18464
.   : milestone, 18281,
tracing (17.929 ms) : 17755, 18104
.   : milestone, 17929,
section candidate
no_agent (19.146 ms) : 18949, 19342
.   : milestone, 19146,
appsec (18.559 ms) : 18373, 18745
.   : milestone, 18559,
code_origins (18.047 ms) : 17867, 18227
.   : milestone, 18047,
iast (18.709 ms) : 18522, 18896
.   : milestone, 18709,
profiling (20.165 ms) : 19964, 20365
.   : milestone, 20165,
tracing (17.753 ms) : 17577, 17928
.   : milestone, 17753,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.013 ms [17.834 ms, 18.193 ms]	-
appsec	18.655 ms [18.467 ms, 18.843 ms]	641.887 µs (3.6%)
code_origins	18.848 ms [18.659 ms, 19.038 ms]	834.766 µs (4.6%)
iast	17.888 ms [17.715 ms, 18.062 ms]	-125.125 µs (-0.7%)
profiling	18.281 ms [18.097 ms, 18.464 ms]	267.322 µs (1.5%)
tracing	17.929 ms [17.755 ms, 18.104 ms]	-83.989 µs (-0.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.146 ms [18.949 ms, 19.342 ms]	-
appsec	18.559 ms [18.373 ms, 18.745 ms]	-586.929 µs (-3.1%)
code_origins	18.047 ms [17.867 ms, 18.227 ms]	-1.099 ms (-5.7%)
iast	18.709 ms [18.522 ms, 18.896 ms]	-436.281 µs (-2.3%)
profiling	20.165 ms [19.964 ms, 20.365 ms]	1.019 ms (5.3%)
tracing	17.753 ms [17.577 ms, 17.928 ms]	-1.393 ms (-7.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/list-iteration-benchmark
git_commit_date	1782158081	1782171008
git_commit_sha	`ea827ba`	`b816fde`
release_version	1.64.0-SNAPSHOT~ea827ba0c4	1.61.0-SNAPSHOT~b816fdeed7

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1782173374	1782173374
ci_job_id	1794044755	1794044755
ci_pipeline_id	120371156	120371156
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-exmlhb4b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-exmlhb4b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.491 ms) : 1479, 1502
.   : milestone, 1491,
appsec (3.856 ms) : 3631, 4080
.   : milestone, 3856,
iast (2.294 ms) : 2224, 2365
.   : milestone, 2294,
iast_GLOBAL (2.33 ms) : 2258, 2401
.   : milestone, 2330,
profiling (2.563 ms) : 2397, 2728
.   : milestone, 2563,
tracing (2.118 ms) : 2063, 2173
.   : milestone, 2118,
section candidate
no_agent (1.497 ms) : 1486, 1509
.   : milestone, 1497,
appsec (3.815 ms) : 3589, 4040
.   : milestone, 3815,
iast (2.277 ms) : 2208, 2346
.   : milestone, 2277,
iast_GLOBAL (2.318 ms) : 2248, 2387
.   : milestone, 2318,
profiling (2.098 ms) : 2043, 2153
.   : milestone, 2098,
tracing (2.069 ms) : 2016, 2122
.   : milestone, 2069,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.491 ms [1.479 ms, 1.502 ms]	-
appsec	3.856 ms [3.631 ms, 4.08 ms]	2.365 ms (158.7%)
iast	2.294 ms [2.224 ms, 2.365 ms]	803.494 µs (53.9%)
iast_GLOBAL	2.33 ms [2.258 ms, 2.401 ms]	839.202 µs (56.3%)
profiling	2.563 ms [2.397 ms, 2.728 ms]	1.072 ms (71.9%)
tracing	2.118 ms [2.063 ms, 2.173 ms]	627.097 µs (42.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.497 ms [1.486 ms, 1.509 ms]	-
appsec	3.815 ms [3.589 ms, 4.04 ms]	2.317 ms (154.8%)
iast	2.277 ms [2.208 ms, 2.346 ms]	779.297 µs (52.0%)
iast_GLOBAL	2.318 ms [2.248 ms, 2.387 ms]	820.478 µs (54.8%)
profiling	2.098 ms [2.043 ms, 2.153 ms]	600.682 µs (40.1%)
tracing	2.069 ms [2.016 ms, 2.122 ms]	571.765 µs (38.2%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~b816fdeed7, baseline=1.64.0-SNAPSHOT~ea827ba0c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.946 s) : 14946000, 14946000
.   : milestone, 14946000,
appsec (14.885 s) : 14885000, 14885000
.   : milestone, 14885000,
iast (18.649 s) : 18649000, 18649000
.   : milestone, 18649000,
iast_GLOBAL (18.034 s) : 18034000, 18034000
.   : milestone, 18034000,
profiling (14.961 s) : 14961000, 14961000
.   : milestone, 14961000,
tracing (14.741 s) : 14741000, 14741000
.   : milestone, 14741000,
section candidate
no_agent (15.423 s) : 15423000, 15423000
.   : milestone, 15423000,
appsec (14.776 s) : 14776000, 14776000
.   : milestone, 14776000,
iast (18.147 s) : 18147000, 18147000
.   : milestone, 18147000,
iast_GLOBAL (18.005 s) : 18005000, 18005000
.   : milestone, 18005000,
profiling (15.464 s) : 15464000, 15464000
.   : milestone, 15464000,
tracing (15.052 s) : 15052000, 15052000
.   : milestone, 15052000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.946 s [14.946 s, 14.946 s]	-
appsec	14.885 s [14.885 s, 14.885 s]	-61.0 ms (-0.4%)
iast	18.649 s [18.649 s, 18.649 s]	3.703 s (24.8%)
iast_GLOBAL	18.034 s [18.034 s, 18.034 s]	3.088 s (20.7%)
profiling	14.961 s [14.961 s, 14.961 s]	15.0 ms (0.1%)
tracing	14.741 s [14.741 s, 14.741 s]	-205.0 ms (-1.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.423 s [15.423 s, 15.423 s]	-
appsec	14.776 s [14.776 s, 14.776 s]	-647.0 ms (-4.2%)
iast	18.147 s [18.147 s, 18.147 s]	2.724 s (17.7%)
iast_GLOBAL	18.005 s [18.005 s, 18.005 s]	2.582 s (16.7%)
profiling	15.464 s [15.464 s, 15.464 s]	41.0 ms (0.3%)
tracing	15.052 s [15.052 s, 15.052 s]	-371.0 ms (-2.4%)

…tion earlier

Adding missing end ul to doc Moving iterator benchmarks next to enhancedFor

…dd-trace-java into dougqh/list-iteration-benchmark

…hmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

dd-octo-sts · 2026-06-23T14:36:16Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.97 s	13.99 s	[-0.9%; +0.7%] (no difference)
startup:insecure-bank:tracing:Agent	12.84 s	12.97 s	[-1.7%; -0.3%] (maybe better)
startup:petclinic:appsec:Agent	16.79 s	16.55 s	[+0.4%; +2.5%] (maybe worse)
startup:petclinic:iast:Agent	16.37 s	16.46 s	[-6.8%; +5.7%] (unstable)
startup:petclinic:profiling:Agent	16.71 s	16.86 s	[-2.0%; +0.2%] (no difference)
startup:petclinic:sca:Agent	16.92 s	16.18 s	[+0.3%; +8.9%] (maybe worse)
startup:petclinic:tracing:Agent	16.18 s	16.09 s	[-0.5%; +1.6%] (no difference)

Commit: a981815e · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

sarahchen6 · 2026-06-23T15:32:37Z

+@Warmup(iterations = 2)
+@Measurement(iterations = 3)
+@Threads(8)
+@State(Scope.Benchmark)


Would it be better to use @State(Scope.Thread) here to avoid different benchmarks iterating on the same fields?

Yeah, that might be a more realistic comparison.

For maps, I think the shared map has some merit for the precomputed case, but that doesn't so much apply to lists.

…hmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

@setup

Build each thread's list (and its Elements) in a Scope.Thread @setup so the manipulate_* mutations stay thread-local. Previously the lists lived in enum constants shared across all 8 threads, so the benchmark measured cross-thread contention on Element.num rather than iteration cost. Also bump to @fork(2) and fix a Javadoc typo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

sarahchen6

I assume the numbers need to be updated with the new changes, but pre-approving!

dougqh · 2026-06-23T20:02:52Z

I assume the numbers need to be updated with the new changes, but pre-approving!

@sarahchen6
Yes, you assume correctly. I'm going to re-run the benchmark tonight, update the results, and then merge. Thanks for the review.

…on-benchmark

List iteration benchmark

d2a2295

dougqh requested a review from a team as a code owner March 18, 2026 14:00

dougqh requested a review from ygree March 18, 2026 14:00

dougqh added comp: core Tracer core type: documentation tag: performance Performance related changes labels Mar 18, 2026

Merge branch 'master' into dougqh/list-iteration-benchmark

372dff0

dougqh added 5 commits March 18, 2026 13:14

Fixing silly oversight - commented out benchmarks to run one in isola…

54883cd

…tion earlier

A bit of clean-up

d002783

Adding missing end ul to doc Moving iterator benchmarks next to enhancedFor

Merge branch 'master' into dougqh/list-iteration-benchmark

0636a14

spotless

27a33f0

Merge branch 'dougqh/list-iteration-benchmark' of github.com:DataDog/…

b19d281

…dd-trace-java into dougqh/list-iteration-benchmark

sarahchen6 reviewed Jun 22, 2026

View reviewed changes

Comment thread internal-api/src/jmh/java/datadog/trace/util/ListIterationBenchmark.java Outdated

Comment thread internal-api/src/jmh/java/datadog/trace/util/ListIterationBenchmark.java

dougqh and others added 2 commits June 22, 2026 19:29

Update internal-api/src/jmh/java/datadog/trace/util/ListIterationBenc…

8056b56

…hmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

Update internal-api/src/jmh/java/datadog/trace/util/ListIterationBenc…

b816fde

…hmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

This comment has been minimized.

Sign in to view

Merge branch 'master' into dougqh/list-iteration-benchmark

1f0f4e5

sarahchen6 reviewed Jun 23, 2026

View reviewed changes

Comment thread internal-api/src/jmh/java/datadog/trace/util/ListIterationBenchmark.java Outdated

sarahchen6 reviewed Jun 23, 2026

View reviewed changes

dougqh and others added 2 commits June 23, 2026 13:36

Update internal-api/src/jmh/java/datadog/trace/util/ListIterationBenc…

600cc9b

…hmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label Jun 23, 2026

sarahchen6 approved these changes Jun 23, 2026

View reviewed changes

Merge remote-tracking branch 'origin/master' into dougqh/list-iterati…

a981815

…on-benchmark

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

List iteration benchmark#10888

List iteration benchmark#10888
dougqh wants to merge 13 commits into
masterfrom
dougqh/list-iteration-benchmark

dougqh commented Mar 18, 2026

Uh oh!

pr-commenter Bot commented Mar 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Jun 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

sarahchen6 Jun 23, 2026

Uh oh!

dougqh Jun 23, 2026 •

edited

Loading

Uh oh!

sarahchen6 left a comment

Uh oh!

dougqh commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dougqh commented Mar 18, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter Bot commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

Uh oh!

sarahchen6 Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

dougqh Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sarahchen6 left a comment

Choose a reason for hiding this comment

Uh oh!

dougqh commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pr-commenter Bot commented Mar 18, 2026 •

edited

Loading

dd-octo-sts Bot commented Jun 23, 2026 •

edited

Loading

dougqh Jun 23, 2026 •

edited

Loading