Skip to content

benchmarks: prepare args outside benchmark - #10388

Merged
cakebaker merged 1 commit into
uutils:mainfrom
xtqqczze:get_benchmark_args
Aug 3, 2026
Merged

benchmarks: prepare args outside benchmark#10388
cakebaker merged 1 commit into
uutils:mainfrom
xtqqczze:get_benchmark_args

Conversation

@xtqqczze

@xtqqczze xtqqczze commented Jan 20, 2026

Copy link
Copy Markdown
Contributor

Currently, we construct uucore::Args within the measured section of benchmarks. That adds allocation and conversion overhead and will skew the results.

Not all benchmarks are converted yet, but perhaps we can take this as a first step.

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

GNU test failed: tests/tail/retry. tests/tail/retry is passing on 'main'. Maybe you have to rebase?

@codspeed-hq

codspeed-hq Bot commented Jan 20, 2026

Copy link
Copy Markdown

Merging this PR will regress 7 benchmarks

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 6 improved benchmarks
❌ 7 regressed benchmarks
✅ 332 untouched benchmarks
⏩ 46 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation cksum_crc32b 40.9 ms 44.1 ms -7.26%
Simulation cksum_default 38.2 ms 40.4 ms -5.28%
Simulation expand_custom_tabstops[50000] 29.8 ms 31.4 ms -4.97%
Simulation cksum_raw_output 38.2 ms 39.8 ms -3.85%
Simulation cksum_multiple_files 58.3 ms 60.6 ms -3.79%
Simulation cksum_crc 39.4 ms 40.9 ms -3.75%
Simulation expand_many_lines[100000] 112.3 ms 116.4 ms -3.51%
Memory du_deep_tree[(100, 3)] 3,209.6 KB 93 KB ×35
Simulation du_wide_tree[(5000, 500)] 22.3 ms 19.5 ms +14.59%
Simulation du_deep_tree[(100, 3)] 2.6 ms 2.3 ms +11.81%
Simulation unexpand_large_file[10] 285.2 ms 267 ms +6.81%
Simulation unexpand_many_lines[100000] 136.3 ms 127.6 ms +6.8%
Simulation du_all_wide_tree[(5000, 500)] 41.8 ms 39.9 ms +4.64%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing xtqqczze:get_benchmark_args (330e0bc) with main (cb8b2b8)2

Open in CodSpeed

Footnotes

  1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (a730551) during the generation of this report, so cb8b2b8 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@xtqqczze

Copy link
Copy Markdown
Contributor Author

@sylvestre Please could you confirm whether these changes are viable to apply in full?

@sylvestre

Copy link
Copy Markdown
Contributor

i guess ? :)

@xtqqczze

Copy link
Copy Markdown
Contributor Author

Need to refactor to handle the following case:

bencher.bench(|| {
fs_utils::remove_path(&dest);
let mut full_args = Vec::with_capacity(args.len() + 2);
full_args.extend_from_slice(args);
full_args.push(source_str);
full_args.push(dest_str);

@oech3

oech3 commented Jan 26, 2026

Copy link
Copy Markdown
Contributor

Should we wait working about #10486 until your PR was merged?
Any chance for splitting PR?

xtqqczze added a commit to xtqqczze/uutils-coreutils that referenced this pull request Jan 26, 2026
@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

GNU test failed: tests/pr/bounded-memory. tests/pr/bounded-memory is passing on 'main'. Maybe you have to rebase?

@oech3

oech3 commented Feb 13, 2026

Copy link
Copy Markdown
Contributor

is this able to merge?

@xtqqczze

Copy link
Copy Markdown
Contributor Author

is this able to merge?

I couldn't work out how to handle all the cases, such as #10388 (comment)

@oech3

oech3 commented Feb 15, 2026

Copy link
Copy Markdown
Contributor

How about splitting this PR?

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/rm/many-dir-entries-vs-OOM is now being skipped but was previously passing.
Congrats! The gnu test tests/cp/link-heap is now passing!
Congrats! The gnu test tests/cut/cut-huge-range is now passing!
Note: The gnu test tests/env/env-signal-handler was skipped on 'main' but is now failing.

@xtqqczze

This comment was marked as outdated.

@oech3

oech3 commented Mar 5, 2026

Copy link
Copy Markdown
Contributor

Any progress about this?

@xtqqczze
xtqqczze force-pushed the get_benchmark_args branch from b2aedf9 to ae7ca7b Compare March 5, 2026 21:23
@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/cut/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/date/resolution (passes in this run but fails in the 'main' branch)

@xtqqczze
xtqqczze force-pushed the get_benchmark_args branch from ae7ca7b to 8b71b7f Compare March 5, 2026 22:06
@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/cut/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/date/resolution (passes in this run but fails in the 'main' branch)

@xtqqczze
xtqqczze force-pushed the get_benchmark_args branch from 8b71b7f to 2ba6b6a Compare March 5, 2026 23:25
@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/cut/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/date/resolution (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tty/tty-eof (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/csplit/csplit-heap is now being skipped but was previously passing.

@github-actions

github-actions Bot commented Mar 8, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/follow-name (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/printf/printf-surprise is now being skipped but was previously passing.
Congrats! The gnu test tests/rm/many-dir-entries-vs-OOM is now passing!
Congrats! The gnu test tests/tail/pipe-f is now passing!

@xtqqczze
xtqqczze force-pushed the get_benchmark_args branch from 1dedff9 to a2788c0 Compare March 8, 2026 13:03
@xtqqczze

xtqqczze commented Mar 8, 2026

Copy link
Copy Markdown
Contributor Author

Any progress about this?

Finally, I have clean benchmark results

@github-actions

github-actions Bot commented Mar 8, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

GNU test failed: tests/tail/pid-pipe. tests/tail/pid-pipe is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tail/inotify-dir-recreate (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tty/tty-eof (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/basenc/bounded-memory is now being skipped but was previously passing.
Note: The gnu test tests/tail/tail-n0f is now being skipped but was previously passing.
Congrats! The gnu test tests/rm/many-dir-entries-vs-OOM is now passing!
Congrats! The gnu test tests/tail/pipe-f is now passing!

@xtqqczze
xtqqczze marked this pull request as ready for review March 8, 2026 13:59
@xtqqczze
xtqqczze force-pushed the get_benchmark_args branch from a2788c0 to 984e285 Compare August 2, 2026 23:20
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/retry (fails in this run but passes in the 'main' branch)

@xtqqczze
xtqqczze force-pushed the get_benchmark_args branch from 984e285 to 4ca86fd Compare August 3, 2026 00:27
@xtqqczze
xtqqczze force-pushed the get_benchmark_args branch from 4ca86fd to 330e0bc Compare August 3, 2026 08:41
@cakebaker
cakebaker merged commit 79fcc9e into uutils:main Aug 3, 2026
163 of 164 checks passed
@cakebaker

Copy link
Copy Markdown
Contributor

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants