Skip to content

Window function perf: Speed up rank window materialization#23021

Draft
Dandandan wants to merge 1 commit into
apache:mainfrom
Dandandan:codex/window-rank-fast-path
Draft

Window function perf: Speed up rank window materialization#23021
Dandandan wants to merge 1 commit into
apache:mainfrom
Dandandan:codex/window-rank-fast-path

Conversation

@Dandandan

@Dandandan Dandandan commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • None.

Rationale for this change

rank, dense_rank, percent_rank, and cume_dist currently materialize rank output arrays through nested iterator pipelines using scan, repeat_n, and flatten. These paths run for every partition for rank-like window functions, so avoiding iterator layering helps both all-distinct and repeated peer-group partitions.

What changes are included in this PR?

  • Adds a singleton peer-group fast path for rank-like window functions when every row has its own rank group.
  • Replaces repeated peer-group iterator materialization with preallocated vector filling for rank, dense_rank, percent_rank, and cume_dist.
  • Adds focused Criterion coverage for rank-family evaluate_all_with_rank materialization.

Short local Criterion run on 8192 rows after the latest update:

  • singleton groups: rank 10.466 us -> 1.090 us, dense_rank 10.491 us -> 1.058 us, percent_rank 13.093 us -> 1.877 us, cume_dist 12.699 us -> 1.796 us
  • group size 8: rank 7.971 us -> 3.975 us, dense_rank 7.208 us -> 3.890 us, percent_rank 8.910 us -> 4.176 us, cume_dist 6.157 us -> 4.108 us
  • group size 64: rank 7.974 us -> 1.283 us, dense_rank 5.518 us -> 1.201 us, percent_rank 8.134 us -> 1.190 us, cume_dist 4.747 us -> 1.211 us

Benchmark command:

cargo bench -p datafusion-functions-window --bench rank -- --sample-size 10 --warm-up-time 1 --measurement-time 2

Are these changes tested?

Yes.

cargo fmt --all
cargo test -p datafusion-functions-window
cargo clippy --all-targets --all-features -- -D warnings
cargo bench -p datafusion-functions-window --bench rank -- --sample-size 10 --warm-up-time 1 --measurement-time 2

Are there any user-facing changes?

No API or behavior changes. This only changes internal array materialization for rank-like window functions.

@github-actions github-actions Bot added the functions Changes to functions implementation label Jun 18, 2026
@Dandandan Dandandan changed the title [codex] Speed up rank window materialization Window function perf: Speed up rank window materialization Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant