perf(row): compact variable-list keys - #9554
Conversation
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Merging this PR will improve performance by 25.68%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | list_i64_vortex |
2.2 ms | 1.6 ms | +34.55% |
| ⚡ | Simulation | list_utf8_vortex |
4.1 ms | 3.5 ms | +17.39% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ngates/vortex-row-list-performance (87996f3) with ngates/vortex-row-dtype-support (41da939)
Footnotes
-
54 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Summary
Depends on #9553.
Performance
cargo bench -p vortex-row --bench row_encode -- --output-sizesList<i64>List<Utf8>Local median encode times with 1,000 lists of 8 elements:
List<i64>List<Utf8>The corresponding
arrow-rowmedians in the before/after runs were 36.83/36.56 us forList<i64>and 94.74/97.62 us forList<Utf8>, providing a noise anchor.Testing
cargo test -p vortex-row(33 tests plus doctest)cargo clippy -p vortex-row --all-targets --all-featurescargo +nightly fmt --all -- --checkgit diff --checkcargo bench -p vortex-row --bench row_encode -- --output-sizescargo bench -p vortex-row --bench row_encode -- list_ --sample-count 20 --min-time 0.2 --max-time 0.5 --color never