Skip to content

feat(row): support remaining ordered dtypes - #9553

Draft
gatesn wants to merge 2 commits into
developfrom
ngates/vortex-row-dtype-support
Draft

feat(row): support remaining ordered dtypes#9553
gatesn wants to merge 2 commits into
developfrom
ngates/vortex-row-dtype-support

Conversation

@gatesn

@gatesn gatesn commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add Decimal256 row keys using the signed 256-bit order-preserving transform
  • add prefix-safe lexicographic encoding for variable-size lists and maps
  • keep Extension rejected until extension dtypes can explicitly declare storage-order equivalence
  • add reproducible List<i64> and List<Utf8> throughput and output-size benchmarks against arrow-row
  • update the row fuzz oracle and byte-format specification

Stack

Baseline benchmarks

cargo bench -p vortex-row --bench row_encode -- --output-sizes

Case arrow-row bytes Vortex bytes
List<i64> 153,000 146,000
List<Utf8> 256,347 546,000

Local median encode times with 1,000 lists of 8 elements:

Case arrow-row Vortex
List<i64> 36.83 us 46.74 us
List<Utf8> 94.74 us 100.7 us

Testing

  • cargo test -p vortex-row
  • cargo check -p vortex-row --bench row_encode
  • cargo clippy -p vortex-row --all-targets --all-features
  • cargo check -p vortex-fuzz --no-default-features --lib
  • cargo +nightly fmt --all -- --check
  • git diff --check

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@codspeed-hq

codspeed-hq Bot commented Aug 23, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 1981 untouched benchmarks
🆕 4 new benchmarks
⏩ 54 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation list_i64_arrow_row N/A 1.7 ms N/A
🆕 Simulation list_i64_vortex N/A 2.2 ms N/A
🆕 Simulation list_utf8_arrow_row N/A 3.4 ms N/A
🆕 Simulation list_utf8_vortex N/A 4.1 ms N/A

Comparing ngates/vortex-row-dtype-support (41da939) with develop (332ae0b)

Open in CodSpeed

Footnotes

  1. 54 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
gatesn force-pushed the ngates/vortex-row-dtype-support branch from 95f3eb6 to 41da939 Compare August 23, 2026 00:52
@gatesn gatesn added the changelog/feature A new feature label Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant