Replace ToCanonical trait usages with execute#8609
Conversation
Merging this PR will degrade performance by 12.31%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | extend_from_array_zctl[(1000, 8)] |
241.1 µs | 400.8 µs | -39.84% |
| ❌ | Simulation | extend_from_array_non_zctl_overlapping[(1000, 8)] |
270 µs | 435.6 µs | -38.02% |
| ❌ | Simulation | extend_from_array_non_zctl_overlapping[(1000, 32)] |
731.8 µs | 886.1 µs | -17.41% |
| ❌ | Simulation | slice_empty_vortex |
339.4 ns | 397.8 ns | -14.66% |
| ❌ | Simulation | extend_from_array_zctl[(1000, 64)] |
1 ms | 1.2 ms | -13.42% |
| ❌ | Simulation | i32_small_overlapping |
39.8 µs | 45 µs | -11.63% |
| ❌ | Simulation | extend_from_array_zctl[(10000, 8)] |
1.9 ms | 2.2 ms | -10.44% |
| ❌ | Simulation | extend_from_array_non_zctl_overlapping[(10000, 8)] |
2.2 ms | 2.5 ms | -10.29% |
| ❌ | Simulation | compact_sliced[(4096, 90)] |
779.2 ns | 867.2 ns | -10.15% |
| ⚡ | Simulation | bench_many_codes_few_values[1024] |
528.1 µs | 364.1 µs | +45.02% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[128] |
273.6 ns | 244.4 ns | +11.93% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/nice-hypatia-g68zgt (e582b35) with develop (733ab9e)
Footnotes
-
4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
This also removes ArrayBuilder::extend_from_array and moves that logic in to ArrayRef::append_to_builder Plumb ExecutionCtx through more methods instead of creating fresh one. Signed-off-by: Robert Kruszewski <github@robertk.io>
5e975ca to
f8f6a90
Compare
| builder: &mut dyn ArrayBuilder, | ||
| ctx: &mut ExecutionCtx, | ||
| ) -> VortexResult<()> { | ||
| let Some(builder) = builder.as_any_mut().downcast_mut::<ListBuilder<u64>>() else { |
There was a problem hiding this comment.
builder_with_capacity(DType::List, ...) returns a ListViewBuilder<u64, u64>, so generic callers that allocate the canonical builder and then append a ListArray now hit this branch and error out instead of appending. For example, zip/case_when build the default list builder and append sliced branches through append_to_builder. This should either append into the canonical ListViewBuilder or fall back through execute::<ListViewArray> before appending.
There was a problem hiding this comment.
This part needs reworking. Needs to do a different downcast here
| )? | ||
| .0; | ||
|
|
||
| let len = u.int_in_range(0..=2048)?; |
There was a problem hiding this comment.
len can now be as high as 2048, but random_strictly_sorted_ends still randomly chooses PType::U8 and later does u8::try_from(e).vortex_expect(...) after forcing the last end toward this target. Ordinary fuzzer input can therefore panic during Arbitrary instead of returning arbitrary::Error. Please choose an end type that can represent the generated max end, cap the target by the chosen type, or return an error instead of vortex_expect.
375f17b to
048b19b
Compare
`builder_with_capacity` produces a `ListViewBuilder` for `DType::List`, but `List::append_to_builder` downcast the builder to `ListBuilder<u64>` and bailed otherwise — so executing a `List`-encoded array into the canonical builder (the one `builder_with_capacity` returns) failed. `ListView::append_to_builder` had the mirror-image assumption, only accepting a `ListViewBuilder<u64, u64>`. Add two macros that enumerate the possible list builder offset/size integer types and try downcasting to each, rather than assuming the types produced by `builder_with_capacity`: - `match_each_list_builder!` enumerates `ListBuilder<O>`. - `match_each_listview_builder!` enumerates `ListViewBuilder<O, S>`. `List::append_to_builder` tries `ListBuilder` (its non-canonical, directly-supplied builder) and otherwise canonicalizes and dispatches, routing the canonical `ListViewBuilder` case through `ListView::append_to_builder`, which tries `ListViewBuilder`. Either encoding works with any unsigned offset/size types. Other encodings that yield list dtypes (constant, dict, chunked, ...) already route here through the canonical-dispatch fallback. Signed-off-by: Robert <robert@spiraldb.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mkrgj5SuJpaKBXK9jydtSc
048b19b to
947a966
Compare
Rebased onto develop and made standalone (no longer stacked on #8609). Instead of deprecating the hidden global session, expose it as `vortex_array::legacy_session()` and add it to the central `clippy.toml` `disallowed-methods` list (denied via `clippy::all`). New call sites are now rejected by clippy; every current invocation is explicitly allowlisted with `#[allow(clippy::disallowed_methods)]` on its enclosing function. - `LEGACY_SESSION` static -> `legacy_session() -> &'static VortexSession` (the `LazyLock` is now a private static inside the accessor). - All call sites renamed and allowlisted across vortex-array, the encodings, vortex-ffi, vortex-cuda, vortex-cxx, vortex-layout, vortex-python and vortex-web. develop has more call sites than #8609 (incl. test/builder code and crates #8609 had refactored away), all covered. Verified with: - cargo clippy -p vortex-array -p vortex-alp -p vortex-runend -p vortex-fsst -p vortex-fastlanes -p vortex-zstd -p vortex-ffi -p vortex-layout -p vortex-web-wasm -p vortex-cxx -p vortex-python --all-targets --all-features (0 disallowed-method hits, 0 warnings) - cargo +nightly fmt --check (touched crates) vortex-cuda could not be compiled here (CUDA toolchain unavailable); its two call sites are allowlisted following the same pattern. Signed-off-by: Robert Kruszewski <robert@spiraldb.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018E5DywGCpq26ndoMHd1vyg
This also removes ArrayBuilder::extend_from_array and moves that logic
in to ArrayRef::append_to_builder
Plumb ExecutionCtx through more methods instead of creating fresh one.
Fixes: #3235
Signed-off-by: Robert Kruszewski github@robertk.io