fix: stabilize conjunct selectivity - #9543
Conversation
Polar Signals Profiling ResultsLatest Run
Previous Runs (15)
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (0.984x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.996x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -43.9% overall, 0↑ 9↓)
Totals:
|
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.018x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.013x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.874x ✅, 2↑ 0↓)
duckdb / parquet / ns (1.009x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -46.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: Clickbench Sorted on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.937x ➖, 1↑ 1↓)
datafusion / parquet / ns (0.995x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.039x ➖, 0↑ 2↓)
duckdb / parquet / ns (1.019x ➖, 0↑ 0↓)
File Size Changes (201 files changed, -42.8% overall, 52↑ 149↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.992x ➖, 1↑ 1↓)
datafusion / parquet / ns (0.995x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.017x ➖, 0↑ 6↓)
duckdb / parquet / ns (0.996x ➖, 6↑ 6↓)
File Size Changes (25 files changed, -43.5% overall, 0↑ 25↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.991x ➖, 1↑ 0↓)
File Size Changes (9 files changed, -44.0% overall, 0↑ 9↓)
Totals:
|
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.984x ➖, 1↑ 1↓)
datafusion / parquet / ns (1.067x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (0.985x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.040x ➖, 4↑ 1↓)
|
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.998x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.992x ➖, 2↑ 0↓)
duckdb / parquet / ns (1.007x ➖, 2↑ 1↓)
File Size Changes (101 files changed, -39.2% overall, 0↑ 101↓)
Totals:
|
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (1.006x ➖, 2↑ 3↓)
duckdb / parquet / ns (0.998x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -32.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.937x ➖, 1↑ 0↓)
datafusion / parquet / ns (0.963x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.032x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.961x ➖, 0↑ 0↓)
|
Signed-off-by: Joseph Isaacs <joe.isaacs@live.co.uk>
95e1ae8 to
fa57791
Compare
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (0.963x ➖, 1↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.998x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (1.007x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.992x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.987x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (0.997x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.992x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.994x ➖, 0↑ 1↓)
datafusion / vortex-compact / ns (1.006x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.882x ✅, 2↑ 0↓)
duckdb / vortex-compact / ns (1.051x ➖, 0↑ 1↓)
duckdb / parquet / ns (1.007x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: Clickbench Sorted on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.925x ➖, 1↑ 1↓)
datafusion / vortex-compact / ns (0.996x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.979x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (1.028x ➖, 0↑ 2↓)
duckdb / vortex-compact / ns (1.015x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.993x ➖, 1↑ 0↓)
File Size Changes (200 files changed, -0.0% overall, 100↑ 100↓)
Totals:
|
Benchmarks: String Encoding 📖Commits: PR vortex / vortex-file-compressed / ms (0.993x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / % (1.000x ➖, 0↑ 0↓)
|
Benchmarks: Appian on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-compact / ns (1.004x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.997x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.002x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.992x ➖, 0↑ 0↓)
File Size Changes (10 files changed, -63.8% overall, 0↑ 10↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.001x ➖, 0↑ 2↓)
datafusion / vortex-compact / ns (0.999x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.999x ➖, 1↑ 1↓)
duckdb / vortex-file-compressed / ns (1.010x ➖, 0↑ 4↓)
duckdb / vortex-compact / ns (1.001x ➖, 0↑ 4↓)
duckdb / parquet / ns (0.998x ➖, 5↑ 7↓)
No file size changes detected. |
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.026x ➖, 0↑ 1↓)
datafusion / vortex-compact / ns (0.864x ➖, 2↑ 0↓)
datafusion / parquet / ns (0.889x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (0.979x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.009x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.284x ➖, 1↑ 2↓)
|
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (1.008x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.994x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.981x ➖, 1↑ 0↓)
No file size changes detected. |
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (1.011x ➖, 1↑ 3↓)
duckdb / vortex-compact / ns (1.019x ➖, 1↑ 2↓)
duckdb / parquet / ns (1.001x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.004x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (1.004x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.006x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.005x ➖, 1↑ 2↓)
duckdb / vortex-compact / ns (0.994x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.008x ➖, 2↑ 4↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.941x ➖, 2↑ 1↓)
datafusion / vortex-compact / ns (0.988x ➖, 1↑ 2↓)
datafusion / parquet / ns (0.975x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (0.978x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.034x ➖, 0↑ 2↓)
duckdb / parquet / ns (0.956x ➖, 0↑ 0↓)
|
Benchmarks: TPC-H SF=10 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-compact / ns (0.929x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.960x ➖, 2↑ 1↓)
duckdb / vortex-compact / ns (0.978x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.964x ➖, 0↑ 0↓)
|
Benchmarks: Random Access 📖Commits: PR How to read Verdict and Engines
random-access / vortex-file-compressed / ns (1.010x ➖, 0↑ 0↓)
random-access / parquet / ns (1.004x ➖, 0↑ 0↓)
random-access / lance / ns (0.998x ➖, 0↑ 0↓)
|
Benchmarks: Compression 📖Commits: PR vortex / vortex-file-compressed / ns (0.965x ➖, 2↑ 0↓)
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / ratio (0.980x ➖, 2↑ 0↓)
vortex / parquet / ns (0.995x ➖, 0↑ 0↓)
vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)
|
| let conjunct_mask = reader | ||
| .filter_evaluation(&row_range, conjunct, MaskFuture::ready(mask))? | ||
| .await?; | ||
| filter.report_selectivity(idx, conjunct_mask.density()); |
There was a problem hiding this comment.
the density method is probably never used after this
Summary
Follow-up to #9383.