Add inverted-index-based distinct operator with runtime cost heuristic by xiangfu0 · Pull Request #17872 · apache/pinot

xiangfu0 · 2026-03-13T08:27:32Z

Summary

Adds InvertedIndexDistinctOperator — a new execution path for single-column DISTINCT queries that leverages dictionary + inverted index to avoid the scan/projection pipeline entirely. Enabled via the existing useIndexBasedDistinctOperator=true query option (shared with JsonIndexDistinctOperator).

Sample queries

-- Basic distinct with inverted index optimization
SELECT DISTINCT city FROM myTable OPTION(useIndexBasedDistinctOperator=true)

-- With filter
SELECT DISTINCT city FROM myTable WHERE country = 'US' OPTION(useIndexBasedDistinctOperator=true)

-- With ORDER BY and LIMIT (exploits early termination — see benchmark below)
SELECT DISTINCT city FROM myTable WHERE country = 'US' ORDER BY city LIMIT 100 OPTION(useIndexBasedDistinctOperator=true)

-- Override cost ratio to force inverted index path (low ratio) or scan path (high ratio)
SELECT DISTINCT city FROM myTable WHERE country = 'US' OPTION(useIndexBasedDistinctOperator=true, invertedIndexDistinctCostRatio=1)

Three execution paths, chosen automatically at runtime:

Path	When	Complexity
Sorted index	Column has a sorted forward index	O(cardinality + filteredDocs) merge-iteration
Bitmap inverted index	Cost heuristic favors it (`dictCard × costRatio ≤ filteredDocs`)	O(cardinality) bitmap `intersects()` checks
Scan fallback	Heuristic favors scanning	O(filteredDocs) via ProjectOperator + DistinctExecutor

Key design decisions

intersects() over and().isEmpty(): Short-circuits on the first common element, avoids full bitmap allocation.
Per-cardinality cost ratio map eliminates wrong-path choices across the full benchmark matrix:
- dictCard ≤ 1K → costRatio 30
- dictCard ≤ 10K → costRatio 10
- dictCard > 10K → costRatio 6
Sorted index path uses PeekableIntIterator.advanceIfNeeded() to merge-iterate the filter bitmap against contiguous doc ranges — no bitmap intersection needed.
ORDER BY + LIMIT early termination: Since dictIds are in sorted value order, ORDER BY col ASC LIMIT N only needs the first N matching dictIds (forward iteration), and ORDER BY col DESC LIMIT N iterates backward. This avoids scanning all dictionary entries, yielding 10x–100,000x speedups on LIMIT queries.
Filter bitmap reuse in scan fallback: buildFilteredDocIds() materializes the filter bitmap upfront. When the scan fallback is chosen, the bitmap is wrapped in a BitmapBasedFilterOperator and passed to the ProjectPlanNode, avoiding redundant filter re-evaluation.
Sorted dictionary gate (isSorted() check in DistinctPlanNode): Excludes mutable/consuming segments whose unsorted dictionaries would break DictIdDistinctTable's assumption that dictId order = value order (required for correct ORDER BY pruning).
Multi-value columns work naturally: each dictionary value's inverted index bitmap already maps to all documents containing that value.
All data types supported: INT, LONG, FLOAT, DOUBLE, BIG_DECIMAL, STRING, BYTES.
Null handling: When nullHandlingEnabled, excludes null placeholder values from dictionary iteration and checks the null value vector separately against the filter bitmap to include null in results.
DictId-based collection: Iterates using DictIdDistinctTable (integer dictIds with dictionary-order comparator for ORDER BY), then converts to typed values once at the end — same pattern as DictionaryBasedSingleColumnDistinctExecutor.

Query options

useIndexBasedDistinctOperator=true — enables the operator (opt-in, shared with JsonIndexDistinctOperator)
invertedIndexDistinctCostRatio=N — overrides the per-cardinality default for the bitmap path heuristic

Changes

File	Change
`InvertedIndexDistinctOperator.java`	New operator with sorted/bitmap/scan paths, cost heuristic, null handling, ORDER BY + LIMIT early termination with directional iteration
`DistinctPlanNode.java`	Constructs new operator when eligible (single column, sorted dictionary + inverted index, opt-in)
`DictIdDistinctTable.java`	New distinct table collecting dictIds, converts to typed values at the end
`QueryOptionsUtils.java`	Parse `invertedIndexDistinctCostRatio` query option
`CommonConstants.java`	New `invertedIndexDistinctCostRatio` query option key
`BenchmarkInvertedIndexDistinct.java`	JMH benchmark for 3 execution paths + 3 LIMIT early termination variants

JMH benchmark results

Setup: 1M docs, 4 cardinalities × 5 selectivities × 3 paths, @Fork(value=2, warmups=0), @Warmup(iterations=2), @Measurement(iterations=3), JDK 17

Sorted index path (μs/op, lower is better)

dictCard	0.1% (1K docs)	1% (10K)	10% (100K)	50% (500K)	100% (1M)
100	1.3	1.6	1.2	2.3	1.2
1K	7.3	13.8	10.5	9.3	8.8
10K	44.3	83.3	97.4	80.9	76.2
100K	227.7	299.4	840.9	746.8	643.3

Bitmap inverted index path (μs/op)

dictCard	0.1% (1K docs)	1% (10K)	10% (100K)	50% (500K)	100% (1M)
100	37.9	17.3	1.5	0.95	0.99
1K	4,148	765	26.8	15.3	12.6
10K	13,529	13,966	370	190	127
100K	26,026	58,851	12,446	5,947	3,856

Scan path (μs/op)

dictCard	0.1% (1K docs)	1% (10K)	10% (100K)	50% (500K)	100% (1M)
100	9.5	123	2,310	10,972	19,931
1K	17.2	247	3,121	15,793	30,229
10K	19.5	371	784	2,418	4,281
100K	19.5	597	1,507	4,220	7,684

ORDER BY + LIMIT 10 early termination (μs/op)

Exploits dictId order = value order: ASC iterates forward, DESC iterates backward, both stop after finding 10 matching values.

dictCard	filter	Inverted (no LIMIT)	LIMIT ASC	Speedup	LIMIT DESC	Speedup
100	1%	18.0	1.57	11x	1.48	12x
100	100%	0.95	0.021	45x	0.021	45x
1K	1%	749	2.88	260x	5.64	133x
1K	100%	12.8	0.021	610x	0.022	582x
10K	1%	23,200	17.2	1,349x	11.2	2,071x
10K	100%	132	0.022	6,000x	0.022	6,000x
100K	1%	60,965	32.0	1,907x	29.7	2,055x
100K	10%	10,700	0.64	16,641x	0.42	25,783x
100K	100%	4,007	0.037	108,297x	0.034	117,853x

dictCard	filter	Sorted (no LIMIT)	Sorted LIMIT DESC	Speedup
1K	100%	8.76	0.027	324x
10K	100%	75.5	0.028	2,696x
100K	10%	831	0.268	3,102x
100K	100%	640	0.028	22,868x

Key takeaways

Sorted index is the fastest path in almost all configurations — always chosen when a sorted column is available, with no heuristic needed.
Bitmap inverted index wins over scan when filteredDocs >> dictCard (e.g., dictCard=100 at 10%+ selectivity: 1.5 μs vs 2,310 μs = 1,540x faster).
Scan wins over bitmap inverted index when dictCard is high relative to filtered docs (e.g., dictCard=1K at 0.1%: scan=17 μs vs inverted=4,148 μs).
ORDER BY + LIMIT early termination delivers 10x–100,000x speedups by exploiting dictionary ordering to stop after finding just the needed values.
The cost heuristic crossover points align well with the configured ratios:
- dictCard=100 (ratio=30): inverted wins at 1%+ selectivity ✓
- dictCard=1K (ratio=30): inverted wins at 10%+ selectivity ✓
- dictCard=10K (ratio=10): inverted wins at 10%+ selectivity ✓
- dictCard=100K (ratio=6): inverted wins at 100% selectivity ✓

Test plan

InvertedIndexDistinctOperatorTest — 17 tests: cost heuristic boundaries, inverted vs scan correctness, MV columns, STRING columns, sorted column path, ORDER BY/LIMIT, null handling, all data types
OfflineClusterIntegrationTest.testDistinctWithInvertedIndex — integration test
JMH benchmark: BenchmarkInvertedIndexDistinct (3 paths × 20 param combos + 3 LIMIT variants)

🤖 Generated with Claude Code

codecov-commenter · 2026-03-13T09:41:50Z

Codecov Report

❌ Patch coverage is 77.89474% with 63 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.27%. Comparing base (43c8ecc) to head (cdf06eb).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
.../operator/query/InvertedIndexDistinctOperator.java	78.40%	32 Missing and 14 partials ⚠️
...core/query/distinct/table/DictIdDistinctTable.java	85.71%	1 Missing and 6 partials ⚠️
...e/pinot/common/utils/config/QueryOptionsUtils.java	53.84%	5 Missing and 1 partial ⚠️
...a/org/apache/pinot/core/plan/DistinctPlanNode.java	55.55%	0 Missing and 4 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #17872      +/-   ##
============================================
- Coverage     63.29%   63.27%   -0.02%     
- Complexity     1525     1542      +17     
============================================
  Files          3194     3197       +3     
  Lines        193645   193977     +332     
  Branches      29787    29864      +77     
============================================
+ Hits         122559   122748     +189     
- Misses        61466    61595     +129     
- Partials       9620     9634      +14

Flag	Coverage Δ
custom-integration1	`100.00% <ø> (ø)`
integration	`100.00% <ø> (ø)`
integration1	`100.00% <ø> (ø)`
integration2	`0.00% <ø> (ø)`
java-11	`63.24% <77.89%> (+0.02%)`	⬆️
java-21	`63.24% <77.89%> (+0.01%)`	⬆️
temurin	`63.27% <77.89%> (-0.02%)`	⬇️
unittests	`63.27% <77.89%> (-0.02%)`	⬇️
unittests1	`55.60% <77.89%> (+0.05%)`	⬆️
unittests2	`34.14% <0.00%> (-0.10%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

Adds a new execution path for single-column DISTINCT on dictionary + inverted-index columns by introducing InvertedIndexDistinctOperator, with a runtime heuristic to choose between inverted-index bitmap intersection vs scan-based distinct.

Changes:

Introduces InvertedIndexDistinctOperator with cost-based runtime path selection and bitmap intersects() usage.
Updates DistinctPlanNode to opt into the new operator via useInvertedIndexDistinct query option (plus cost-ratio override option).
Adds unit/integration tests and JMH benchmarks to validate correctness and performance.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java`	Adds query option keys for enabling inverted-index distinct and overriding heuristic cost ratio.
`pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java`	Adds helpers to parse new query options.
`pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java`	Wires plan construction to use `InvertedIndexDistinctOperator` when eligible and opted in.
`pinot-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java`	New operator implementing inverted-index-based distinct with scan fallback and heuristic.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctCostHeuristicTest.java`	New unit tests for heuristic behavior and path selection.
`pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java`	Adds integration coverage for DISTINCT with inverted index (with/without filter).
`pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java`	New JMH benchmark comparing inverted-index vs scan distinct paths.
`pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkBitmapIntersectionVsAnd.java`	New JMH benchmark comparing bitmap `intersects()` vs full `and()` intersection.

You can also share your feedback on Copilot code review. Take the survey.

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java

pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java

Copilot

Pull request overview

Adds an opt-in (OPTION(useInvertedIndexDistinct=true)) single-segment DISTINCT execution path that can answer single-column DISTINCT from dictionary + inverted index (or sorted forward index) without going through the scan/projection pipeline, choosing between sorted/bitmap/scan paths via a runtime heuristic.

Changes:

Introduces InvertedIndexDistinctOperator with sorted-index, bitmap-inverted-index, and scan-fallback execution paths plus a cost heuristic (and override).
Wires the new operator into DistinctPlanNode behind a query option, and adds query option parsing/constants.
Adds unit/integration tests plus a JMH benchmark for the new operator/heuristic.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
pinot-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java	New DISTINCT operator implementing sorted/bitmap/scan paths and heuristic selection.
pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java	Constructs the new operator when eligible and opted-in.
pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java	Adds parsing helpers for the new query options.
pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java	Adds query option keys for enabling/tuning inverted-index DISTINCT.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctCostHeuristicTest.java	Tests heuristic boundary/override behavior and correctness parity vs scan.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctMultiValueTest.java	Tests MV column correctness and path selection behavior.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctSortedColumnTest.java	Tests sorted-index path selection and correctness parity vs scan.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctStringTest.java	Tests STRING column correctness and path selection parity vs scan.
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java	Adds an integration test covering DISTINCT with the opt-in option.
pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java	Adds a JMH benchmark comparing the three execution paths.

You can also share your feedback on Copilot code review. Take the survey.

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java

...on-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java

Copilot

Pull request overview

Adds an opt-in execution path for single-column DISTINCT queries that can bypass scan/projection by using dictionary + inverted index (with a runtime heuristic), plus tests and benchmarks to validate/measure the new behavior.

Changes:

Introduces InvertedIndexDistinctOperator with sorted-index, bitmap-inverted-index, and scan-fallback paths plus a cost heuristic and null handling.
Wires the operator into DistinctPlanNode behind useInvertedIndexDistinct=true and adds invertedIndexDistinctCostRatio parsing + query option keys.
Adds unit/integration tests and a JMH benchmark for the new distinct execution paths.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java`	Adds new query option keys for enabling/tuning inverted-index DISTINCT.
`pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java`	Parses the new query options (`useInvertedIndexDistinct`, `invertedIndexDistinctCostRatio`).
`pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java`	Constructs `InvertedIndexDistinctOperator` when eligible and opted-in.
`pinot-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java`	New operator implementing the distinct logic and heuristic.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctCostHeuristicTest.java`	Unit tests for heuristic behavior and scan-vs-inverted correctness.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctMultiValueTest.java`	Unit tests for MV column behavior and ORDER BY / LIMIT interactions.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctSortedColumnTest.java`	Unit tests for sorted-index path selection and correctness.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctStringTest.java`	Unit tests for STRING column behavior and ordering/limits.
`pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java`	Adds an integration test covering opt-in DISTINCT with/without filter.
`pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java`	Adds a JMH benchmark comparing sorted vs bitmap-inverted vs scan paths.

You can also share your feedback on Copilot code review. Take the survey.

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

Copilot

Pull request overview

Adds an opt-in, inverted-index-driven execution path for single-column DISTINCT queries to avoid scan/projection where possible, with a runtime heuristic to choose between sorted-index, bitmap-inverted-index, and scan fallback paths.

Changes:

Introduces InvertedIndexDistinctOperator and wires it into DistinctPlanNode when useInvertedIndexDistinct=true.
Adds query options for enabling the operator and tuning the cost heuristic (invertedIndexDistinctCostRatio).
Adds unit/integration tests and a JMH benchmark covering sorted/bitmap/scan paths, MV, STRING, and null handling.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java`	Adds query option keys for enabling inverted-index distinct and setting the cost ratio.
`pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java`	Adds parsing helpers for the new query options.
`pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java`	Constructs `InvertedIndexDistinctOperator` when eligible and opted-in.
`pinot-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java`	Implements sorted-index, bitmap-inverted-index, and scan-fallback execution paths with null handling and heuristic selection.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctCostHeuristicTest.java`	Unit tests for heuristic boundaries and forced-path behavior via cost ratio override.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctMultiValueTest.java`	Unit tests for MV columns (correctness, ORDER BY, LIMIT, empty result).
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctNullHandlingTest.java`	Unit tests for null handling behavior and scan-vs-inverted equivalence.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctSortedColumnTest.java`	Unit tests for sorted-index path selection and correctness vs scan.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctStringTest.java`	Unit tests for STRING distinct via inverted index and correctness vs scan.
`pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java`	Adds an integration test exercising `DISTINCT` with `useInvertedIndexDistinct=true`.
`pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java`	Adds JMH benchmark comparing sorted vs bitmap-inverted vs scan distinct paths.

You can also share your feedback on Copilot code review. Take the survey.

pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java

Copilot

Pull request overview

Adds a new opt-in execution path for single-column DISTINCT queries that can leverage dictionary + inverted index (and a sorted-index fast path) to avoid the scan/projection pipeline, selected at runtime via a cost heuristic.

Changes:

Introduces InvertedIndexDistinctOperator with sorted-index, bitmap-inverted-index, and scan-fallback paths (plus cost heuristic and null handling).
Updates DistinctPlanNode to select the new operator when useInvertedIndexDistinct=true and the column is eligible.
Adds query option keys/parsing plus new unit/integration tests and a JMH benchmark.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pinot-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java	New distinct operator with runtime path selection and null handling.
pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java	Wires in the operator behind a query option and eligibility checks.
pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java	Adds query option parsing helpers for enable flag + heuristic override.
pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java	Adds query option keys for the new operator and heuristic override.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctCostHeuristicTest.java	Tests heuristic selection and correctness vs scan.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctMultiValueTest.java	Tests MV-column correctness and scan parity.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctNullHandlingTest.java	Tests null handling behavior and scan parity.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctSortedColumnTest.java	Tests sorted-column path selection and correctness.
pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctStringTest.java	Tests STRING-column correctness and ordering/limit behavior.
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java	Adds an integration test for DISTINCT using the opt-in operator.
pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java	Adds a JMH benchmark to compare sorted/bitmap/scan paths.

You can also share your feedback on Copilot code review. Take the survey.

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java

Copilot

Pull request overview

Adds a new single-segment execution operator that can answer single-column SELECT DISTINCT using dictionary + inverted index (and a sorted-index fast path when available), selected at runtime via a cost heuristic and gated behind a query option.

Changes:

Introduces InvertedIndexDistinctOperator with sorted-index / bitmap-inverted-index / scan-fallback paths and a runtime cost heuristic (plus filter-bitmap caching and null handling support).
Wires the operator into DistinctPlanNode behind useIndexBasedDistinctOperator=true, and adds a new tuning option invertedIndexDistinctCostRatio.
Adds unit/integration tests and a JMH benchmark covering path selection and correctness (SV/MV, ORDER BY, LIMIT, null handling).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java`	Adds query option key `invertedIndexDistinctCostRatio` and expands `useIndexBasedDistinctOperator` doc to cover both distinct operators.
`pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java`	Parses `invertedIndexDistinctCostRatio` query option.
`pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java`	Constructs `InvertedIndexDistinctOperator` when eligible and opted-in.
`pinot-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java`	New operator implementing sorted/bitmap/scan paths with heuristic selection and null handling.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctCostHeuristicTest.java`	Unit tests for heuristic boundary behavior and override.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctMultiValueTest.java`	Unit tests for MV columns (filter/order/limit/empty result) and scan parity.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctNullHandlingTest.java`	Unit tests for null inclusion/exclusion and scan parity under null handling.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctSortedColumnTest.java`	Unit tests verifying sorted-index path selection and correctness.
`pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctStringTest.java`	Unit tests for STRING columns with inverted index and scan parity.
`pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java`	Adds an integration test exercising DISTINCT on an inverted-indexed column with/without filter.
`pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java`	Adds JMH benchmark comparing sorted vs inverted-index vs scan behavior.

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java

pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java

pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctCostHeuristicTest.java

pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkInvertedIndexDistinct.java

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

Jackie-Jiang · 2026-03-23T23:29:41Z

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

+
+  // ==================== Scan Path (Fallback) ====================
+
+  private DistinctResultsBlock executeScanPath() {


Since we already have the filtered bitmap, short-circuit the project operator and directly read the filtered docs

xiangfu0 · 2026-03-24T07:27:02Z

Updated JMH Benchmark Results

Setup: 1M docs, 4 cardinalities × 5 selectivities × 3 paths, @Fork(value=2, warmups=0), @Warmup(iterations=2), @Measurement(iterations=3), JDK 17, ~34 min total

Sorted index path (μs/op, lower is better)

dictCard	0.1% (1K docs)	1% (10K)	10% (100K)	50% (500K)	100% (1M)
100	1.3	1.6	1.1	1.3	1.2
1K	7.7	13.8	10.5	9.6	9.1
10K	45.7	81.3	101	86.0	95.7
100K	243	301	888	822	646

Bitmap inverted index path (μs/op)

dictCard	0.1% (1K docs)	1% (10K)	10% (100K)	50% (500K)	100% (1M)
100	38	19	1.6	1.0	1.0
1K	4,256	767	27	16	13
10K	12,979	14,053	369	179	137
100K	35,459	61,037	10,635	5,780	3,522

Scan path (μs/op)

dictCard	0.1% (1K docs)	1% (10K)	10% (100K)	50% (500K)	100% (1M)
100	9	120	2,357	11,064	20,114
1K	17	246	3,157	15,996	30,353
10K	19	342	697	2,388	4,323
100K	19	609	1,518	4,220	7,680

Cost heuristic crossover validation

dictCard	configured ratio	heuristic threshold	actual crossover
100	30x	3,000 filtered docs	inverted wins at 1%+ (10K docs) ✓
1K	30x	30,000 filtered docs	inverted wins at 10%+ (100K docs) ✓
10K	10x	100,000 filtered docs	inverted wins at 10%+ (100K docs) ✓
100K	6x	600,000 filtered docs	inverted wins at 100% (1M docs) ✓

Key takeaways

Sorted index is the fastest path across all configurations (1–888 μs), always chosen when a sorted column is available
Bitmap inverted index dominates scan when filteredDocs >> dictCard (e.g., dictCard=100 at 10%: 1.6 μs vs 2,357 μs = 1,473x faster)
Scan dominates inverted index when dictCard is high relative to filtered docs (e.g., dictCard=1K at 0.1%: 17 μs vs 4,256 μs)
Per-cardinality cost ratios (30x / 10x / 6x) correctly steer the heuristic at all crossover points

Raw JMH output

Benchmark                                         (_dictionaryCardinality)  (_filterSelectivity)  (_numDocs)  Mode  Cnt      Score       Error  Units
BenchmarkInvertedIndexDistinct.invertedIndexPath                       100                 0.001     1000000  avgt    6     38.231 ±     1.357  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                       100                  0.01     1000000  avgt    6     18.694 ±     4.612  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                       100                   0.1     1000000  avgt    6      1.584 ±     0.168  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                       100                   0.5     1000000  avgt    6      0.983 ±     0.104  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                       100                   1.0     1000000  avgt    6      0.972 ±     0.017  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                      1000                 0.001     1000000  avgt    6   4256.134 ±   157.568  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                      1000                  0.01     1000000  avgt    6    766.973 ±    25.905  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                      1000                   0.1     1000000  avgt    6     26.504 ±     1.263  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                      1000                   0.5     1000000  avgt    6     15.891 ±     0.426  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                      1000                   1.0     1000000  avgt    6     12.598 ±     0.295  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                     10000                 0.001     1000000  avgt    6  12979.261 ±  6223.626  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                     10000                  0.01     1000000  avgt    6  14053.085 ±   417.580  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                     10000                   0.1     1000000  avgt    6    369.112 ±    22.802  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                     10000                   0.5     1000000  avgt    6    179.168 ±    14.620  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                     10000                   1.0     1000000  avgt    6    137.372 ±     6.651  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                    100000                 0.001     1000000  avgt    6  35458.867 ± 25631.306  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                    100000                  0.01     1000000  avgt    6  61036.517 ±  4329.001  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                    100000                   0.1     1000000  avgt    6  10635.409 ±   501.969  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                    100000                   0.5     1000000  avgt    6   5779.762 ±   394.506  us/op
BenchmarkInvertedIndexDistinct.invertedIndexPath                    100000                   1.0     1000000  avgt    6   3521.908 ±   160.910  us/op
BenchmarkInvertedIndexDistinct.scanPath                                100                 0.001     1000000  avgt    6      9.273 ±     0.510  us/op
BenchmarkInvertedIndexDistinct.scanPath                                100                  0.01     1000000  avgt    6    119.694 ±    19.105  us/op
BenchmarkInvertedIndexDistinct.scanPath                                100                   0.1     1000000  avgt    6   2356.700 ±   129.508  us/op
BenchmarkInvertedIndexDistinct.scanPath                                100                   0.5     1000000  avgt    6  11064.090 ±   314.943  us/op
BenchmarkInvertedIndexDistinct.scanPath                                100                   1.0     1000000  avgt    6  20114.177 ±   520.504  us/op
BenchmarkInvertedIndexDistinct.scanPath                               1000                 0.001     1000000  avgt    6     17.063 ±     0.130  us/op
BenchmarkInvertedIndexDistinct.scanPath                               1000                  0.01     1000000  avgt    6    245.697 ±    21.773  us/op
BenchmarkInvertedIndexDistinct.scanPath                               1000                   0.1     1000000  avgt    6   3156.635 ±    65.038  us/op
BenchmarkInvertedIndexDistinct.scanPath                               1000                   0.5     1000000  avgt    6  15996.288 ±  1633.042  us/op
BenchmarkInvertedIndexDistinct.scanPath                               1000                   1.0     1000000  avgt    6  30352.554 ±   535.360  us/op
BenchmarkInvertedIndexDistinct.scanPath                              10000                 0.001     1000000  avgt    6     19.081 ±     0.530  us/op
BenchmarkInvertedIndexDistinct.scanPath                              10000                  0.01     1000000  avgt    6    342.450 ±    20.136  us/op
BenchmarkInvertedIndexDistinct.scanPath                              10000                   0.1     1000000  avgt    6    697.304 ±    12.680  us/op
BenchmarkInvertedIndexDistinct.scanPath                              10000                   0.5     1000000  avgt    6   2388.036 ±    35.300  us/op
BenchmarkInvertedIndexDistinct.scanPath                              10000                   1.0     1000000  avgt    6   4323.113 ±   342.050  us/op
BenchmarkInvertedIndexDistinct.scanPath                             100000                 0.001     1000000  avgt    6     19.445 ±     2.081  us/op
BenchmarkInvertedIndexDistinct.scanPath                             100000                  0.01     1000000  avgt    6    608.963 ±    51.412  us/op
BenchmarkInvertedIndexDistinct.scanPath                             100000                   0.1     1000000  avgt    6   1517.888 ±   137.011  us/op
BenchmarkInvertedIndexDistinct.scanPath                             100000                   0.5     1000000  avgt    6   4219.561 ±   248.307  us/op
BenchmarkInvertedIndexDistinct.scanPath                             100000                   1.0     1000000  avgt    6   7680.396 ±   136.475  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                         100                 0.001     1000000  avgt    6      1.264 ±     0.144  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                         100                  0.01     1000000  avgt    6      1.629 ±     0.034  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                         100                   0.1     1000000  avgt    6      1.125 ±     0.085  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                         100                   0.5     1000000  avgt    6      1.341 ±     0.092  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                         100                   1.0     1000000  avgt    6      1.161 ±     0.042  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                        1000                 0.001     1000000  avgt    6      7.746 ±     0.986  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                        1000                  0.01     1000000  avgt    6     13.783 ±     0.340  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                        1000                   0.1     1000000  avgt    6     10.462 ±     0.610  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                        1000                   0.5     1000000  avgt    6      9.629 ±     0.501  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                        1000                   1.0     1000000  avgt    6      9.134 ±     0.771  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                       10000                 0.001     1000000  avgt    6     45.663 ±     3.165  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                       10000                  0.01     1000000  avgt    6     81.316 ±     3.230  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                       10000                   0.1     1000000  avgt    6    101.034 ±     8.342  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                       10000                   0.5     1000000  avgt    6     85.989 ±    12.125  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                       10000                   1.0     1000000  avgt    6     95.657 ±    56.279  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                      100000                 0.001     1000000  avgt    6    242.897 ±    34.804  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                      100000                  0.01     1000000  avgt    6    300.715 ±    54.706  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                      100000                   0.1     1000000  avgt    6    888.095 ±   202.339  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                      100000                   0.5     1000000  avgt    6    821.560 ±   244.944  us/op
BenchmarkInvertedIndexDistinct.sortedIndexPath                      100000                   1.0     1000000  avgt    6    646.489 ±    24.215  us/op

Adds a new execution path for single-column DISTINCT queries that leverages dictionary + inverted index to avoid the scan/projection pipeline entirely. Three paths chosen at runtime: sorted index (merge-iterate filter vs contiguous ranges), bitmap inverted index (intersects() per dictionary entry), and scan fallback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Exploit sorted dictionary ordering to enable early termination in both the inverted index and sorted index paths. For ORDER BY ASC, iterate forward and stop after finding LIMIT matching values. For ORDER BY DESC, iterate backward. This avoids scanning all dictionary entries when only the top-N values are needed. Benchmark shows 10x-100,000x speedup for LIMIT queries depending on dictionary cardinality and filter selectivity (e.g., 100K cardinality at 100% selectivity: 4,007us -> 0.037us with LIMIT 10). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Copilot · 2026-03-25T10:09:38Z

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java

+    // Process null handling: exclude null docs from filter and determine if nulls are present
+    NullFilterResult nullResult = processNullDocs(filteredDocIds);
+    ImmutableRoaringBitmap nonNullFilteredDocIds = nullResult._nonNullFilteredDocIds;
+


When null handling is enabled and the filtered docs contain nulls, this operator never reserves a slot for null while collecting dictIds (DictIdDistinctTable never calls addNull()). If the query has a LIMIT and no ORDER BY, this can fill the table to LIMIT dictIds and later cause the broker-side toResultTableWithoutOrderBy() logic to drop null (it only includes null when numValues < limit). Consider calling dictIdTable.addNull() as soon as nullResult indicates nulls are present (before addDictId loop), or otherwise ensure at most (limit-1) non-null values are collected when null is present.

Suggested change

if (nullResult._hasNull) {

// Reserve one slot in the distinct table for null so that at most (limit - 1) non-null values are collected

dictIdTable.addNull();

}

Copilot · 2026-03-25T10:09:38Z

pinot-core/src/test/java/org/apache/pinot/queries/InvertedIndexDistinctOperatorTest.java

+  @Test
+  public void testNullIncludedWithWideFilter() {
+    _activeSegment = _nullSegment;
+
+    BaseOperator<DistinctResultsBlock> op = getOperator(
+        "SELECT DISTINCT intColumn FROM testTable WHERE filterColumn >= 0 LIMIT 1000 "
+            + OPT + ", invertedIndexDistinctCostRatio=1, enableNullHandling=true)");
+    DistinctTable table = op.nextBlock().getDistinctTable();


The null-handling tests validate server-side DistinctTable contents, but they don’t exercise broker-side limiting behavior (DistinctTable#toResultTableWithoutOrderBy only includes null when numValues < limit). Please add a regression test that calls toResultTable() (or otherwise verifies final results) for a query with nulls, no ORDER BY, and LIMIT equal to the number of non-null distinct values so that null must be preserved under the limit.

xiangfu0 force-pushed the inverted-index-distinct-operator branch 3 times, most recently from 80749f8 to 8e47159 Compare March 15, 2026 09:14

xiangfu0 requested review from Jackie-Jiang, Copilot and yashmayya March 16, 2026 22:22

Copilot started reviewing on behalf of xiangfu0 March 16, 2026 22:23 View session

Copilot AI reviewed Mar 16, 2026

View reviewed changes

xiangfu0 added query Related to query processing inverted-index Related to inverted index implementation performance Related to performance optimization index Related to indexing (general) labels Mar 17, 2026

xiangfu0 force-pushed the inverted-index-distinct-operator branch 2 times, most recently from 3f376a8 to 6ed2aee Compare March 17, 2026 20:53

xiangfu0 requested a review from Copilot March 17, 2026 21:25

Copilot started reviewing on behalf of xiangfu0 March 17, 2026 21:25 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

xiangfu0 force-pushed the inverted-index-distinct-operator branch 2 times, most recently from cded79e to 0339dfb Compare March 18, 2026 08:00

xiangfu0 requested a review from Copilot March 18, 2026 08:35

Copilot started reviewing on behalf of xiangfu0 March 18, 2026 08:36 View session

Copilot AI reviewed Mar 18, 2026

View reviewed changes

xiangfu0 force-pushed the inverted-index-distinct-operator branch from 0339dfb to c200936 Compare March 18, 2026 09:12

xiangfu0 requested a review from Copilot March 18, 2026 18:40

Copilot started reviewing on behalf of xiangfu0 March 18, 2026 18:40 View session

Copilot AI reviewed Mar 18, 2026

View reviewed changes

pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java Outdated Show resolved Hide resolved

xiangfu0 force-pushed the inverted-index-distinct-operator branch 2 times, most recently from 6f54657 to 13acf80 Compare March 18, 2026 19:04

xiangfu0 requested a review from Copilot March 18, 2026 19:06

xiangfu0 requested a review from Copilot March 18, 2026 22:15

xiangfu0 force-pushed the inverted-index-distinct-operator branch from ace9d3b to 3dc88f8 Compare March 18, 2026 22:18

Copilot AI reviewed Mar 18, 2026

View reviewed changes

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java Outdated Show resolved Hide resolved

raghavyadav01 reviewed Mar 19, 2026

View reviewed changes

pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java Outdated Show resolved Hide resolved

xiangfu0 force-pushed the inverted-index-distinct-operator branch 2 times, most recently from 16c2d8d to 179e9f6 Compare March 20, 2026 01:38

xiangfu0 requested review from Copilot and raghavyadav01 March 20, 2026 01:46

Copilot started reviewing on behalf of xiangfu0 March 20, 2026 01:47 View session

Copilot AI reviewed Mar 20, 2026

View reviewed changes

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java Outdated Show resolved Hide resolved

pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java Show resolved Hide resolved

xiangfu0 force-pushed the inverted-index-distinct-operator branch from 179e9f6 to 861a9ce Compare March 20, 2026 05:13

xiangfu0 requested a review from Copilot March 20, 2026 05:15

Copilot started reviewing on behalf of xiangfu0 March 20, 2026 05:18 View session

Copilot AI reviewed Mar 20, 2026

View reviewed changes

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java Outdated Show resolved Hide resolved

...t-core/src/main/java/org/apache/pinot/core/operator/query/InvertedIndexDistinctOperator.java Outdated Show resolved Hide resolved

Jackie-Jiang reviewed Mar 20, 2026

View reviewed changes

xiangfu0 force-pushed the inverted-index-distinct-operator branch from 861a9ce to 1a07567 Compare March 21, 2026 09:36

xiangfu0 requested a review from Jackie-Jiang March 22, 2026 05:20

xiangfu0 force-pushed the inverted-index-distinct-operator branch from 1a07567 to 1734ec6 Compare March 22, 2026 05:44

Jackie-Jiang reviewed Mar 23, 2026

View reviewed changes

xiangfu0 force-pushed the inverted-index-distinct-operator branch 3 times, most recently from 1749c73 to 65a4371 Compare March 24, 2026 07:26

xiangfu0 force-pushed the inverted-index-distinct-operator branch from 65a4371 to bb6f548 Compare March 25, 2026 02:25

xiangfu0 requested review from Jackie-Jiang and Copilot March 25, 2026 10:01

Copilot started reviewing on behalf of xiangfu0 March 25, 2026 10:02 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes


		// ==================== Scan Path (Fallback) ====================

		private DistinctResultsBlock executeScanPath() {

+    if (nullResult._hasNull) {
+      // Reserve one slot in the distinct table for null so that at most (limit - 1) non-null values are collected
+      dictIdTable.addNull();
+    }

Conversation

xiangfu0 commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Sample queries

Three execution paths, chosen automatically at runtime:

Key design decisions

Query options

Changes

JMH benchmark results

Sorted index path (μs/op, lower is better)

Bitmap inverted index path (μs/op)

Scan path (μs/op)

ORDER BY + LIMIT 10 early termination (μs/op)

Key takeaways

Test plan

Uh oh!

codecov-commenter commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xiangfu0 commented Mar 13, 2026 •

edited

Loading

codecov-commenter commented Mar 13, 2026 •

edited

Loading