Add broker config to enable streaming group-by by default for a cluster#18510
Open
yashmayya wants to merge 1 commit into
Open
Add broker config to enable streaming group-by by default for a cluster#18510yashmayya wants to merge 1 commit into
yashmayya wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18510 +/- ##
============================================
+ Coverage 63.66% 63.69% +0.02%
Complexity 1684 1684
============================================
Files 3266 3266
Lines 199903 199934 +31
Branches 31061 31069 +8
============================================
+ Hits 127272 127349 +77
+ Misses 62466 62423 -43
+ Partials 10165 10162 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #18035. Adds a broker-level config to opt a cluster into the MSE streaming group-by leaf-stage operator by default, while leaving per-query control to operators and users.
pinot.broker.mse.streaming.group.by.flush.threshold(default0, disabled).streamingGroupByFlushThresholdquery option for MSE queries that don't already specify it.SET streamingGroupByFlushThreshold = Nalways wins viaputIfAbsent, includingSET streamingGroupByFlushThreshold = 0to disable streaming group-by for a specific query when the cluster default is non-zero.EXPLAIN PLAN FOR …reflects the same options the executed query would see.Rollout / backwards compatibility
0→ no behavior change for existing deployments. Existing query option still works exactly as before.Test plan
MultiStageBrokerRequestHandlerTestcovering: injection when option absent, per-query SET=0 / SET=N overriding the broker default, no injection when config unset../mvnw spotless:apply checkstyle:check license:check -pl pinot-spi,pinot-brokerclean../mvnw -pl pinot-broker -Dtest=MultiStageBrokerRequestHandlerTest test— 4/4 passing.