Skip to content

Document that PPL fetch_size limits aggregation buckets - #5671

Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom
ahkcs:docs/ppl-fetch-size-aggregation
Open

Document that PPL fetch_size limits aggregation buckets#5671
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom
ahkcs:docs/ppl-fetch-size-aggregation

Conversation

@ahkcs

@ahkcs ahkcs commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Description

fetch_size is applied to the final rows of a PPL query, so on an aggregating query it limits the number of aggregation buckets returned rather than sampling documents. The existing docs described it only as "limits the number of rows returned," which reads like DSL size — where aggregation results come back in full.

This adds a short section covering:

  • fetch_size is equivalent to appending | head <fetch_size>, so for stats/timechart/top/rare/patterns ... mode=aggregation it truncates buckets.
  • Aggregate values stay correct — unlike DSL terminate_after, which undercounts because documents are never collected. But unlike DSL size, aggregation results are truncated. It matches neither analogue exactly, which is what made the behavior easy to misread.
  • Bucket counts grow with the queried time range rather than data volume, so a fixed limit truncates unexpectedly on wide ranges.

Raised in review of opensearch-project/OpenSearch-Dashboards#12511, where this behavior caused Discover to silently drop buckets from PPL visualizations.

Issues Resolved

None; documentation gap found during review of OpenSearch-Dashboards#12511.

Check List

  • New functionality has been documented.
  • Commits are signed per the DCO using --signoff

Docs-only change. docs/user/interfaces/ is not covered by the doctest suite, so no queries here are executed as assertions; the example values were confirmed by hand against a local 3.8.0-SNAPSHOT cluster.

fetch_size applies to the final rows of the query, so on an aggregating query
it limits the number of buckets returned rather than sampling documents. The
existing docs described it only as a row limit, which reads as DSL size --
where aggregation results are returned in full.

Contrast both DSL analogues: unlike terminate_after it does not corrupt
aggregate values, and unlike size it does truncate aggregation results.

Signed-off-by: Kai Huang <ahkcs@amazon.com>
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Standardize code example formatting

The code examples have inconsistent spacing around the query and comment separator.
The first line uses multiple spaces before ->, while the second uses spaces around
(fetch_size=3) and before ->. Standardize the formatting for better readability.

docs/user/interfaces/endpoint.rst [338-339]

-source=logs | stats count() by extension                  -> 6 buckets
-source=logs | stats count() by extension  (fetch_size=3)   -> first 3 buckets only
+source=logs | stats count() by extension                   -> 6 buckets
+source=logs | stats count() by extension (fetch_size=3)    -> first 3 buckets only
Suggestion importance[1-10]: 3

__

Why: The suggestion correctly identifies minor spacing inconsistencies in the code examples. However, the impact is minimal as both formats are readable, and the improved_code still shows inconsistent spacing (extra spaces after extension vs. after (fetch_size=3)). This is a minor stylistic improvement.

Low

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant