[Feature] `percentfield` and `showperc` with `top` and `rare` commands. by AjimelecGonzalez · Pull Request #5642 · opensearch-project/sql

AjimelecGonzalez · 2026-07-21T21:44:06Z

Description

Add percentfield and showperc options to the top and rare PPL commands. When showperc=true is specified, a percent column is added to the output showing each value's percentage of the total count within its partition (group-by clause). The percentfield option allows customizing the name of the percentage column (default: percent).

The percentage is computed as ROUND(100.0 * count / SUM(count) OVER (PARTITION BY group_fields), 6) and is rounded to 6 decimal places. When used with a by clause, percentages are calculated within each partition (summing to 100% per group).

Syntax

source=index | top [percentfield='<name>'] showperc=true field_name [by group_field]
source=index | rare [percentfield='<name>'] showperc=true field_name [by group_field]

`Top` Example

source=otellogs | top showperc=true severityText

Returns:

severityText	count	percent
ERROR	7	35.0
INFO	6	30.0
WARN	4	20.0
DEBUG	3	15.0

`Rare` Example

source=otellogs | rare showperc=true severityText

Returns:

severityText	count	percent
DEBUG	3	15.0
WARN	4	20.0
INFO	6	30.0
ERROR	7	35.0

Custom Percentage Field Name Example

source=otellogs | top percentfield='pct' showperc=true severityText

Returns:

severityText	count	pct
ERROR	7	35.0
INFO	6	30.0
WARN	4	20.0
DEBUG	3	15.0

Related Issues

Resolves #5530

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2026-07-21T21:45:31Z

PR Reviewer Guide 🔍

(Review updated until commit `2a1e12d`)

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review Typo Fixed The old code had a typo 'countield' instead of 'countfield'. The new code fixes this typo. While this is a correction, it changes the output format of anonymized queries. If any downstream systems or tests rely on the exact string format of anonymized queries (including the typo), they may break. Verify that no code depends on the old misspelled format. "countfield='%s' showcount=%s percentfield='%s' showperc=%s usenull=%s ", countField, showCount, percentField, showPerc, useNull) : "";

github-actions · 2026-07-21T21:46:05Z

PR Code Suggestions ✨

Latest suggestions up to 2a1e12d

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Handle division by zero The percentage calculation doesn't handle the case where `totalWindowOver` is zero, which would result in division by zero. Add a null check or conditional logic to prevent runtime errors when the total count is zero. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3285-3290] RexNode hundred = context.relBuilder.literal(new BigDecimal("100.0")); RexNode countCast = context.relBuilder.cast(context.relBuilder.field(countFieldName), SqlTypeName.DOUBLE); RexNode totalCast = context.relBuilder.cast(totalWindowOver, SqlTypeName.DOUBLE); RexNode numerator = context.relBuilder.call(SqlStdOperatorTable.MULTIPLY, hundred, countCast); -RexNode percValue = context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast); +RexNode zero = context.relBuilder.literal(0.0); +RexNode isZero = context.relBuilder.equals(totalCast, zero); +RexNode percValue = context.relBuilder.call( + SqlStdOperatorTable.CASE, + isZero, + zero, + context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast)); Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a potential division by zero issue when `totalWindowOver` is zero. However, in the context of `rare`/`top` commands that aggregate counts, having a total of zero would mean no rows exist, which is unlikely in practice. The suggestion is valid but addresses an edge case that may not occur in typical usage.	Medium

Previous suggestions

Suggestions up to commit 461f758

Category	Suggestion	Impact
Possible issue	Handle division by zero in percentage The percentage calculation doesn't handle the case where `totalWindowOver` is zero, which would cause a division by zero error. Add a check or use a CASE expression to return null or zero when the total is zero. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3285-3290] RexNode hundred = context.relBuilder.literal(new BigDecimal("100.0")); RexNode countCast = context.relBuilder.cast(context.relBuilder.field(countFieldName), SqlTypeName.DOUBLE); RexNode totalCast = context.relBuilder.cast(totalWindowOver, SqlTypeName.DOUBLE); RexNode numerator = context.relBuilder.call(SqlStdOperatorTable.MULTIPLY, hundred, countCast); -RexNode percValue = context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast); +RexNode zero = context.relBuilder.literal(0.0); +RexNode isZero = context.relBuilder.equals(totalCast, zero); +RexNode percValue = context.relBuilder.call( + SqlStdOperatorTable.CASE, + isZero, + context.relBuilder.literal(0.0), + context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast)); Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a potential division by zero issue when `totalWindowOver` is zero. However, in the context of `rare`/`top` commands that aggregate counts, having a total of zero would mean no rows exist, which is unlikely in practice. The suggestion is valid but addresses an edge case that may not occur in typical usage scenarios.	Medium

Suggestions up to commit ca50dab

Category	Suggestion	Impact
Possible issue	Handle division by zero The percentage calculation doesn't handle the case where `totalWindowOver` is zero or null, which could result in division by zero errors or null propagation. Add a null check or use a CASE expression to return 0 or null when the total is zero. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3291-3296] RexNode hundred = context.relBuilder.literal(new BigDecimal("100.0")); RexNode countCast = context.relBuilder.cast(context.relBuilder.field(countFieldName), SqlTypeName.DOUBLE); RexNode totalCast = context.relBuilder.cast(totalWindowOver, SqlTypeName.DOUBLE); RexNode numerator = context.relBuilder.call(SqlStdOperatorTable.MULTIPLY, hundred, countCast); -RexNode percValue = context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast); +RexNode zero = context.relBuilder.literal(0.0); +RexNode isZero = context.relBuilder.equals(totalCast, zero); +RexNode division = context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast); +RexNode percValue = context.relBuilder.call(SqlStdOperatorTable.CASE, isZero, zero, division); Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a potential division by zero issue in the percentage calculation. While the scenario is unlikely in typical use (requires all counts to be zero), adding protection would improve robustness. However, the suggested fix using `CASE` may not be the most idiomatic approach for Calcite's `RexNode` API, and the score is moderate because this is a defensive programming improvement rather than fixing an actual bug in normal operation.	Medium

Suggestions up to commit 840ef88

Category	Suggestion	Impact
Possible issue	Handle division by zero in percentage The percentage calculation doesn't handle the case where `totalCast` could be zero, which would result in division by zero. Add a check or use a CASE expression to handle this edge case and prevent runtime errors. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3257-3262] RexNode hundred = context.relBuilder.literal(new BigDecimal("100.0")); RexNode countCast = context.relBuilder.cast(context.relBuilder.field(countFieldName), SqlTypeName.DOUBLE); RexNode totalCast = context.relBuilder.cast(totalWindowOver, SqlTypeName.DOUBLE); +RexNode zero = context.relBuilder.literal(0.0); RexNode numerator = context.relBuilder.call(SqlStdOperatorTable.MULTIPLY, hundred, countCast); -RexNode percValue = context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast); +RexNode percValue = context.relBuilder.call( + SqlStdOperatorTable.CASE, + context.relBuilder.call(SqlStdOperatorTable.EQUALS, totalCast, zero), + zero, + context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast)); Suggestion importance[1-10]: 7 __ Why: This is a valid concern about potential division by zero when `totalWindowOver` (the sum of counts) is zero. However, in the context of `rare`/`top` commands, this scenario is unlikely since these commands operate on aggregated data where at least one row must exist. Still, adding defensive programming for edge cases is good practice and would prevent potential runtime errors.	Medium
General	Add duplicate field check for count The duplicate field check should also be performed for `countFieldName` to prevent conflicts when both count and percent fields are shown. This ensures consistency with the error handling pattern and prevents potential field name collisions. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3241-3246] if (context.relBuilder.peek().getRowType().getFieldNames().contains(percentFieldName)) { throw new IllegalArgumentException( "Field `" + percentFieldName + "` already exists in the output. Consider renaming with percentfield='xyz'"); } +if (context.relBuilder.peek().getRowType().getFieldNames().contains(countFieldName)) { + throw new IllegalArgumentException( + "Field `" + + countFieldName + + "` already exists in the output. Consider renaming with countfield='xyz'"); +} Suggestion importance[1-10]: 6 __ Why: The suggestion correctly identifies that `countFieldName` should also be checked for duplicates, similar to `percentFieldName`. However, this check should occur earlier in the code flow (before the percentage calculation block), and the existing code already handles count field conflicts in the earlier aggregation step at line 3225. The suggestion has merit but may be redundant given the existing logic.	Low

Suggestions up to commit 19f05c0

Category	Suggestion	Impact
Possible issue	Handle division by zero scenario Division by zero can occur when the total count is zero (empty partition). Add a null check or use a CASE expression to handle this edge case and prevent potential runtime errors or NaN values in the output. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3291-3296] RexNode hundred = context.relBuilder.literal(new BigDecimal("100.0")); RexNode countCast = context.relBuilder.cast(context.relBuilder.field(countFieldName), SqlTypeName.DOUBLE); RexNode totalCast = context.relBuilder.cast(totalWindowOver, SqlTypeName.DOUBLE); RexNode numerator = context.relBuilder.call(SqlStdOperatorTable.MULTIPLY, hundred, countCast); -RexNode percValue = context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast); +RexNode zero = context.relBuilder.literal(0.0); +RexNode isZero = context.relBuilder.equals(totalCast, zero); +RexNode percValue = context.relBuilder.call( + SqlStdOperatorTable.CASE, + isZero, + zero, + context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast)); Suggestion importance[1-10]: 7 __ Why: This is a valid concern about potential division by zero when `totalWindowOver` is zero (empty partition). The suggested CASE expression would prevent NaN or error values. However, in practice, the total count from `SUM(COUNT(*))` over a partition should never be zero if there are rows in that partition, making this an edge case that may not occur in normal operation.	Medium
General	Add duplicate field check for count The duplicate field check should also be performed for `countFieldName` to prevent conflicts when both count and percent fields are shown. This ensures consistency with the error handling pattern and prevents potential field name collisions. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3275-3280] if (context.relBuilder.peek().getRowType().getFieldNames().contains(percentFieldName)) { throw new IllegalArgumentException( "Field `" + percentFieldName + "` already exists in the output. Consider renaming with percentfield='xyz'"); } +if (context.relBuilder.peek().getRowType().getFieldNames().contains(countFieldName)) { + throw new IllegalArgumentException( + "Field `" + + countFieldName + + "` already exists in the output. Consider renaming with countfield='xyz'"); +} Suggestion importance[1-10]: 5 __ Why: The suggestion correctly identifies that `countFieldName` should also be checked for duplicates, similar to `percentFieldName`. However, the `countFieldName` field is added earlier in the code (before line 3269), so this check would need to be placed before that point to be effective. The suggestion's placement after the percent field check is logically inconsistent with the code flow.	Low

Suggestions up to commit 840ef88

Category	Suggestion	Impact
Possible issue	Handle division by zero in percentage Division by zero can occur when `totalWindowOver` is zero (empty partition or all null values). This will cause runtime errors or produce NaN/Infinity values in the percentage calculation. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3257-3262] RexNode hundred = context.relBuilder.literal(new BigDecimal("100.0")); RexNode countCast = context.relBuilder.cast(context.relBuilder.field(countFieldName), SqlTypeName.DOUBLE); RexNode totalCast = context.relBuilder.cast(totalWindowOver, SqlTypeName.DOUBLE); RexNode numerator = context.relBuilder.call(SqlStdOperatorTable.MULTIPLY, hundred, countCast); -RexNode percValue = context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast); +RexNode zero = context.relBuilder.literal(0.0); +RexNode percValue = context.relBuilder.call( + SqlStdOperatorTable.CASE, + context.relBuilder.call(SqlStdOperatorTable.EQUALS, totalCast, zero), + zero, + context.relBuilder.call(SqlStdOperatorTable.DIVIDE, numerator, totalCast)); Suggestion importance[1-10]: 8 __ Why: This identifies a legitimate edge case where division by zero could occur when `totalWindowOver` is zero (empty partition). The suggestion provides a proper CASE statement to handle this scenario, preventing potential runtime errors or NaN/Infinity values in percentage calculations.	Medium
Possible issue	Prevent percentfield and countfield name collision The duplicate field check should also verify against `countFieldName` to prevent conflicts when both fields are shown. If `percentFieldName` equals `countFieldName`, this creates ambiguity in the output schema. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [3241-3246] if (context.relBuilder.peek().getRowType().getFieldNames().contains(percentFieldName)) { throw new IllegalArgumentException( "Field `" + percentFieldName + "` already exists in the output. Consider renaming with percentfield='xyz'"); } +if (percentFieldName.equals(countFieldName)) { + throw new IllegalArgumentException( + "percentfield and countfield cannot have the same name: `" + percentFieldName + "`"); +} Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a potential issue where `percentFieldName` and `countFieldName` could have the same value, causing schema ambiguity. However, this is a validation check that asks users to verify their input rather than fixing a critical bug, so it should not receive a score above 7.	Medium

github-actions · 2026-07-21T21:58:53Z

Persistent review updated to latest commit ca7e450

github-actions · 2026-07-21T22:07:18Z

Persistent review updated to latest commit dae1ecd

github-actions · 2026-07-22T16:09:28Z

Persistent review updated to latest commit 0b5b6ed

github-actions · 2026-07-22T20:52:37Z

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit 0039627.

Path	Line	Severity	Description
core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java	3237	low	Conflict-detection check tests for misspelled field name 'precent' instead of 'percent', so the guard against duplicate output fields will never fire. This renders the protective error message dead code, meaning a user with a pre-existing 'percent' field will get a silently incorrect result rather than the intended error. Appears to be a typo rather than intentional, but worth flagging as an anomaly.

The table above displays the top 10 most important findings.

Total: 1 | Critical: 0 | High: 0 | Medium: 0 | Low: 1

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

github-actions · 2026-07-22T20:53:27Z

Persistent review updated to latest commit 0039627

github-actions · 2026-07-22T20:56:15Z

Persistent review updated to latest commit b576e5a

github-actions · 2026-07-24T18:55:23Z

Persistent review updated to latest commit be841df

github-actions · 2026-07-28T21:34:49Z

Persistent review updated to latest commit 84c6094

github-actions · 2026-07-29T16:11:59Z

Persistent review updated to latest commit 3bea92e

github-actions · 2026-07-30T18:16:23Z

Persistent review updated to latest commit 840ef88

github-actions · 2026-07-30T18:29:34Z

Persistent review updated to latest commit 19f05c0

github-actions · 2026-07-30T18:38:40Z

Persistent review updated to latest commit 840ef88

github-actions · 2026-07-30T18:51:10Z

Persistent review updated to latest commit ca50dab

github-actions · 2026-08-03T18:25:24Z

Persistent review updated to latest commit 461f758

* Clean docs * Fix CalcitePPLRareTopNTest showperc tests * Calculate percentages before filtering * Added percentfield and changed decimal places from 2 to 6 * Update test errors, SUM to CHECKED_LONG_SUM * Remove percent field check Signed-off-by: Ajimelec Gonzalez <ajimelec@amazon.com>

github-actions · 2026-08-03T18:30:26Z

Persistent review updated to latest commit 2a1e12d

AjimelecGonzalez requested review from LantaoJin, RyanL1997, Swiddis, acarbonetto, ahkcs, anirudha, dai-chen, joshuali925, mengweieric, noCharger, penghuo, ps48, qianheng-aws, songkant-aws, vamsimanohar, ykmr1224 and yuancu as code owners July 21, 2026 21:44

AjimelecGonzalez force-pushed the feature/showperc branch from 54ab043 to ca7e450 Compare July 21, 2026 21:57

AjimelecGonzalez force-pushed the feature/showperc branch from ca7e450 to dae1ecd Compare July 21, 2026 22:06

AjimelecGonzalez force-pushed the feature/showperc branch from dae1ecd to 0b5b6ed Compare July 22, 2026 16:08

ahkcs added the feature label Jul 22, 2026

ahkcs reviewed Jul 22, 2026

View reviewed changes

AjimelecGonzalez force-pushed the feature/showperc branch from 0b5b6ed to 0039627 Compare July 22, 2026 20:51

AjimelecGonzalez force-pushed the feature/showperc branch from 0039627 to b576e5a Compare July 22, 2026 20:54

ahkcs approved these changes Jul 22, 2026

View reviewed changes

AjimelecGonzalez force-pushed the feature/showperc branch from b576e5a to be841df Compare July 24, 2026 18:53

AjimelecGonzalez force-pushed the feature/showperc branch from be841df to 84c6094 Compare July 28, 2026 21:33

AjimelecGonzalez force-pushed the feature/showperc branch from 84c6094 to 3bea92e Compare July 29, 2026 16:10

AjimelecGonzalez changed the title ~~[Feature] showperc with top and rare commands.~~ [Feature] showperc and percentfield with top and rare commands. Jul 30, 2026

AjimelecGonzalez changed the title ~~[Feature] showperc and percentfield with top and rare commands.~~ [Feature] percentfield and showperc with top and rare commands. Jul 30, 2026

AjimelecGonzalez force-pushed the feature/showperc branch from 3bea92e to 840ef88 Compare July 30, 2026 18:14

AjimelecGonzalez force-pushed the feature/showperc branch from 19f05c0 to 840ef88 Compare July 30, 2026 18:36

AjimelecGonzalez force-pushed the feature/showperc branch from 840ef88 to ca50dab Compare July 30, 2026 18:49

Swiddis reviewed Jul 30, 2026

View reviewed changes

Comment thread core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java Outdated

Comment thread core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java

AjimelecGonzalez force-pushed the feature/showperc branch from ca50dab to 461f758 Compare August 3, 2026 18:23

AjimelecGonzalez force-pushed the feature/showperc branch from 461f758 to 2a1e12d Compare August 3, 2026 18:29

Swiddis merged commit 2d1256f into opensearch-project:main Aug 4, 2026
40 checks passed

Uh oh!

Conversation

AjimelecGonzalez commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Syntax

Top Example

Rare Example

Custom Percentage Field Name Example

Related Issues

Check List

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Reviewer Guide 🔍

(Review updated until commit 2a1e12d)

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Suggestions ✨

Previous suggestions

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 22, 2026

PR Code Analyzer ❗

Uh oh!

github-actions Bot commented Jul 22, 2026

Uh oh!

github-actions Bot commented Jul 22, 2026

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 29, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AjimelecGonzalez commented Jul 21, 2026 •

edited

Loading

`Top` Example

`Rare` Example

github-actions Bot commented Jul 21, 2026 •

edited

Loading

(Review updated until commit `2a1e12d`)

github-actions Bot commented Jul 21, 2026 •

edited

Loading