Skip to content

[flink] Fix scan partitions with partial partition spec - #9225

Open
ArnavBalyan wants to merge 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/flink-scan-partitions
Open

[flink] Fix scan partitions with partial partition spec#9225
ArnavBalyan wants to merge 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/flink-scan-partitions

Conversation

@ArnavBalyan

Copy link
Copy Markdown
Member

Purpose

  • scan.partitions incorrectly returns data of all partitions, when partial partition spec is provided in the query.
  • The loader passes the incomplete specification to the partition converter. This leads to an exception which is swallowed and the partition filter is completely discarded as a result.
  • Once the partition filter is discarded, the original partial spec is not honored and all rows are returned instead of the partition values.
  • Ensure that missing partial keys are filled with partition.default-name to safely create the partition spec.

Tests

  • UT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant