Core: Handle UnknownType in SingleValueParser - #17672
Open
Abhishek-Gawande wants to merge 1 commit into
Open
Conversation
anoopj
reviewed
Aug 17, 2026
anoopj
left a comment
anoopj
left a comment
Member
There was a problem hiding this comment.
Thanks @Abhishek-Gawande for the PR. I have a spec related question.
| generator.writeEndObject(); | ||
| break; | ||
| case UNKNOWN: | ||
| generator.writeString(String.valueOf(defaultValue)); |
Member
There was a problem hiding this comment.
I thought the spec treats unknown as null. Should this be generator.writeNull()?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #17324
What changes were proposed in this pull request?
This PR adds support for
TypeID.UNKNOWNinSingleValueParser, fixing an issue where the REST catalog's/planendpoint can fail with a truncatedHTTP 200 OKresponse.Why are the changes needed?
When a partition field references a column that is subsequently dropped from the table schema,
PartitionSpec.resultType()falls back toTypes.UnknownType.get().However, when
ContentFileParserserializes the data file's partition values,SingleValueParser.toJson()does not handleUNKNOWNand throws anUnsupportedOperationException.This can occur while the JSON response is being streamed, resulting in a partially written response when serialization fails.
How was this patch tested?
TestSingleValueParser.unknownTypeRoundTripto verify thatUnknownTypecorrectly handles nulls, integer values serialized as strings, and string values.TestContentFileParser.partitionSerializationWithDroppedSourceColumnto reproduce the issue by simulating a dropped source column, re-binding the historical partition spec, and verifying that the partition data can be serialized and deserialized successfully../gradlew :iceberg-core:test../gradlew spotlessApply.