[VL][TEST] Repro for Delta stats ClassCastException (DO NOT MERGE)#12293
Closed
felipepessoto wants to merge 1 commit into
Closed
[VL][TEST] Repro for Delta stats ClassCastException (DO NOT MERGE)#12293felipepessoto wants to merge 1 commit into
felipepessoto wants to merge 1 commit into
Conversation
This branch contains only the regression test from the companion fix PR, without the fix, to demonstrate that the test fails on current main and is therefore a valid reproduction. The test writes a Delta table whose per-file min/max statistics over a TIMESTAMP_NTZ column cannot be offloaded to Velox. Without the fix the write task crashes while collecting statistics with: java.lang.ClassCastException: org.apache.spark.sql.execution.ProjectExec cannot be cast to org.apache.gluten.execution.WholeStageTransformer The actual fix lives in the companion PR; this PR is for demonstration only and must not be merged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
felipepessoto
force-pushed
the
test-only-delta-stats-classcast
branch
from
42348bb to
db89d71
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
This PR is for demonstration only and must not be merged. The fix lives in #12292.
It contains only the regression test from #12292, without the fix, to prove the test is a valid reproduction: it fails on current
mainand passes once the fix is applied.GlutenDeltaStatsSuitewrites a Delta table whose per-file min/max statistics over aTIMESTAMP_NTZcolumn cannot be offloaded to Velox. Without the fix,GlutenDeltaJobStatsTrackerunconditionally casts the statistics projection to aWholeStageTransformer, so the write task crashes with:How was this patch tested?
This is the test. CI on this PR is expected to FAIL (red) at
GlutenDeltaStatsSuite, demonstrating the bug on currentmain. The companion fix PR #12292 runs the same test green.Locally verified (Spark 3.5, Scala 2.12): without the fix the suite fails with the ClassCastException above (
Tests: succeeded 0, failed 1); with the fix it passes (succeeded 1, failed 0).Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI (claude-opus-4.8)