Skip to content

Spark: Reuse core TestTables in Spark tests - #17689

Open
charliec05 wants to merge 1 commit into
apache:mainfrom
charliec05:agent/spark-reuse-core-test-tables
Open

Spark: Reuse core TestTables in Spark tests#17689
charliec05 wants to merge 1 commit into
apache:mainfrom
charliec05:agent/spark-reuse-core-test-tables

Conversation

@charliec05

Copy link
Copy Markdown

Summary

  • remove the duplicated TestTables helper from Spark 3.5, 4.0, and 4.1 tests
  • migrate the affected Spark tests to the core org.apache.iceberg.TestTables helper
  • load catalog test tables using their metadata location and commit projection-test schema changes through TableOperations

Testing

  • ./gradlew :iceberg-spark:iceberg-spark-4.1_2.13:test --tests org.apache.iceberg.spark.source.TestBaseReader --tests org.apache.iceberg.spark.source.TestSparkCatalog --tests org.apache.iceberg.spark.source.TestSparkMetadataColumns --tests org.apache.iceberg.spark.source.TestSparkReadProjection
  • ./gradlew -DsparkVersions=4.0 :iceberg-spark:iceberg-spark-4.0_2.13:test --tests org.apache.iceberg.spark.source.TestBaseReader --tests org.apache.iceberg.spark.source.TestSparkCatalog --tests org.apache.iceberg.spark.source.TestSparkMetadataColumns --tests org.apache.iceberg.spark.source.TestSparkReadProjection
  • ./gradlew -DsparkVersions=3.5 :iceberg-spark:iceberg-spark-3.5_2.12:test --tests org.apache.iceberg.spark.source.TestBaseReader --tests org.apache.iceberg.spark.source.TestSparkCatalog --tests org.apache.iceberg.spark.source.TestSparkMetadataColumns --tests org.apache.iceberg.spark.source.TestSparkReadProjection
  • ./gradlew -DsparkVersions=3.5,4.0,4.1 spotlessCheck

Closes #17508


AI Disclosure

@charliec05
charliec05 marked this pull request as ready for review August 17, 2026 06:56
@uros-b

uros-b commented Aug 17, 2026

Copy link
Copy Markdown
Member

LGTM, thank you @charliec05!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spark: Remove duplicated TestTables and use core test helper

2 participants