diff --git a/docs/source/contributor-guide/adding_a_new_spark_version.md b/docs/source/contributor-guide/adding_a_new_spark_version.md index 45c9ebc15c..64f90bec5c 100644 --- a/docs/source/contributor-guide/adding_a_new_spark_version.md +++ b/docs/source/contributor-guide/adding_a_new_spark_version.md @@ -62,7 +62,8 @@ existing profile (for example `spark-4.1`) and update the version properties: - `spark.version`: the full upstream version, including any qualifier - (for example `4.2.0-preview4`). + (for example `4.3.0` for a final release, or `4.3.0-preview1` while only a + preview has been published). - `spark.version.short`: the major.minor (for example `4.2`). - `parquet.version`, `slf4j.version`, `scala.version`, `scala.binary.version`, `java.version`: align with what the new Spark @@ -317,7 +318,7 @@ issue keeps the diff small and makes regressions easy to bisect. The user guide currently uses two tiers, "Supported" and "Experimental". Comet uses "Experimental" to describe its confidence in its own integration with a Spark version, distinct from Spark's "preview" tag -(which refers to upstream release qualifiers like `4.2.0-preview4`). The +(which refers to upstream release qualifiers like `4.3.0-preview1`). The term is already established in `installation.md`, `operators.md`, and `datasources.md`, so keep using it rather than introducing a new label. diff --git a/docs/source/user-guide/latest/compatibility/spark-versions.md b/docs/source/user-guide/latest/compatibility/spark-versions.md index 0675aa83ec..090a1df85e 100644 --- a/docs/source/user-guide/latest/compatibility/spark-versions.md +++ b/docs/source/user-guide/latest/compatibility/spark-versions.md @@ -91,9 +91,9 @@ Spark 4.1.3 is supported with Java 17/21 and Scala 2.13. ## Spark 4.2 (Experimental) -Spark 4.2.0-preview4 is provided as experimental support with Java 17 and Scala 2.13. +Spark 4.2.0 is provided as experimental support with Java 17 and Scala 2.13. ```{warning} -Spark 4.2 support is experimental and targets a preview release of Spark. It is intended for early -evaluation only and should not be used in production. +Spark 4.2 support is experimental. Comet tests run in CI for this version, but the Spark SQL tests +do not yet. It is intended for early evaluation only and should not be used in production. ``` diff --git a/docs/source/user-guide/latest/installation.md b/docs/source/user-guide/latest/installation.md index 9fc350c25f..7fc4a81d89 100644 --- a/docs/source/user-guide/latest/installation.md +++ b/docs/source/user-guide/latest/installation.md @@ -62,9 +62,9 @@ Note that we do not test the full matrix of supported Java and Scala versions in Experimental support is provided for the following versions of Apache Spark and is intended for development/testing use only and should not be used in production yet. -| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark SQL Tests in CI | -| -------------- | ------------ | ------------- | ----------------- | --------------------- | -| 4.2.0-preview4 | 17 | 2.13 | No | No | +| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark SQL Tests in CI | +| ------------- | ------------ | ------------- | ----------------- | --------------------- | +| 4.2.0 | 17 | 2.13 | Yes | No | Note that Comet may not fully work with proprietary forks of Apache Spark such as the Spark versions offered by Cloud Service Providers.