Core: Retry Hadoop version hint before metadata scan

[majian1998]

Summary

Retry reading Hadoop version-hint.text briefly before falling back to scanning the metadata directory.

Problem

HadoopTableOperations updates version-hint.text as a best-effort pointer after committing a new metadata file. On object stores such as OSS/S3/GCS, the delete/rename sequence can make the hint file briefly unavailable or not yet visible to readers.

Today, a transient read failure immediately falls back to listing the metadata directory. For tables with many metadata files, especially on object-store-backed metadata directories, that listing can be significantly more expensive than retrying the small hint file read.

Fix

Add configurable retries for reading version-hint.text before metadata directory listing.

Defaults:

iceberg.version-hint.retry.num-retries = 2
iceberg.version-hint.retry.initial-wait-ms = 100
iceberg.version-hint.retry.max-wait-ms = 800

The retry uses exponential backoff capped by the configured max wait. If the metadata directory does not exist, findVersion() keeps the existing fast path and returns 0 without retrying. If retry sleep is interrupted, the interrupt is surfaced instead of falling through to metadata listing.

The configuration keys intentionally avoid the iceberg.hadoop.* prefix to prevent ambiguity with integrations that use iceberg.hadoop.* as a pass-through prefix for Hadoop configuration.

Test plan

git diff --check

Attempted:

./gradlew :iceberg-core:spotlessCheck :iceberg-core:test --tests org.apache.iceberg.hadoop.TestHadoopTableOperations --tests org.apache.iceberg.hadoop.TestHadoopCatalog.testVersionHintFileErrorWithFile --tests org.apache.iceberg.hadoop.TestHadoopCatalog.testVersionHintFileMissingMetadata --tests org.apache.iceberg.hadoop.TestHadoopCatalog.testMetadataFileMissing

Gradle did not reach test execution because plugin dependency downloads failed with TLS handshake errors from the Gradle plugin repository.

[wombatu-kun]

retryReadVersionHint and sleepBeforeVersionHintRetry reimplement retry, capped exponential backoff, and re-interrupt-then-throw that org.apache.iceberg.util.Tasks already provides (its exponentialBackoff(min, max, totalTimeout, scale) plus the sleep/interrupt handling in runSingleThreaded, which does the same Thread.currentThread().interrupt(); throw new RuntimeException(...)). AGENTS.md lists Tasks.foreach as the standard retry utility and this package already uses it in HadoopFileIO. Consider Tasks.foreach(versionHintFile).retry(numRetries).exponentialBackoff(initialWaitMs, maxWaitMs, totalTimeoutMs, 2.0).onlyRetryOn(IOException.class).run(...), falling back to the metadata listing in the catch when retries are exhausted. onlyRetryOn(IOException.class) also avoids retrying a deterministic parse failure (NumberFormatException from a corrupt hint), which the current catch (Exception) re-reads with full backoff.