Search before asking
Motivation
When Paimon syncs a table to an Iceberg REST catalog (metadata.iceberg.storage = rest-catalog), the only properties ever written to the Iceberg table are the two internal retention knobs (write.metadata.previous-versions-max, write.metadata.delete-after-commit.enabled) set in IcebergRestMetadataCommitter. Any other table option a user sets is silently dropped and never reaches the Iceberg table's properties.
This matters for us because we have downstream tooling and conventions (table categorization tags, ownership/color-coding metadata, etc.) that rely on custom Iceberg table properties, and today there's no way to set them on a Paimon-produced table without hand-editing the table after the fact outside Paimon.
Paimon already supports this pattern for its own native catalogs — HiveCatalog forwards table options into Hive/JDBC metastore table parameters by default (CatalogOptions.SYNC_ALL_PROPERTIES, defaulting to true). The Iceberg REST path just never got the equivalent treatment.
Solution
Add an explicit opt-in prefix, e.g. metadata.iceberg.table-properties. → , mirroring the existing metadata.iceberg.rest.* (IcebergOptions.REST_CONFIG_PREFIX) pattern already used to configure the REST catalog client. On commit, strip the prefix and merge the resulting keys into the Iceberg table's properties, alongside the two existing hardcoded ones.
Two things worth getting right:
- Filter out Iceberg's reserved property keys (TableProperties.RESERVED_PROPERTIES) before merging, since Iceberg throws if they're present.
- Apply the properties on both the create path and the drop-and-recreate path (the REST committer recreates the table on metadata-base mismatch), not just the update-in-place path — otherwise custom properties would silently disappear whenever that happens.
An explicit opt-in prefix (rather than forwarding all Paimon table options) avoids leaking Paimon-internal config (bucket, merge-engine, storage credentials, etc.).
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Motivation
When Paimon syncs a table to an Iceberg REST catalog (metadata.iceberg.storage = rest-catalog), the only properties ever written to the Iceberg table are the two internal retention knobs (write.metadata.previous-versions-max, write.metadata.delete-after-commit.enabled) set in IcebergRestMetadataCommitter. Any other table option a user sets is silently dropped and never reaches the Iceberg table's properties.
This matters for us because we have downstream tooling and conventions (table categorization tags, ownership/color-coding metadata, etc.) that rely on custom Iceberg table properties, and today there's no way to set them on a Paimon-produced table without hand-editing the table after the fact outside Paimon.
Paimon already supports this pattern for its own native catalogs — HiveCatalog forwards table options into Hive/JDBC metastore table parameters by default (CatalogOptions.SYNC_ALL_PROPERTIES, defaulting to true). The Iceberg REST path just never got the equivalent treatment.
Solution
Add an explicit opt-in prefix, e.g. metadata.iceberg.table-properties. → , mirroring the existing metadata.iceberg.rest.* (IcebergOptions.REST_CONFIG_PREFIX) pattern already used to configure the REST catalog client. On commit, strip the prefix and merge the resulting keys into the Iceberg table's properties, alongside the two existing hardcoded ones.
Two things worth getting right:
An explicit opt-in prefix (rather than forwarding all Paimon table options) avoids leaking Paimon-internal config (bucket, merge-engine, storage credentials, etc.).
Anything else?
No response
Are you willing to submit a PR?