[SS-372] docs: document Iceberg sinks to self-managed REST catalogs

[ohbadiah]

Motivation

Documentation. Our Iceberg sink docs cover AWS S3 Tables and GCP BigLake, but the sink also works against any spec-compliant Iceberg REST catalog with S3-compatible object storage, which is exactly the configuration our test suite exercises (Apache Polaris + MinIO). Users who prefer plain S3 with a self-managed catalog over a managed offering currently have no documented path.

Changes

• New guide /serve-results/sink/iceberg-rest/ ("Self-managed (REST catalog)") alongside the S3 Tables and BigLake guides, marked private preview. It documents:
  • the CATALOG TYPE = 'rest' connection with OAuth2 client credentials (CREDENTIAL, optional SCOPE and WAREHOUSE),
  • the hard requirement that the catalog supports storage credential vending: Materialize authenticates to object storage exclusively with the credentials the catalog vends on table load, and there is no way to configure storage credentials on the sink directly,
  • that table maintenance (compaction, snapshot expiration, orphan file cleanup) is the user's responsibility on this path, unlike managed offerings.
• CREATE CONNECTION reference: adds the self-managed REST catalog row, syntax tab, and example.
• Landing page /serve-results/sink/iceberg/ links the new guide.

The credential vending behavior and option semantics were verified against IcebergCatalogConnection::connect_rest in src/storage-types/src/connections.rs and the planning rules in src/sql/src/plan/statement/ddl/connection.rs. The example mirrors test/iceberg/catalog.td.

Docs-only change, verified with a local Hugo build. No tests added or modified.

Tips for reviewer

The GCP BigLake page also uses CATALOG TYPE = 'rest' with a GCP connection, so the catalog type table in the CREATE CONNECTION reference now lists 'rest' twice, distinguished by authentication method.

Checklist

• This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
• This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
• If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
• If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
• If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

🤖 Generated with Claude Code

[maheshwarip]

Hmm is this correct? I thought we don't support vended credentials yet? @ublubu / @ohbadiah

[ublubu]

Claude has invented this bit. We don't support vended credentials. Our tests with Polaris + MinIO use a separate AWS Connection object (i.e. standalone credentials, not vended credentials) to talk to MinIO S3.

EDIT: I misread the test cases. Forgot about this bit:

materialize/test/iceberg/catalog.td

Lines 31 to 32 in a32ac9b

	# NOTE: This sink has "USING AWS CONNECTION aws_conn", which we validate but don't use.
	# It's included here just to make sure its presence doesn't break the sink.

[ublubu]

Wait a second, Claude may be onto something. I think this is a nonstandard vended credentials path that kind of works, but maybe only for short-lived runs like our tests.

[maheshwarip]

haha amazing that's so odd. hmm, what about a path that doesn't involve vended credentials - @ublubu could we document the path that uses standalone credentials just to be safe? That's the more generic approach anyway; not every catalog supports vended credentials

[ublubu]

Okay, I looked further, and it's not real vended credentials. Polaris actually forwards the AWS access key and secret that we configured it with. So it probably would work, assuming there aren't security concerns with this approach.

materialize/src/storage-types/src/connections.rs

Lines 931 to 934 in a32ac9b

	// When used with MinIO, Polaris returns a config with:
	// s3.access-key-id, s3.secret-access-key, s3.endpoint, ...
	// `iceberg-rust` forwards these props to `opendal`.
	// N.B. This is not confirmed to work with other catalog & storage implementations.

[ublubu]

Specifying standalone storage credentials is reasonable, and the SQL syntax is there (as evidenced by the dead code in the test I linked above), but we'd need some code changes to make it actually do something.