feat(plugins): add a Cloudflare R2 SQL driver - #2030
Open
datlechin wants to merge 2 commits into
Open
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Signed-off-by: Ngô Quốc Đạt <datlechin@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a registry-only driver for Cloudflare R2 SQL, the read-only SQL engine over Apache Iceberg tables in an R2 bucket.
What you get
Connect with an Account ID, a bucket, and a Cloudflare API token. The sidebar lists Iceberg namespaces as schemas and their tables underneath.
SELECTqueries run in the editor and render in the grid. Export works; nothing offers a write.Notes on the request
Two premises in the upstream issue turned out to be wrong, verified against wrangler's own source (
packages/wrangler/src/r2/sql.ts) rather than the docs:{"warehouse": ..., "query": ...}. The docs' curl example omitswarehouse; the CLI sends it.{accountId}_{bucket}, split on the first underscore. Asking for it as a third field would add a class of typos with no information gain, so the form takes Account ID + Bucket and derives it.The issue also suggested an Iceberg REST catalog client for schema browsing. Not needed: R2 SQL exposes
SHOW NAMESPACES,SHOW TABLES IN, andDESCRIBEon the same endpoint, free of charge, under the same auth. That avoids a second client whose auth failures come back as plain text, and Iceberg's%1Fmulti-level namespace encoding.Two app-core capabilities this needed
Both are app-side flags on
PluginMetadataRegistry.CapabilityFlags, so there is no PluginKit change and no ABI bump.isEngineReadOnly— TablePro already had the mechanism (SafeModeLevel.readOnly->blocksAllWrites, enforced inDefaultExecutionGateplus ~22 UI sites), but it was a user setting defaulting to.silentwith no way for an engine to pin it. Now such connections seed to Read-Only, the Safe Mode picker is disabled with an explanation, and the execution gate backstops it regardless of persisted state.Note
DriverPlugin.supportsReadOnlyModeis a false friend: it means "TablePro can tell reads from writes in this query language" (it feedsforcesWriteResolver), not "this engine is read-only". Overloading it would have changed behaviour for every other driver.supportsOffsetPagination— R2 SQL rejectsOFFSEToutright, evenOFFSET 0(unsupported feature: OFFSET clause is not supported), and capsLIMITat 10,000. BothPaginationStylecases emitOFFSET, and the enum is@frozen, so there was no honest way to say "this engine cannot page". Table tabs for such engines now show a single capped page labelled "First N rows" instead of fake page numbers; the nav buttons, the shortcut actions and the restore path are all suppressed so nothing can move the offset.An earlier design fetched
LIMIT offset+pageSizeand sliced client-side. It was dropped: each page rescans from row 0 on an engine billed by bytes scanned with a 10 MB floor, and the SQL shown in the tab would not reproduce the grid.QUALIFY ROW_NUMBER()is expressible but Cloudflare documents window functions, including throughQUALIFY, as budget-gated and rejected with a 400 when too much data would be scanned. Cloudflare's own documented pagination strategy is keyset (WHERE k > :last ORDER BY k LIMIT n), which is a cross-cutting change to TablePro's pagination model and belongs in its own PR.Also fixed
ColumnTypeClassifiermatchedARRAY/MAP/ROWbut notSTRUCT, so nested-struct columns lost JSON rendering. R2 SQL's Iceberg nested type is literallystruct.supportsSchemaEditing. That affects Cloudflare D1, MongoDB, DynamoDB and BigQuery today.Architecture
Pure logic lives in a new zero-dependency SPM module
TableProR2SQLCore(URL and body construction, response decoding, Iceberg/Arrow type mapping, SQL generation, error classification, identifier quoting). The plugin bundle holds only theURLSessiontransport and thePluginDatabaseDriverconformance. This mirrorsTableProTrinoCoreand is what makes the tests actually execute: suites inside a plugin bundle silently run zero assertions in the headless test host.Tests
82 new tests in
TableProR2SQLCoreTests, running underswift test. They cover the endpoint URL, both body fields being present, envelope decoding (includingsuccess: falseunder HTTP 200 and 500, and a non-JSON body becoming a typed error rather than a decode crash),Int64/UInt64precision never routed throughDouble, schema-ordered row mapping with missing keys as NULL, type normalization, and SQL generation asserting the output never containsOFFSETand that a'; DROP TABLEvalue stays inside a string literal.Plus
DatabaseTypeTestsfor the new type, and aColumnTypeClassifierTestscase forSTRUCT.Verification
swift test --package-path Packages/TableProCore: all suites pass.xcodebuild -scheme TablePro build: BUILD SUCCEEDED.xcodebuild -scheme CloudflareR2SQLDriverPlugin build: BUILD SUCCEEDED; bundle carries the right principal class, bundle id and PluginKit version 19.xcodebuild test -only-testing:TableProTests/ColumnTypeClassifierTests -only-testing:TableProTests/ExecutionGateTests: TEST SUCCEEDED.swiftlint --strict: clean.Not verified: no live R2 SQL account was available, so the authenticated round trip is untested. Everything above is pure-logic or compile-level. The exact column headers returned by
SHOW/DESCRIBEare read positionally rather than by name specifically because of that. A manual smoke test against a real R2 SQL bucket should happen before the registry release.https://claude.ai/code/session_01NtGEvGSCkym8Kb24YeXFez