feat(stack,cli): EQL v3 bigint + CLI install path, re-expressed on protect-ffi 0.28#583
Merged
tobyhede merged 9 commits intoJul 9, 2026
Merged
Conversation
🦋 Changeset detectedLatest commit: a72a113 The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
coderdan
reviewed
Jul 8, 2026
Comment on lines
+122
to
+130
| // v3 supports the direct install path only. Explicit --drizzle/--migration | ||
| // are rejected up-front by validateInstallFlags; auto-DETECTED drizzle or | ||
| // migration modes fall back to direct here rather than erroring. | ||
| if (eqlVersion === 3 && resolved.drizzle) { | ||
| p.log.info( | ||
| 'EQL v3 does not support the Drizzle migration path yet — installing directly.', | ||
| ) | ||
| resolved.drizzle = false | ||
| } |
Contributor
There was a problem hiding this comment.
Is there an issue tracked for this?
coderdan
reviewed
Jul 8, 2026
Comment on lines
+53
to
+55
| /** The v3 (`eql_v3`) Supabase grants block. See {@link supabasePermissionsSql}. */ | ||
| export const SUPABASE_PERMISSIONS_SQL_V3 = | ||
| supabasePermissionsSql(EQL_V3_SCHEMA_NAME) |
Contributor
There was a problem hiding this comment.
Clarification: the EQLV3 install now gracefully degrades to OPE if no operator class/family support is available. Is the dedicated Supabase installer needed for V3 now?
coderdan
reviewed
Jul 8, 2026
Comment on lines
+287
to
+293
| if (latest && eqlVersion === 3) { | ||
| // No public v3 release artifacts exist yet — the v3 bundles are vendored | ||
| // from the generated monolith (see scripts/build-eql-v3-sql.mjs). | ||
| throw new Error( | ||
| '`--latest` is not supported for EQL v3 yet: no public v3 release artifacts exist. Use the bundled install.', | ||
| ) | ||
| } |
coderdan
reviewed
Jul 8, 2026
| ``` | ||
|
|
||
| State is tracked in an append-only `cipherstash.cs_migrations` table installed by `stash db install`. The EQL intent (which indexes, which cast_as) continues to live in `eql_v2_configuration` so Proxy continues to work against the same database. | ||
| State is tracked in an append-only `cipherstash.cs_migrations` table installed by `stash eql install`. The EQL intent (which indexes, which cast_as) continues to live in `eql_v2_configuration` so Proxy continues to work against the same database. |
Contributor
There was a problem hiding this comment.
This might need to be updated to differentiate between V2 and V3.
coderdan
reviewed
Jul 8, 2026
Comment on lines
+85
to
+91
| * const users = encryptedTable('users', { | ||
| * email: types.TextSearch('email'), // public.text_search | ||
| * amount: types.IntegerOrd('amount'), // public.integer_ord | ||
| * }) | ||
| * | ||
| * const client = await Encryption({ schemas: [users] }) | ||
| * const es = encryptedSupabaseV3({ encryptionClient: client, supabaseClient: supabase }) |
Contributor
There was a problem hiding this comment.
This interface differs to what is currently on the launch blog post. I think a Rails-style auto-config loading from the schema is the money shot of V3. I'd love for us to be able to make it happen if we can. If not, we'll need to update the blog post.
coderdan
reviewed
Jul 8, 2026
coderdan
left a comment
coderdan
left a comment
Contributor
There was a problem hiding this comment.
This is looking really solid. A couple of things I'd like to align on before explicitly approving, but the direction is really good.
Base automatically changed from
chore/eql-v3-ffi-0-28
to
feat/eql-v3-text-search-schema
July 9, 2026 00:28
The bulk model helpers key per-field operation results by
`${modelIndex}-${fieldKey}` ids and reconstructed models with a naive
key.split('-'), truncating any field key containing a hyphen
(some-field -> some): the value landed under the truncated key and the
real field silently vanished. Duplicated across all four multi-model
reconstruction sites.
Extract one fieldsForModelIndex helper that splits at the FIRST hyphen
only and use it at all four sites. Offline regression tests (mocked
protect-ffi) prove hyphenated and multi-hyphen field names survive
bulkEncryptModels / bulkDecryptModels -- both fail against the naive
split.
Re-expressed from james/cip-3291-bigint-stack 632bdf6.
…n protect-ffi 0.28) Add BIGINT/BIGINT_EQ/BIGINT_ORD_ORE/BIGINT_ORD domain definitions against the public.bigint* concrete domains, the EncryptedBigint*Column classes, types.Bigint* factories, barrel exports, and union membership, following the existing per-domain pattern. The SDK-wide Plaintext union gains bigint, and PlaintextKind/PlaintextFromKind map cast_as 'bigint' to the JS bigint type (bigint columns always decrypt to a JS bigint). Shipped UN-GATED: protect-ffi 0.28 marshals a native JS bigint across the Neon boundary losslessly (encode on the way in, JsBigInt::from_i64 on decrypt), so no reconstruction is needed on the decrypt path and the live matrix suites exercise the round-trip directly. i64 bounds are enforced at the protect-ffi boundary. Index emission follows the numeric rule: bigint_eq -> unique (hm); bigint_ord/bigint_ord_ore -> ore (equality via ob). Removes the base's bigint skip changeset and flips the schema-v3 type guard from a negative (@ts-expect-error) to a positive assertion. The catalog-driven capability sweep and live-PG matrix pick up the four public.bigint* rows automatically (no liveGate). Re-expressed from james/cip-3291-bigint-stack 2495c65 + e593415 against the public.* domain surface and protect-ffi 0.28.
… eql install Vendor the v3 SQL bundles into packages/cli/src/sql via a checked-in derivation script (scripts/build-eql-v3-sql.mjs): the full bundle is a byte-identical copy of the stack public.* fixture monolith; the Supabase variant strips the two CREATE OPERATOR CLASS/FAMILY chunks at their --! @file markers (they need superuser). EQLInstaller gains an eqlVersion option on install/isInstalled/ getInstalledVersion; v3 keys install-detection, version(), and Supabase grants to the eql_v3 schema (type domains live in public, operators in eql_v3, constructors in eql_v3_internal). v3 + --latest is rejected (no public artifacts). eql install --eql-version 3 is direct-install only — --drizzle/--migration/--migrations-dir/--latest are rejected up front and auto-detected drizzle falls back to direct with a notice. Move db install under a dedicated eql command group as eql install; db install stays as a deprecated alias that warns and forwards. Help text, next-step hints, generated Supabase migration headers, wizard allowlist + steps, docs, and tests all reference eql install. Re-expressed from james/cip-3291-bigint-stack d15414a + e027b74 + 64fdeb2 against the public.* v3 bundle (scoped to the install rename; the eqlVersion runtime client flag and eql upgrade/status moves are intentionally not ported).
…ent --eql-version Update the stash-cli skill and @cipherstash/migrate README to the renamed `stash eql install` command (keeping `stash db install` documented as the deprecated alias), and document the new `--eql-version <2|3>` install flag and its v3 direct-install-only constraints in the skill. Re-expressed from james/cip-3291-bigint-stack 64fdeb2/d15414a5 docs. Historical CHANGELOGs and dated design docs are left as point-in-time records; the base already migrated the domain surface to public.*.
tobyhede
force-pushed
the
feat/eql-v3-bigint-restack
branch
from
f2d38d8 to
d1c2db6
Compare
…cally The plaintext comparator fell through to a String() compare for `bigint` samples, so the two bigint ORE domains derived a numerically wrong expected order (String(-42n) < String(-9223372036854775808n) while -42n > -9.2e18n). The ORE ciphertext order was correct; only the expectation was wrong.
…DME to v2 Addresses review feedback on #583: - Note that gating --drizzle and --latest behind --eql-version 2 is tracked in #585, at both sites where v3 declines the v2-only path. - State plainly that @cipherstash/migrate targets EQL v2, that v3 is opt-in and direct-install-only, and that v3 columns are outside this lifecycle.
tobyhede
force-pushed
the
feat/eql-v3-bigint-restack
branch
from
d1c2db6 to
dafcb02
Compare
tobyhede
changed the title
bigint domains map to Postgres int8, so a plaintext outside the signed 64-bit range cannot be stored. Add an i64-range check to the shared numeric guard (validateNumericValue / assertValidNumericValue) — the bigint analog of the existing NaN/Infinity guard — and thread it through the model / bulk-model encrypt path, which built the FFI payload directly and validated nothing. Also extend the match-index guard to reject bigint. protect-ffi 0.28 already i64-bounds-checks (encodeBigIntPlaintext throws a RangeError on every encrypt entry), so this is defense-in-depth: it fails earlier, before the network call, with a typed EncryptionError / Result. Correct the stale "enforced at the protect-ffi boundary" comments to reflect that the SDK now enforces it too. Matrix parity: widen DomainSpec.errorSamples to accept bigint and add BIGINT_ERR (i64::MAX + 1, i64::MIN - 1) to the four bigint domains so the live suite exercises rejection alongside the numeric families. Extend the offline guard suite with the single, lock-context and model/bulk bigint rejection cases.
WasmPlaintext omitted a bigint arm, so bigint (int8) columns were unusable on the wasm / edge entry point even though protect-ffi 0.28's wasm build already carries a native bigint across the boundary (encode_plaintext on encrypt, js_sys::BigInt on decrypt). Widen the union — a type-only change, the runtime is already supported upstream — and add a type-level test.
…ings toPlaintext fell through to String(value), so a bigint query value was encrypted as the text "1" and silently mismatched its column's bigint domain — data corruption the FFI could not catch, because the coercion happened before it. - stack (EQL v3): preserve the native bigint. The encryptQuery term type is Plaintext, which carries bigint, so protect-ffi 0.28 encrypts it against the int8 domain and i64-bounds-checks it. - @cipherstash/drizzle (EQL v2): the protect query term type is JsPlaintext | null and cannot carry a bigint, so reject loudly with a ProtectOperatorError instead of corrupting. Regression tests for both packages.
tobyhede
added a commit
that referenced
this pull request
Jul 9, 2026
…DME to v2 Addresses review feedback on #583: - Note that gating --drizzle and --latest behind --eql-version 2 is tracked in #585, at both sites where v3 declines the v2-only path. - State plainly that @cipherstash/migrate targets EQL v2, that v3 is opt-in and direct-install-only, and that v3 columns are outside this lifecycle.
tobyhede
added a commit
that referenced
this pull request
Jul 9, 2026
…DME to v2 Addresses review feedback on #583: - Note that gating --drizzle and --latest behind --eql-version 2 is tracked in #585, at both sites where v3 declines the v2-only path. - State plainly that @cipherstash/migrate targets EQL v2, that v3 is opt-in and direct-install-only, and that v3 columns are outside this lifecycle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-expresses the net-new work from
origin/james/cip-3291-bigint-stack(protect-ffi 0.27, oldeql_v3.*naming) onto this base's newer EQL v3 surface:public.*type domains, schema-drivencast_asrouting, and protect-ffi 0.28 (native JSbigintround-trip). Because the branches diverged on the domain surface, each feature is re-derived against BASE conventions rather than cherry-picked. Stacked onchore/eql-v3-ffi-0-28(PR #575) — set that as the merge base, notmain.Commits
fix(stack)hyphen fix —fieldsForModelIndex()splits bulk-model result ids at the first hyphen only, so field keys containing-survivebulkEncrypt/DecryptModels. Regression test fails against the old naivesplit('-'). Independent of EQL v3 entirely.feat(stack)bigint domain family (un-gated) —types.Bigint*+EncryptedBigint*Columnover thepublic.bigint*domains,bigintPlaintextKind/Plaintextarm. Shipped un-gated: protect-ffi 0.28 marshals a native JSbigintlosslessly (bigintWire: encode on encrypt,JsBigInt::from_i64on decrypt), so no reconstruction is needed and the live matrix exercises the round-trip. Removes the base'sskip-v3-bigintchangeset and flips the@ts-expect-errorguard to a positive assertion.feat(cli)EQL v3 install path +db install→eql install—--eql-version <2|3>installs vendoredpublic.*bundles (derived from the stack fixture viascripts/build-eql-v3-sql.mjs), keyed to theeql_v3schema; v3 is direct-install-only.db installbecomes a deprecated alias. (Scoped to the install rename — theeqlVersionruntime client flag is intentionally not ported; BASE supersedes it withcast_asrouting.)docs— reconcile living docs (stash-cli skill, migrate README) tostash eql install+ document--eql-version.test(stack)bigint ORE comparator — the plaintext comparator fell through to aString()compare forbigintsamples, so the two bigint ORE domains derived a numerically wrong expected order (String(-42n) < String(-9223372036854775808n)while-42n > -9.2e18n). Ciphertext order was correct; only the expectation was wrong.docs(cli,migrate)— review feedback: reference EQL v3: make it the default, and stop the CLI recommendingdb push(a v2/Proxy-only step) #585 where v3 declines the v2-only--drizzle/--latestpaths; scope the migrate README explicitly to EQL v2.Note: ~86.6k of this PR's added lines are the two vendored SQL bundles (
cipherstash-encrypt-v3.sql,-v3-supabase.sql) in commit 3. The hand-written surface is small.Review status
Resolved this round: the #585 tracking references (
db/install.ts,installer/index.ts) and the migrate README v2/v3 distinction.Still open — @coderdan on
installer/index.ts:55, "Is the dedicated Supabase installer needed for V3 now?" Answer, for the record: yes, but only for the role grants. Your premise is right — OPE degradation is now automatic (install.ts:200-209), and for v3resolveBundledFilenamemaps both--supabaseand--exclude-operator-familyto the same bundle (index.ts:414-428), so--supabaseis no longer what avoids the operator classes. What it still uniquely does is issueSUPABASE_PERMISSIONS_SQL_V3—GRANT USAGE/SELECT/EXECUTEoneql_v3toanon/authenticated/service_role(index.ts:318-319 → 346-351). The generic OPE fallback issues no grants, and those roles don't own the schema. (Related, unverified: those grants may not covereql_v3_internal, which backs theMIN/MAXaggregates and jsonb operators — see #588.)Verification
pnpm --filter @cipherstash/stack test:types→ 67 ✓ (theRecord<EqlV3TypeName, DomainSpec>catalog gate covers the newpublic.bigint*rows)Client key not configuredenvironmental baseline (unchanged with/without this branch, confirmed by stashing)matrix-live-pg/schema-v3-pgprove the bigint round-trip end-to-end but need CS workspace creds (CS_*) — run in CI or locally with creds. Code is verified against 0.28'sbigintWireFFI contract.