Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/eql-v3-ffi-0-28-concrete-types.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/stack': minor
---

Upgrade `@cipherstash/protect-ffi` to 0.28.0 and update EQL v3 concrete Postgres domain names to match the SQL fixture (`integer*`, `smallint*`, `bool`, `real*`, and `double*`). The public factories remain semantic (`Integer`, `Smallint`, `Boolean`, `Real`, `Double`) while their concrete domains change, so this is a minor release.
5 changes: 5 additions & 0 deletions .changeset/eql-v3-public-domains.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/stack': patch
---

Re-vendor the EQL v3 SQL bundle and align the v3 DSL to it: encrypted type domains now live in the `public` schema (`public.text`, `public.integer`, …) rather than `eql_v3`, and the boolean domain is `public.boolean` (was `eql_v3.bool`). The `eql_v3` schema now holds only the operator-backing functions, and the index-term constructors (`hmac_256`, `ore_block_256`, `bloom_filter`) moved to `eql_v3_internal`. This keeps the SDK's emitted domain names byte-matched to the installed bundle so `CREATE TABLE`/cast resolution succeeds.
6 changes: 3 additions & 3 deletions .changeset/eql-v3-typed-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,19 @@ Every method derives its types from the concrete `table` / `column` builder
arguments:

- `encrypt` / `encryptQuery` pin the plaintext to the column's domain type
(`text → string`, `int8 → bigint`, `timestamp → Date`, …).
(`text → string`, `timestamp → Date`, …).
- `encryptQuery` constrains `queryType` to the column's capabilities and rejects
storage-only columns at compile time.
- `encryptModel` / `bulkEncryptModels` validate schema-column fields against their
inferred plaintext type (passthrough fields are untouched) and return a precise
encrypted model.
- `decryptModel` / `bulkDecryptModels` return the precise plaintext model,
reconstructing `Date` / `bigint` values from the encrypt-config `cast_as`.
reconstructing `Date` values from the encrypt-config `cast_as`.

Because the typed methods bind to the concrete branded v3 classes, a hand-rolled
structural table/column is rejected — closing the soundness gap where a non-branded
table could be encrypted at runtime while typed as plaintext.

Runtime behaviour is unchanged: the encrypt/query paths return the same operations
as the base client; only the model-decrypt paths add a per-column `Date` / `bigint`
as the base client; only the model-decrypt paths add a per-column `Date`
reconstruction step. The v2 client surface (`Encryption`) is untouched.
4 changes: 2 additions & 2 deletions .changeset/eql-v3-typed-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
'@cipherstash/stack': minor
---

Add EQL v3 schema builders for all generated SQL domains under `@cipherstash/stack/eql/v3`, exposed as the `types` namespace (one member per EQL v3 domain, e.g. `types.TextEq` / `types.Int4Ord` / `types.Timestamp`), including explicit query capability metadata (`getQueryCapabilities()` / `isQueryable()`) and v3 table support in model encryption helpers (`encryptModel` / `bulkEncryptModels`).
Add EQL v3 schema builders for supported generated SQL domains under `@cipherstash/stack/eql/v3`, exposed as the `types` namespace (one member per supported EQL v3 domain, e.g. `types.TextEq` / `types.IntegerOrd` / `types.Timestamp`), including explicit query capability metadata (`getQueryCapabilities()` / `isQueryable()`) and v3 table support in model encryption helpers (`encryptModel` / `bulkEncryptModels`).

Also widen the accepted plaintext input type for `encrypt` / `encryptQuery` to include `Date` and `bigint` (via the new `Plaintext` type), so v3 `date` / `timestamp` / `int8` domains can be encrypted and queried with their natural JavaScript values.
Also widen the accepted plaintext input type for `encrypt` / `encryptQuery` to include `Date` (via the new `Plaintext` type), so v3 `date` / `timestamp` domains can be encrypted and queried with their natural JavaScript values.
5 changes: 5 additions & 0 deletions .changeset/skip-v3-bigint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/stack': patch
---

Omit EQL v3 bigint factories from the public DSL and test matrix until native bigint round-tripping is supported.
43 changes: 43 additions & 0 deletions .github/actions/require-cs-secrets/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Require CipherStash secrets
description: >-
Fail the job when any CS_* credential is missing. The live test suites gate
themselves on these vars (vitest `describe.skip`, Deno `test.ignore`), so a
rotated / cleared / fork-PR-absent secret would make them silently skip and
let CI go green with ZERO live coverage. This turns that silent skip into a
loud failure. Requires the repo to be checked out first (local action path).

inputs:
workspace-crn:
description: secrets.CS_WORKSPACE_CRN
required: true
client-id:
description: secrets.CS_CLIENT_ID
required: true
client-key:
description: secrets.CS_CLIENT_KEY
required: true
client-access-key:
description: secrets.CS_CLIENT_ACCESS_KEY
required: true

runs:
using: composite
steps:
- name: Assert CS_* secrets are present
shell: bash
env:
CS_WORKSPACE_CRN: ${{ inputs.workspace-crn }}
CS_CLIENT_ID: ${{ inputs.client-id }}
CS_CLIENT_KEY: ${{ inputs.client-key }}
CS_CLIENT_ACCESS_KEY: ${{ inputs.client-access-key }}
run: |
missing=0
for v in CS_WORKSPACE_CRN CS_CLIENT_ID CS_CLIENT_KEY CS_CLIENT_ACCESS_KEY; do
if [ -z "${!v}" ]; then
echo "::error::Required secret $v is not set on this runner — live test suites would silently skip."
missing=1
fi
done
if [ "$missing" -ne 0 ]; then
exit 1
fi
10 changes: 10 additions & 0 deletions .github/workflows/prisma-example-readme-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,16 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# A missing / rotated / fork-PR-absent secret makes the walkthrough skip
# its live steps silently, hiding regressions behind a green job. Fail loud.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

# Build via turbo so `^build` on `@cipherstash/prisma-next` and
# its `@cipherstash/stack` peer is honoured. The test's
# `pnpm install` subprocess inside `examples/prisma/` is a no-op
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/prisma-next-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,17 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# The global-setup hook hard-errors without CS_WORKSPACE_CRN, but a
# missing sibling secret could still degrade coverage silently — assert
# all four up front so a rotated / fork-PR-absent secret fails loudly.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

# Write the CS_* credentials and the harness DATABASE_URL into the
# example app's .env so the runtime + the `prisma-next migration
# apply` invocation in global-setup both pick them up. The harness
Expand Down
44 changes: 31 additions & 13 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,16 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# Fail loudly if any live-test credential is missing, so the v3 live
# matrix (and every other `describeLive` suite) can't silently skip.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

- name: Type tests (stack)
run: pnpm --filter @cipherstash/stack run test:types

Expand Down Expand Up @@ -152,6 +162,16 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# Auth-dependent `e2e/` suites skip themselves without these vars — a
# silent skip would hide regressions behind a green job. Fail loudly.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

# Run the standalone `e2e/` workspace via turbo so the `^build`
# dep on the `test:e2e` task builds cli + wizard first. CLI's own
# E2E (`packages/cli/tests/e2e/**`) is covered by the `run-tests`
Expand Down Expand Up @@ -219,19 +239,17 @@ jobs:
- name: Build stack
run: pnpm exec turbo run build --filter @cipherstash/stack

# roundtrip.test.ts skips itself (Deno.test.ignore) when any of
# the four CS_* env vars is missing — that's the right behaviour
# for local runs, but in CI a silent skip would mean a rotated /
# cleared secret hides a real WASM regression behind a green job.
# Fail loudly instead.
- name: Assert CS_* secrets are present
run: |
for v in CS_WORKSPACE_CRN CS_CLIENT_ID CS_CLIENT_KEY CS_CLIENT_ACCESS_KEY; do
if [ -z "${!v}" ]; then
echo "::error::Required secret $v is not set on this runner — the WASM smoke test would silently skip."
exit 1
fi
done
# roundtrip.test.ts skips itself (Deno.test.ignore) when any of the four
# CS_* env vars is missing — fine locally, but in CI a silent skip would
# let a rotated / cleared secret hide a real WASM regression behind a
# green job. Fail loudly instead.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

- name: Run Deno WASM smoke test
working-directory: e2e/wasm
Expand Down
135 changes: 135 additions & 0 deletions docs/eql-v3-ord-term-ordering-defect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# EQL v3 non-superuser limitation: `ORDER BY eql_v3.ord_term(col)` does not sort in ORE order

**Status:** Root-caused with live evidence. This is **not** a bug in our SDK and
not a defect in the EQL bundle — it is a **known platform limitation** of the
current (ORE-based) vendored EQL v3 bundle on non-superuser Postgres
(e.g. Supabase). The superuser gate on the ORE operator class is an intentional
install constraint so the bundle installs at all on managed Postgres, where an
ORE ordered index genuinely cannot exist. Our tests/docs are corrected to stop
relying on the construct that is silently wrong in that state; the resolution
(an upstream `_ord`→OPE migration) is tracked as a follow-up (see
[Resolution](#resolution)).

## Summary

For an encrypted ordering column (any `*_ord` / `text_search` domain), the range
comparison functions `eql_v3.gte`/`lte`/`lt`/`gt` sort **correctly** (true ORE
order), but on a non-superuser install `ORDER BY eql_v3.ord_term(col)` sorts
**incorrectly** — deterministic but not the ORE/lexical order.

Reproduced against live Postgres (role `is_superuser = off`) with emails
`ada@example.com`, `grace@example.com`, `alan@example.net`, `zora@example.org`:

| Construct | Result | Correct? |
|---|---|---|
| pairwise `eql_v3.lt(col, col)` (the ORE comparator) | `ada < alan < grace < zora` | ✅ ORE/lexical |
| `ORDER BY eql_v3.ord_term(col)` | `zora, alan, ada, grace` | ❌ record_ops |
| `ORDER BY col` (domain default) | `grace, ada, alan, zora` (structural) | ❌ jsonb_ops |
| `ORDER BY col USING <` / `USING OPERATOR(public.<)` | error: *operator < is not a valid ordering operator* | ❌ no orderable family |

So on a non-superuser install there is **no** `ORDER BY` construct that yields
ORE order. Only the boolean comparison operators (used in `WHERE` predicates) are
ORE-correct — and they are correct regardless of superuser.

## Why this happens on Supabase

File: `packages/stack/__tests__/fixtures/eql-v3/cipherstash-encrypt-v3.sql`
(the vendored bundle — do not hand-edit; it is regenerated on re-vendor).

1. `eql_v3.ord_term(a public.text_search)` (and every `*_ord` domain, e.g.
`public.integer_ord` at line 18117) returns the **composite** type
`eql_v3_internal.ore_block_256` — line **38141-38144**:
```sql
CREATE FUNCTION eql_v3.ord_term(a public.text_search)
RETURNS eql_v3_internal.ore_block_256 ...
```
`ore_block_256` is `CREATE TYPE ... AS (terms eql_v3_internal.ore_block_256_term[])`
at line **193**.

2. The ORE-correct default btree opclass for that composite type,
`eql_v3_internal.ore_block_256_operator_class` (backed by
`eql_v3_internal.compare_ore_block_256_terms`), is created inside a
superuser-gated `DO` block — lines **3117-3138**. Creating an operator
class requires superuser, so on managed/non-superuser Postgres it is
**intentionally skipped** on `insufficient_privilege` (lines 3134-3136):
```
EQL: skipped operator class eql_v3_internal.ore_block_256_operator_class
(requires superuser); ORE ordered indexes on ore_block_256 unavailable ...
```
This gate is deliberate: it is what lets the EQL v3 bundle install cleanly on
Supabase and other non-superuser platforms, where an ORE ordered index cannot
exist at all. Live check confirms:
`SELECT ... FROM pg_opclass ... WHERE typname='ore_block_256'` returns `[]`.

3. With the opclass absent, `ORDER BY eql_v3.ord_term(col)` still "works"
because PostgreSQL supplies a built-in **record comparison** for any
composite type. That compares the `ore_block_256_term[]` fields by their raw
bytes — deterministic, but unrelated to ORE order. This is the silent-wrong
footgun: it does not error, it just sorts wrong.

4. Meanwhile `eql_v3.gte/lte/lt/gt` are ORE-correct **regardless of superuser**
because they do not go through index/sort machinery. `eql_v3.lt(text_search)`
(line 38206) is `SELECT eql_v3.ord_term(a) < eql_v3.ord_term(b)`, and the
`public.<` operator on `ore_block_256` is wired directly to
`eql_v3_internal.compare_ore_block_256_terms`. The domain-level `<` operator
(`CREATE OPERATOR <`, line 38566, `FUNCTION = eql_v3.lt`) exists, but it is
**not** a member of any btree operator family, so `ORDER BY col USING <`
raises "operator < is not a valid ordering operator".

The bundle comment (lines 3104-3114) documents the missing opclass as the
accepted managed-Postgres limitation ("ORE ordered scans ... unavailable"). The
practical consequence to be aware of is that `ORDER BY ord_term(col)` **silently
returns wrong results** in that state instead of failing, and there is no
non-superuser ORE ordering path on the current bundle.

## Minimal repro

```sql
-- role without superuser; EQL v3 bundle installed
CREATE TABLE t (email public.text_search NOT NULL, label text);
-- insert encrypted 'ada@example.com','grace@example.com',
-- 'alan@example.net','zora@example.org' (labels ada/grace/alan/zora)

SELECT current_setting('is_superuser'); -- off
SELECT count(*) FROM pg_opclass c JOIN pg_type ty
ON ty.oid=c.opcintype WHERE ty.typname='ore_block_256'; -- 0

SELECT label FROM t ORDER BY eql_v3.ord_term(email); -- zora,alan,ada,grace (WRONG)
-- vs the ORE comparator, which is correct:
SELECT x.label, y.label, eql_v3.lt(x.email, y.email) FROM t x CROSS JOIN t y; -- ada<alan<grace<zora
```

## Resolution

The fix is an upstream change in `cipherstash/encrypt-query-language`, already in
progress, that migrates ordering types off superuser-only ORE:

- All `_ord` types move to **OPE** (Order-Preserving Encryption). OPE orders via
a **native `bytea` btree**, so `ORDER BY` works **everywhere — including
Supabase — without superuser**.
- `_ord_ore` types remain **ORE** and are installed **only when superuser is
available**. On non-superuser installs the ORE types are **not installed at
all**, so there is no silent-wrong `ORDER BY` path to fall into.
- Net future ordering story: OPE `_ord` orders everywhere; ORE `_ord_ore` is
superuser-only (and absent on Supabase).

This is a **follow-up** to the current protect-ffi 0.28 PR — it lands as a
separate design spec plus the re-vendored OPE bundle, not as part of this change.

## What we changed in this repo

- `packages/stack/__tests__/schema-v3-pg.test.ts`: corrected the misleading
comment on the range test and added an explicit ORE-total-order proof using
pairwise `eql_v3.lt` (the only ORE-correct path on the current bundle), so the
guarantee is asserted positively rather than merely avoided. These assertions
go through the comparison operators (`eql_v3.lt`/`gte`/`lte`), which are
correct regardless of superuser.
- No change to the vendored `.sql` (it is regenerated on re-vendor).
- Docs that prescribe `orderBy: (col) => eql_v3.ord_term(col)` for a future v3
Drizzle/Prisma adapter (e.g.
`docs/superpowers/2026-06-12-eql-v3-drizzle-adapter-walkthrough.md`,
`docs/superpowers/plans/2026-07-06-eql-v3-drizzle-concrete-types.md`) would hit
this limitation on the current ORE bundle if implemented as written. Any v3
`orderBy` must not lower to a bare `ORDER BY ord_term(...)` on the current ORE
bundle; once the OPE `_ord` migration lands, `ORDER BY` on `_ord` becomes
correct everywhere.
23 changes: 22 additions & 1 deletion e2e/wasm/deno.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,34 @@
"//2": "stack is imported via a file URL because the wasm-inline subpath isn't on a published version yet — once stack ships with /wasm-inline this can switch to a plain npm: specifier. protect-ffi and auth resolve via npm: against the workspace's installed versions (Deno's nodeModulesDir: auto lets it use what pnpm fetched).",
"//3": "No --allow-ffi grant. If protect-ffi ever silently fell back to a native binding under Deno, the test would fail on missing FFI permission — this is the WASM guarantee.",
"nodeModulesDir": "auto",
"//4": "Deno 2.9+ enforces a 24h minimumDependencyAge cooldown by default. Mirror pnpm-workspace.yaml's supply-chain policy (minimumReleaseAge: 7d + minimumReleaseAgeExclude) so the two toolchains agree: keep the 7-day cooldown for third-party deps, but exempt the first-party CipherStash packages this test imports — they ship on their own cadence and are bumped in lockstep. Without the exclude, a just-published protect-ffi/auth (e.g. 0.28.0) fails resolution until it ages past the window.",
"//5": "Deno's exclude does not support globs, so the per-platform native binary sub-packages (optional deps published in lockstep at the same version) are enumerated explicitly — mirrors pnpm-workspace.yaml's protect-ffi-* / auth-* wildcards.",
"minimumDependencyAge": {
"age": "P7D",
"exclude": [
"npm:@cipherstash/protect-ffi",
"npm:@cipherstash/protect-ffi-darwin-arm64",
"npm:@cipherstash/protect-ffi-darwin-x64",
"npm:@cipherstash/protect-ffi-linux-arm64-gnu",
"npm:@cipherstash/protect-ffi-linux-x64-gnu",
"npm:@cipherstash/protect-ffi-linux-x64-musl",
"npm:@cipherstash/protect-ffi-win32-x64-msvc",
"npm:@cipherstash/auth",
"npm:@cipherstash/auth-darwin-arm64",
"npm:@cipherstash/auth-darwin-x64",
"npm:@cipherstash/auth-linux-arm64-gnu",
"npm:@cipherstash/auth-linux-x64-gnu",
"npm:@cipherstash/auth-linux-x64-musl",
"npm:@cipherstash/auth-win32-x64-msvc"
]
},
"tasks": {
"//task": "--no-check because TypeScript can't infer the column-on-table intersection types when stack is loaded via file URL (the dist .d.ts strips brand info). This test exists to verify RUNTIME WASM round-trips — type narrowing is covered by the package's own vitest suite.",
"test": "deno test --no-check --allow-env --allow-net --allow-read --allow-sys --no-prompt"
},
"imports": {
"@cipherstash/stack/wasm-inline": "../../packages/stack/dist/wasm-inline.js",
"@cipherstash/protect-ffi/wasm-inline": "npm:@cipherstash/protect-ffi@0.26.0/wasm-inline",
"@cipherstash/protect-ffi/wasm-inline": "npm:@cipherstash/protect-ffi@0.28.0/wasm-inline",
"@cipherstash/auth/wasm-inline": "npm:@cipherstash/auth@0.40.0/wasm-inline"
}
}
Loading
Loading