From c5e5b080a56cd2c246578e9403ae7dc7f984285d Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sat, 8 Aug 2026 23:28:42 -0400 Subject: [PATCH 1/8] =?UTF-8?q?=F0=9F=92=A5=20Contain=20document=20filesys?= =?UTF-8?q?tem=20access=20behind=20API.Files=20(#227)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ``, `` and `` reached the host filesystem directly, so `xmd run` and a workflow run could not mean the same thing for one document. Document filesystem access now goes through `API.Files`, a contextual Api of whole semantic operations with no host default. The four CLI entrypoints install the host provider explicitly; a run with none installed fails rather than reaching the host. The three components make no filesystem call of their own and import no host path or fs module. What they keep is order: a write's lexical check runs before its children, and the semantic write that follows repeats admission and owns every later phase, so the earlier check authorizes nothing. Ordinary failures cross the boundary as frozen structural data — a reason from a fixed vocabulary and the phase it came from — and every printed message is byte-identical to before. A provider that is absent, refuses an operation, or breaks its own contract throws instead, and core's fatal traversal ranks it between a durability failure and a documentation failure, by identity and by structural tag. --- .github/workflows/ci.yml | 96 ++- architecture.md | 58 +- packages/cli/src/bun.ts | 7 +- packages/cli/src/compiled.ts | 7 +- packages/cli/src/deno.ts | 7 +- packages/cli/src/node.ts | 7 +- packages/core/src/components/File.ts | 458 ++++------- packages/core/src/components/Glob.ts | 145 ++-- packages/core/src/components/TempDir.ts | 60 +- .../core/src/components/fs-error-phrases.ts | 76 +- packages/core/src/errors.ts | 67 +- packages/core/src/files.ts | 112 +++ .../core/tests/component-registration.test.ts | 6 +- packages/core/tests/fatal-cause.test.ts | 193 ++++- packages/core/tests/file-component.test.ts | 10 +- packages/core/tests/files-fatal.test.ts | 499 ++++++++++++ packages/core/tests/glob-component.test.ts | 5 +- packages/core/tests/inline-root.test.ts | 3 +- packages/core/tests/loop.test.ts | 18 + packages/core/tests/output-error-mode.test.ts | 4 +- packages/core/tests/temp-dir.test.ts | 9 + packages/runtime/apis.ts | 16 +- packages/runtime/files.ts | 653 +++++++++++++++ packages/runtime/host-files.ts | 545 +++++++++++++ packages/runtime/mod.ts | 52 +- packages/runtime/tests/host-files.test.ts | 748 ++++++++++++++++++ scripts/files-contract-probe.ts | 173 ++++ .../filesystem-contract-workflow.test.ts | 152 ++++ specs/executable-mdx-spec.md | 395 +++++++-- 29 files changed, 3988 insertions(+), 593 deletions(-) create mode 100644 packages/core/src/files.ts create mode 100644 packages/core/tests/files-fatal.test.ts create mode 100644 packages/runtime/files.ts create mode 100644 packages/runtime/host-files.ts create mode 100644 packages/runtime/tests/host-files.test.ts create mode 100644 scripts/files-contract-probe.ts create mode 100644 scripts/tests/filesystem-contract-workflow.test.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3cdffcba..3ca07ce3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -255,6 +255,89 @@ jobs: --component-dir smoke-test/agent/components \ --raw + # The host document-filesystem contract, on every target a release ships. + # + # `test-deno`, `test-node`, and `test-bun` run the whole corpus on Linux x64 + # and prove the contract holds there. What they cannot prove is that it holds + # on the other four triples: the host adapter's containment is path arithmetic + # plus `realpath`, and both are the platform's — Windows has drive letters, + # UNC paths, junctions, and reparse points that POSIX does not, and the two + # macOS rows resolve `/var` through a symlink that a naive comparison reads as + # an escape. + # + # So this row is focused rather than exhaustive: one suite, plus a compiled + # probe, on each of the five. The compiled probe is the second half of the + # claim — the shipped artifact is a binary, and the adapter reaches + # `node:path`, `node:fs`, and `node:os` through whatever `deno compile` put in + # its graph. + filesystem-contract: + strategy: + fail-fast: false + matrix: + include: + - runner: macos-15 + target: aarch64-apple-darwin + - runner: macos-15-intel + target: x86_64-apple-darwin + - runner: ubuntu-24.04 + target: x86_64-unknown-linux-gnu + - runner: ubuntu-24.04-arm + target: aarch64-unknown-linux-gnu + - runner: windows-2025 + target: x86_64-pc-windows-msvc + runs-on: ${{ matrix.runner }} + defaults: + run: + shell: bash + steps: + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + + - uses: denoland/setup-deno@e95548e56dfa95d4e1a28d6f422fafe75c4c26fb # v2.0.3 + with: + deno-version: v2.9.5 + + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: "22" + + - uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4 + + - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 + with: + bun-version: 1.3.14 + + - name: Install dependencies + run: deno task deps + + # Deno first, and the compile with it: `pnpm install` and `bun install` + # each rewrite `node_modules` into their own layout, so a Deno step after + # one of them resolves through links the other pruned (#279). + - name: Host contract under Deno + run: deno test --allow-all --frozen packages/runtime/tests/host-files.test.ts + + - name: Host contract as a compiled binary + run: | + set -eu + deno compile --allow-all --frozen \ + --output dist/files-contract-probe scripts/files-contract-probe.ts + if [ -f dist/files-contract-probe.exe ]; then + ./dist/files-contract-probe.exe + else + ./dist/files-contract-probe + fi + + - name: Install the Node layout + run: pnpm install + + - name: Host contract under Node + run: pnpm exec tsx --tsconfig tsconfig.node.json --test packages/runtime/tests/host-files.test.ts + + - name: Install the Bun layout + run: bun install + + - name: Host contract under Bun + run: bun test --timeout=300000 packages/runtime/tests/host-files.test.ts + # The same chain `deno task verify:clean` runs locally. It is the regression # for #279's ownership claim: a build that installs anything moves the # prepared-state fingerprint, prunes pnpm's links, and fails the resolution @@ -384,7 +467,18 @@ jobs: run: bun run packages/cli/src/bun.ts test smoke-test/test-agent/README.md --raw green: - needs: [lint, test-deno, jsr, smoke, composability, site, test-node, test-bun] + needs: + [ + lint, + test-deno, + jsr, + smoke, + filesystem-contract, + composability, + site, + test-node, + test-bun, + ] if: always() runs-on: ubuntu-latest steps: diff --git a/architecture.md b/architecture.md index f8d18266..6f2dbba2 100644 --- a/architecture.md +++ b/architecture.md @@ -42,6 +42,10 @@ Existing documents and code get aligned to this section retroactively. | blocker | execution needs input (auth, a human answer); not a failure | | suspension | a durable wait: a crash restarts into the same wait | | Workspace | the provider-neutral, run-owned environment that supplies retained filesystem, repository, process and working-directory capabilities to a workflow | +| document filesystem | the files a document names in its own text, reached only through `API.Files`; distinct from the host paths the engine's own control plane reads | +| Files provider | the installed implementation of `API.Files` for a document execution: the host provider under `xmd run`, a transaction-bound provider under a workflow run. There is no default, and no provider falls back to another | +| stable host namespace | the condition the host Files provider's containment is stated under: no other process replaces a directory, symlink, junction or reparse point between the moment a path is observed and the moment it is used | +| Files infrastructure failure | a Files provider that is absent, that refuses an operation, or that broke its own contract; fatal like a durability failure, and never a printed error | | ephemeral | a replay classification for an operation, context or attachment that runs again to reconstruct live execution; its result is not substituted from the journal and it owns no durable workflow state | | live binding | an execution-owned value reconstructed ephemerally for the current document execution; it is visible only to constructs that explicitly consume the live binding overlay and never enters interpolation or the journal | | attached service | a scoped host process that publishes its authenticated loopback endpoint through the XMD service handshake protocol and remains supervised for the lifetime of its service attachment | @@ -688,7 +692,7 @@ the process may terminate, and a later document execution arrives back at the same wait. An error that reveals a blocker (an expired login) reaches suspension through middleware; waiting itself is never raised. -### 8. Durability failures are outside the model +### 8. Durability and Files infrastructure failures are outside the model A durability failure (§6.11) says the journal no longer describes the document execution. No middleware sees it; it is never the document's own outcome. A @@ -705,6 +709,55 @@ throws. A pre-persistence policy rejection remains the policy's ordinary document failure; the guarded stream marks that boundary before the backing append and does not activate the fail-stop state. +A Files infrastructure failure is outside the model on the same terms. A +missing provider, a refused operation, and a provider that broke its own +contract are none of them things the document did or can act on, and printing +one would let every step after the file work run as though the file work had +happened. No middleware converts one, and no printing boundary prints one. + +Both are discovered through one cycle-safe traversal of the whole cause graph, +and precedence is decided by kind rather than by position: a durability failure +first, then a Files infrastructure failure, then a documentation failure. The +selected failure comes back by identity, because a fail-stop that records "the +first error" has to record the one that happened. Recognition is structural — +a stable tag on frozen data, never `instanceof` — so a failure a separately +loaded copy constructed is found on the same terms as one this copy did. + +## The document filesystem boundary + +Contextual routing is not authority. `API.Files` decides *which* provider +answers a document's file operations; what a provider is allowed to do is +decided by the provider, from identities the contextual layer cannot supply. + +The Api's operations are whole semantic acts — read this path, replace this +path, list what these patterns select — rather than steps a caller sequences. +The one preliminary operation, `checkFilePath`, is deliberately weak: pure path +arithmetic, no filesystem access, and it returns nothing usable. `` calls +it to decide whether a write's children may expand, and the later write repeats +the same admission from the same authored path. Nothing is handed between them, +so a check that was skipped or answered elsewhere authorizes nothing. + +The two providers make different containment claims, and both are stated rather +than implied: + +- **`xmd run`** resolves document paths in the caller's own filesystem. It + refuses empty, absolute, and lexically escaping paths without touching the + filesystem, and refuses an observed outward symlink once resolution can see + one. That holds while the host namespace is stable; it is not a sandbox, and + closing the replacement window would require a native dependency this project + does not take. +- **A workflow run** resolves document paths in the run-owned Workspace's + logical filesystem, inside the caller-owned transaction. A document path + never becomes a host path, so there is no host namespace to replace. + +Neither claim covers a native command a document runs. + +Failure data crosses the boundary as a plain frozen object under a stable tag, +carrying a reason from a fixed vocabulary and the phase it came from. No +message, errno code, resolved path, temporary name, or symlink target crosses. +Consumers parse that data before reading a field; a write whose data does not +validate is a provider-contract failure rather than a commit state to invent. + ## Attempts - The journal records every attempt in full; replay restores the outcome @@ -831,6 +884,9 @@ Status is measured against main. | Workspace coordination API | fails closed by default and lets a Workspace operation explicitly select provider coordination | built on the #365 stack; the atomic Deno Workspace handler is unbuilt | | explicit WorkflowRun journal route | binds one already-filtered publication to one exact active transaction and otherwise uses ordinary serialized journal storage | built on the #365 stack | | `API.Service` / `startService()` | creates an authenticated, supervised loopback service attachment through a provider-neutral operation | built on main | +| `API.Files` | routes every document filesystem operation to the installed provider, with no host default and structural failure data | built on the #227 stack | +| host Files provider / `useHostFiles()` | resolves document paths in the caller's filesystem, containing them while the host namespace is stable; installed by all four CLI entrypoints | built on the #227 stack | +| transaction-bound Files provider | resolves document paths in the run-owned logical Workspace inside the caller-owned transaction | unbuilt; the adapter is #227's second layer, and workflow effect coordination and CLI reachability remain unbuilt | | `service=` | publishes the attachment's endpoint into the live binding overlay for its invocation | built on main | | `ephemeral eval` | reconstructs live middleware and bindings without a journal entry | built on main | | `useWorkflowServiceDenial()` | provides and tests a non-delegating workflow service denial provider; #366 will install it in future start and resume scopes | built on main; no workflow CLI execution branch exists yet | diff --git a/packages/cli/src/bun.ts b/packages/cli/src/bun.ts index 5a72f684..6f8ba15f 100644 --- a/packages/cli/src/bun.ts +++ b/packages/cli/src/bun.ts @@ -8,7 +8,7 @@ import { main } from "effection"; import { fileURLToPath } from "node:url"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useBunService } from "./bun-service.ts"; @@ -30,5 +30,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useBunService); }); diff --git a/packages/cli/src/compiled.ts b/packages/cli/src/compiled.ts index 3ac11182..e1abf66a 100644 --- a/packages/cli/src/compiled.ts +++ b/packages/cli/src/compiled.ts @@ -7,7 +7,7 @@ import { main } from "effection"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useCompiledService } from "./compiled-service.ts"; @@ -27,5 +27,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useCompiledService); }); diff --git a/packages/cli/src/deno.ts b/packages/cli/src/deno.ts index 8d0ff3ea..4415aebc 100644 --- a/packages/cli/src/deno.ts +++ b/packages/cli/src/deno.ts @@ -10,7 +10,7 @@ import { main } from "effection"; import { fileURLToPath } from "node:url"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useDenoService } from "./deno-service.ts"; @@ -32,5 +32,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useDenoService); }); diff --git a/packages/cli/src/node.ts b/packages/cli/src/node.ts index 01d20f05..878ab6fc 100755 --- a/packages/cli/src/node.ts +++ b/packages/cli/src/node.ts @@ -14,7 +14,7 @@ import { main } from "effection"; import { fileURLToPath } from "node:url"; import process from "node:process"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { compileTempFile } from "@executablemd/core"; import { runXmd } from "./cli.ts"; import { useNodeService } from "./node-service.ts"; @@ -37,5 +37,10 @@ await main(function* (args) { }, { at: "min" }, ); + // Document filesystem access resolves in the caller's own filesystem here. + // It is installed explicitly, and at the same depth, because `API.Files` has + // no host default: a run with no provider must fail rather than reach the + // host by accident. + yield* useHostFiles(); yield* runXmd(args, useNodeService); }); diff --git a/packages/core/src/components/File.ts b/packages/core/src/components/File.ts index 8b3607b1..b0e7cb83 100644 --- a/packages/core/src/components/File.ts +++ b/packages/core/src/components/File.ts @@ -4,77 +4,69 @@ * * Both forms take one relative `path`, resolved against `Env.cwd`, so a * document composes with `` without choosing where anything lives. - * Everything the component touches is confined to that directory, checked in - * two stages that answer different questions at different times. + * What "resolved" means belongs to the installed `API.Files` provider: under + * `xmd run` it is a path in the caller's own filesystem, and under a workflow + * run it names an entry in a logical filesystem the run owns. This component + * names neither, and makes no filesystem call of its own. * - * The first stage is pure path arithmetic against `Env.cwd`: an empty path, an - * absolute path, and a lexical `..` escape are refused with no filesystem call - * at all. It runs **before** the children expand, so an unusable path costs - * nothing and its printed error is written before there is any child failure to - * report alongside it. + * What it does own is **order**, and the order is the whole design. A write + * happens in two stages that answer different questions at different times. * - * The second stage resolves what is actually on disk and re-checks the result, - * which is what catches a symlink leaving the workspace. It runs **after** the - * children have finished, because a child can change what a path means — - * replacing a directory with a symlink out of the workspace, for instance — - * and a destination resolved earlier would not be the one the write lands on. + * The first is `checkFilePath`: pure path arithmetic, no filesystem access, and + * nothing usable comes back. An empty path, an absolute path, and a lexical + * `..` escape are refused there. It runs **before** the children expand, so an + * unusable path costs nothing and its printed error is written before there is + * any child failure to report alongside it. * - * Writes go through a sibling temporary file and a rename. The rename is the - * **commit point**: everything before it can fail or be cancelled with the - * previous file untouched, and once it begins the result is the complete old - * file or the complete new one, never a partial write. It is not a - * transaction — a commit that has happened is not rolled back by a later - * cancellation. The temporary also closes the one containment hole resolution - * cannot: a dangling symlink has nothing to resolve, and `rename` replaces the - * link rather than following it wherever it points. + * The second is `writeTextFile`, which runs **after** the children have + * finished, because a child can change what a path means — replacing a + * directory with a symlink out of the workspace, for instance — and a + * destination resolved earlier would not be the one the write lands on. That + * call repeats admission from the same authored path and then owns every later + * step: resolution, target classification, parent creation, and the commit. The + * earlier check therefore authorizes nothing; it only decides whether children + * run. * - * Printed errors name only the path the document wrote. A resolved workspace - * path, the destination a symlink pointed at, a temporary file, and a rejected - * absolute path are all withheld — §1.2 keeps absolute paths out of - * printed errors, and a containment failure is the last place to start reporting - * them. A platform error carries the path it failed on, so every filesystem - * call is wrapped and nothing from the error it caught is reproduced: the errno - * code *selects* a phrase from the allowlist in `fs-error-phrases.ts`. + * Printed errors name only the path the document wrote. A resolved path, the + * destination a symlink pointed at, a temporary file, and a rejected absolute + * path never cross the provider boundary at all — what comes back is a + * `FilesReason` from a fixed vocabulary, which *selects* a phrase from + * `fs-error-phrases.ts`. §1.2 keeps absolute paths out of printed errors, and a + * containment failure is the last place to start reporting them. * - * A failed cleanup of the temporary is the one thing reported that the document - * did not ask for, and it is reported alongside the write's own outcome rather - * than instead of it — a file the document did not create may be sitting in its - * directory, which it cannot learn any other way. + * A failed cleanup of a provider's temporary is the one thing reported that the + * document did not ask for, and it is reported alongside the write's own + * outcome rather than instead of it — a file the document did not create may be + * sitting in its directory, which it cannot learn any other way. * - * Every filesystem call goes through the contextual `API.Fs`, so a host can - * observe or sandbox a document's own file access on the same terms as the - * engine's component resolution. + * ## Failure, and the two kinds of it + * + * An ordinary filesystem condition arrives as a structured `Err` and becomes a + * printed error. A provider that is not installed, that refuses an operation, + * or that breaks its own contract throws instead, and `invokeFiles` keeps it + * fatal: there is nothing for the document to read in "no filesystem provider + * is installed", and printing it would let the siblings after this one run as + * though the file work had happened. * * ## Threat model * - * Containment is judged against the filesystem as this component observes it. - * That is sound while the filesystem is stable, and every guarantee here is - * stated on that basis. It is not a sandbox: nothing prevents another process - * from replacing a directory with a symlink between the moment a path is - * validated and the moment it is used. Deferring resolution until immediately - * before the write narrows that window and covers the document's own children, - * which is the case a document controls; closing it entirely needs a - * capability or a platform-enforced sandbox, and is issue #227. + * Containment is the provider's claim, not this component's. The host provider + * judges it against the filesystem as it observes it, which is sound while the + * host pathname namespace is stable; a transaction-bound provider resolves + * logical paths that never reach a host filesystem call at all. Deferring the + * second stage until immediately before the write is what covers the + * document's own children under either. */ -import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; -import { randomUUID } from "node:crypto"; -import { ensure, scoped } from "effection"; import type { Operation } from "effection"; import { printErrors } from "../component-failures.ts"; -import { - cwd, - ensureDir, - readTextFile, - realpath, - remove, - rename, - stat, - writeTextFile, -} from "@executablemd/runtime"; +import { cwd } from "@executablemd/runtime"; +import { parseFilesFailure } from "@executablemd/runtime"; +import type { FilesFailureData, FileWriteFailureData, FilesReason } from "@executablemd/runtime"; import { content } from "../component-api.ts"; import { hasContent } from "../content-context.ts"; import { ContentError } from "../errors.ts"; +import { checkFilePath, readFileText, writeFileText, writeReport } from "../files.ts"; import type { Json } from "../types.ts"; import { reason } from "./fs-error-phrases.ts"; @@ -95,45 +87,32 @@ export class FileAccessError extends Error { } } -/** One sanitized sentence: the document's own path, and an allowlisted phrase. */ -function sentence(requested: string, verb: string, error: unknown): string { - return `cannot ${verb} "${requested}": ${reason(error)}.`; -} - -/** - * Run a filesystem operation, replacing whatever it throws with a printed error - * built only from the path the document wrote and an allowlisted phrase. - * - * Nothing is passed through, including a `FileAccessError`. This component's - * own checks throw outside guarded calls, so an error surfacing from inside one - * came from the Api — and an error's class says nothing about whether its - * message is safe to show. Trusting one would let middleware choose the text of - * a printed error by choosing what to throw. - * - * Cancellation is not a thrown error in Effection — halting resumes the - * generator through `return()` — so this never converts a halt into a failure. - */ -function* guard(requested: string, verb: string, operation: Operation): Operation { - try { - return yield* operation; - } catch (error) { - throw new FileAccessError(sentence(requested, verb, error)); - } -} - export default printErrors(function* (props: Record): Operation { const requested = String(props.path); - const admitted = yield* admissible(requested); + const directory = yield* cwd(); if (yield* hasContent()) { + const admitted = yield* checkFilePath({ cwd: directory, path: requested }); + if (!admitted.ok) { + throw new FileAccessError(refusal(requested, "write", parseFilesFailure(admitted.error))); + } + // The children run only once the path is known to be usable, and the // destination is resolved only once they are done. const text = yield* rendered(requested); - yield* write(requested, yield* destination(admitted), text); + const written = yield* writeFileText({ cwd: directory, path: requested, content: text }); + const failure = writeReport(written); + if (failure !== undefined) { + throw new FileAccessError(report(requested, failure)); + } return ""; } - return yield* read(requested, yield* destination(admitted)); + const read = yield* readFileText({ cwd: directory, path: requested }); + if (!read.ok) { + throw new FileAccessError(refusal(requested, "read", parseFilesFailure(read.error))); + } + return read.value; }); /** @@ -145,7 +124,7 @@ export default printErrors(function* (props: Record): Operation` renders nothing: this @@ -173,259 +152,130 @@ function* rendered(requested: string): Operation { } } -/** A path that has passed the lexical stage, carried to the resolving one. */ -interface Admissible { - /** The path the document wrote, for printed errors. */ - requested: string; - /** `requested` joined onto the contextual directory and normalized. */ - lexical: string; -} +const EMPTY = "path is empty; give a path relative to the working directory."; +const ABSOLUTE = "an absolute path is not accepted; give a path relative to the working directory."; /** - * Stage one: what can be decided without touching the filesystem. + * The sentence for a failure that stopped before anything was written. * - * An empty path, an absolute path, and a `..` escape are all answerable from - * `Env.cwd` and path arithmetic alone. Deciding them here means an unusable - * path is refused before the children of a write run at all, and that the - * printed error never has to mention a path that was rejected for being absolute. + * The lexical refusals and the two containment escapes are named rather than + * phrased, because each says something about the path the document wrote rather + * than about a filesystem condition. A rejected absolute path is deliberately + * not echoed: §1.2 keeps absolute paths out of printed errors, and the whole + * reason this one was refused is that it named somewhere else. * - * `resolve` normalizes `..` lexically, so this holds against the contextual - * directory as given — canonicalizing it is stage two's job and would only - * move the same comparison onto a different pair of strings. + * Everything else is the verb, the document's own path, and an allowlisted + * phrase. */ -function* admissible(requested: string): Operation { - if (requested.length === 0) { - throw new FileAccessError("path is empty; give a path relative to the working directory."); +function refusal( + requested: string, + verb: "read" | "write", + data: FilesFailureData | FileWriteFailureData | undefined, +): string { + if (data === undefined) { + return `cannot ${verb} "${requested}": ${reason(undefined)}.`; } - if (isAbsolute(requested)) { - throw new FileAccessError( - "an absolute path is not accepted; give a path relative to the working directory.", - ); + if (data.reason === "empty-path") { + return EMPTY; } - - const directory = yield* cwd(); - const lexical = resolve(directory, requested); - if (!within(directory, lexical)) { - throw new FileAccessError(`"${requested}" resolves outside the working directory.`); + if (data.reason === "absolute-path") { + return ABSOLUTE; } - - return { requested, lexical }; -} - -/** - * Stage two: the path as the filesystem currently has it. - * - * Resolves the part of the path that is already on disk — the file itself when - * it is there, the deepest existing ancestor when it is not — and re-checks - * the result, which is what catches a symlink pointing out of the workspace. - * What comes back is that resolved path, so an internal symlink is followed to - * the file it names rather than being replaced by the write. - * - * Both sides of the comparison are canonical here, so a working directory - * reached through a symlink — macOS's `/var` against `/private/var` — does not - * read as an escape. - */ -function* destination({ requested, lexical }: Admissible): Operation { - const directory = yield* cwd(); - const base = (yield* guard(requested, "resolve", realpath(directory))) ?? directory; - - const effective = yield* guard(requested, "resolve", resolveExisting(lexical)); - if (!within(base, effective)) { - throw new FileAccessError( - `"${requested}" leads through a symlink outside the working directory.`, - ); + if (data.reason === "lexical-escape") { + return `"${requested}" resolves outside the working directory.`; } - - return effective; -} - -/** - * Whether `path` names the working directory or something inside it. - * - * The directory itself is contained — `.` is not an escape. What it is instead - * is a directory, which is a question about the target rather than about - * containment, so it belongs to the read and write checks that come after. - * - * Only a complete `..` segment leaves. A name that merely starts with two dots - * — `..notes.md`, `..config/settings.json` — is an ordinary file inside, and a - * prefix test would refuse it. - */ -function within(base: string, path: string): boolean { - const rel = relative(base, path); - if (isAbsolute(rel)) { - return false; + if (data.reason === "resolved-escape") { + return `"${requested}" leads through a symlink outside the working directory.`; + } + if (data.phase === "resolution") { + return `cannot resolve "${requested}": ${reason(data.reason)}.`; } - if (rel.length === 0) { - return true; + if (data.phase === "target") { + return classified(requested, verb, data.reason); } - return rel !== ".." && !rel.startsWith(`..${sep}`); + return `cannot ${verb} "${requested}": ${reason(data.reason)}.`; } /** - * `path` with every symlink in its existing prefix resolved. + * What the target turned out to be, when that is the answer rather than an + * errno. * - * `realpath` needs the whole path to exist, and a write commonly names one - * that does not yet, so the walk gives up one trailing segment at a time until - * something answers and then puts the segments back. The working directory - * always exists, so the loop terminates there at the latest. + * A directory is not text, and saying so is better than whatever reading one + * produces on a given platform. "No such file" is the read's own phrasing of + * the same idea: the document asked for a file, and there is not one. */ -function* resolveExisting(path: string): Operation { - const trailing: string[] = []; - let current = path; - - while (true) { - const resolved = yield* realpath(current); - if (resolved !== undefined) { - return trailing.length === 0 ? resolved : join(resolved, ...trailing); +function classified(requested: string, verb: "read" | "write", why: FilesReason | undefined) { + if (verb === "read") { + if (why === "missing") { + return `cannot read "${requested}": no such file.`; } - const parent = dirname(current); - if (parent === current) { - return join(current, ...trailing); + if (why === "directory") { + return `cannot read "${requested}": it is a directory, not a text file.`; } - trailing.unshift(basename(current)); - current = parent; - } -} - -function* read(requested: string, target: string): Operation { - const info = yield* guard(requested, "read", stat(target)); - if (!info.exists) { - throw new FileAccessError(`cannot read "${requested}": no such file.`); - } - if (info.isDirectory) { - throw new FileAccessError(`cannot read "${requested}": it is a directory, not a text file.`); - } - if (!info.isFile) { - throw new FileAccessError(`cannot read "${requested}": it is not a regular file.`); - } - - return yield* guard(requested, "read", readTextFile(target)); -} - -/** - * Which step a write failed at, which is what decides what can be said about - * the target afterwards. - */ -type Step = "preparation" | "commit"; - -/** - * Replace `target` with exactly `text`. - * - * The content is whole by the time this runs — the children expanded first — - * so a child that failed never reaches the filesystem at all. What remains is - * the write, and it has one commit point. - * - * Everything up to the rename is preparation: a failure or a cancellation - * there leaves the previous file untouched, because nothing has replaced it - * yet. The rename is the commit, and it is a commit rather than a transaction — - * `rename` is a single filesystem call that cannot be interrupted once started, - * and a cancellation arriving after it has completed does not undo it. What is - * promised is that no write is ever half visible, not that a finished write can - * be taken back. - * - * A rename that *throws* is the one case where the outcome cannot be inferred. - * `rename` is an operation on the contextual Fs Api, and an `around` handler - * legitimately does work on both sides of `next()` — so a throw may arrive - * before the underlying rename ran or after it succeeded, and there is no way - * from here to tell which. What still holds is atomicity: the target is the - * complete previous content or the complete replacement, never a partial write. - * Claiming the previous file survived would be a guess, and it would be wrong - * exactly when a handler failed after committing. - * - * Removal of the temporary is registered before it is written rather than - * after. `writeTextFile` is where an interruption is most likely to land, and - * a cleanup installed on the far side of it would not run for the one failure - * it exists to handle. `remove` is forced, so registering it for a file that - * was never created — or one the rename has already consumed — is a no-op. - * - * A removal that does fail is reported, because a file the document did not - * create may be left in its directory. It is reported *alongside* whatever the - * write did rather than instead of it: both outcomes are collected here and - * raised together, so neither hides the other. - */ -function* write(requested: string, target: string, text: string): Operation { - const info = yield* guard(requested, "write", stat(target)); - if (info.exists && !info.isFile) { - throw new FileAccessError( - `cannot write "${requested}": it is a ${info.isDirectory ? "directory" : "special file"}, ` + - "not a text file.", - ); - } - - yield* guard(requested, "write", ensureDir(dirname(target))); - - // Both halves are collected rather than thrown, then reported together. A - // destructor that threw would replace the failure it is unwinding, or be - // aggregated into it in a shape a document cannot read; and the write's own - // failure must not hide the fact that a temporary was left behind. - const failed: string[] = []; - const uncleaned: string[] = []; - let step: Step = "preparation"; - - yield* scoped(function* () { - const temporary = `${target}.xmd-${randomUUID().slice(0, 8)}.tmp`; - yield* ensure(() => discard(requested, temporary, uncleaned)); - try { - yield* writeTextFile(temporary, text); - step = "commit"; - yield* rename(temporary, target); - } catch (error) { - failed.push(sentence(requested, "write", error)); + if (why === "special-file") { + return `cannot read "${requested}": it is not a regular file.`; } - }); - - if (failed.length === 0 && uncleaned.length === 0) { - return; } - throw new FileAccessError( - [ - ...failed, - ...uncleaned, - outcome(failed.length > 0 ? step : undefined), - ...(uncleaned.length > 0 ? [LEFTOVER] : []), - ].join(" "), - ); + if (verb === "write" && (why === "directory" || why === "special-file")) { + const kind = why === "directory" ? "directory" : "special file"; + return `cannot write "${requested}": it is a ${kind}, not a text file.`; + } + return `cannot ${verb} "${requested}": ${reason(why)}.`; } /** * What can be said about the target, given where the write stopped. * - * Only two of these are conclusions. A preparation failure changed nothing, - * and a rename that returned committed. A rename that threw is the honest - * "unknown": atomicity still holds, so the answer is one of two whole files, - * but which one is not observable from here. + * Only three of these are conclusions. Everything up to the commit changed + * nothing, a commit that returned committed, and a transaction that rolled its + * change back is back where it started. A commit that *threw* is the honest + * "unknown": the provider cannot tell whether the underlying replacement ran, so + * the answer is one of two whole files, but not which. + * + * The phases before the commit have no outcome sentence at all. Nothing was + * attempted on the target, so there is nothing to report about it beyond why the + * write did not start. */ -function outcome(failedAt: Step | undefined): string { - if (failedAt === undefined) { - return "The file was written."; - } - if (failedAt === "preparation") { - return "The previous file is unchanged."; - } - return ( +const OUTCOMES: ReadonlyMap = new Map([ + ["temporary", "The previous file is unchanged."], + [ + "commit", "Whether the replacement committed is unknown: the target holds either the " + - "complete previous content or the complete replacement, never a partial write." - ); -} + "complete previous content or the complete replacement, never a partial write.", + ], + ["cleanup", "The file was written."], + ["transaction", "The Workspace change was rolled back."], +]); /** - * Orthogonal to `outcome`: a temporary that could not be removed is a separate - * fact about the directory, and composes with any of the three. + * Orthogonal to the outcome: a temporary that could not be removed is a separate + * fact about the directory, and composes with any of them. */ const LEFTOVER = "A temporary file beside it may remain."; /** - * Remove the temporary, recording a sanitized sentence if it cannot be. + * The complete report for a failed write. * - * Runs during teardown, sometimes the teardown of a write that is already - * failing, so it records rather than throws — see `write`. What it records - * names the document's own path: the temporary is generated, and a document - * that never chose that name cannot be shown it. + * The write's own failure and a failed cleanup are both reported, in that order, + * because neither may hide the other: one says what is known about the target, + * the other that something was left behind, and a reader needs both to know what + * the directory now holds. The outcome sentence follows them, and the leftover + * warning follows that. */ -function* discard(requested: string, temporary: string, uncleaned: string[]): Operation { - try { - yield* remove(temporary, { force: true }); - } catch (error) { - uncleaned.push(sentence(requested, "clean up", error)); +function report(requested: string, data: FileWriteFailureData): string { + const parts: string[] = []; + if (data.reason !== undefined) { + parts.push(refusal(requested, "write", data)); + } + if (data.cleanup !== undefined) { + parts.push(`cannot clean up "${requested}": ${reason(data.cleanup)}.`); + } + const outcome = OUTCOMES.get(data.phase); + if (outcome !== undefined) { + parts.push(outcome); + } + if (data.cleanup !== undefined) { + parts.push(LEFTOVER); } + return parts.join(" "); } diff --git a/packages/core/src/components/Glob.ts b/packages/core/src/components/Glob.ts index 94f8845e..7c0d8436 100644 --- a/packages/core/src/components/Glob.ts +++ b/packages/core/src/components/Glob.ts @@ -16,17 +16,17 @@ * be the answer to "that pattern was a mistake". The same stage refuses an * empty pattern, which matches nothing by construction. * - * Everything else about matching belongs to the Fs Api's `glob`, which is the - * dialect. This component adds no syntax of its own, which is why a leading dot - * needs no special prop: `*` matches one, so a pattern that names a hidden file - * finds it and a pattern that does not, does not. + * Everything else about matching belongs to the `API.Files` provider, which is + * the dialect and owns the whole search. This component adds no syntax of its + * own, which is why a leading dot needs no special prop: `*` matches one, so a + * pattern that names a hidden file finds it and a pattern that does not, does + * not. * * Only regular files come back. A symbolic link is a link rather than a file, * so it is never a result and a link to a directory is never descended into — - * which is also what keeps traversal inside `Env.cwd` and free of cycles. The - * Fs Api exposes symlink following, but nothing there confines a resolved - * destination to the root or detects a traversal cycle, so following one cannot - * be offered safely yet. + * which is also what keeps traversal inside `Env.cwd` and free of cycles. + * Following one cannot be offered safely yet: nothing here confines a resolved + * destination to the root or detects a traversal cycle. * * Printed errors name the pattern the document wrote, or nothing at all. A * traversal failure names no path: what failed is a directory somewhere under @@ -40,10 +40,11 @@ * the search runs again against whatever is on disk now. */ -import { isAbsolute } from "node:path"; import type { Operation } from "effection"; import { printErrors } from "../component-failures.ts"; -import { cwd, glob, stat } from "@executablemd/runtime"; +import { cwd, parseFilesFailure } from "@executablemd/runtime"; +import type { FilesFailureData } from "@executablemd/runtime"; +import { globFiles } from "../files.ts"; import type { Json } from "../types.ts"; import { reason } from "./fs-error-phrases.ts"; @@ -78,14 +79,11 @@ export default printErrors(function* (props: Record): Operation entry.isFile).map((entry) => entry.path); - - return [...new Set(files)].sort(byCodePoint); + const found = yield* globFiles({ cwd: yield* cwd(), include, exclude }); + if (!found.ok) { + throw failed(parseFilesFailure(found.error), [...include, ...exclude]); + } + return found.value; }); /** @@ -117,7 +115,7 @@ function patterns(prop: string, value: Json | undefined): string[] { "give a pattern relative to the working directory.", ); } - if (isAbsolute(pattern)) { + if (absolute(pattern)) { throw new GlobError( `${prop} pattern "${pattern}" is absolute; ` + "give a pattern relative to the working directory.", @@ -132,95 +130,44 @@ function patterns(prop: string, value: Json | undefined): string[] { } /** - * The contextual working directory, once it is known to be searchable. + * Whether a pattern names an absolute location. * - * Checked here rather than left to the traversal, because the two failures read - * very differently to an author: a directory that is missing or is a file is - * something about the document's own environment, while a traversal failure is - * something about one entry inside a directory that was fine. + * Decided from the pattern's own grammar rather than the running platform's. + * Patterns match POSIX-relative paths on every host — that is what makes one + * document mean one thing everywhere — so a leading `/` is absolute wherever + * this runs, and so is a drive-letter prefix, which is absolute on the host that + * has drives and matches nothing on the hosts that do not. A leading backslash + * is left alone: in this dialect it escapes the character after it. */ -function* directory(): Operation { - const root = yield* cwd(); - - const info = yield* guard(stat(root)); - if (!info.exists) { - throw new GlobError("the working directory does not exist."); - } - if (!info.isDirectory) { - throw new GlobError("the working directory is not a directory."); - } - - return root; -} - -interface Search { - root: string; - patterns: string[]; - exclude: string[]; +function absolute(pattern: string): boolean { + return pattern.startsWith("/") || /^[A-Za-z]:[\\/]/.test(pattern); } /** - * Run the search, reporting a pattern that cannot be compiled as the authoring - * error it is. + * One sanitized sentence for a failed search. * - * The Api compiles patterns as it starts, so an unusable one — an unterminated - * character class — arrives as a `SyntaxError` from `RegExp` rather than as an - * errno. It is separated from a traversal failure because it is the one failure - * here the document can fix by editing a pattern, and because a `RegExp` - * message describes a translated regular expression the author never wrote. + * The two questions an author can act on are separated from the rest. A working + * directory that is missing or is a file is something about the document's own + * environment; a pattern the dialect cannot compile — an unterminated character + * class — is something about the document's own text. Which pattern it was does + * not survive the provider boundary, so the sentence lists the candidates + * instead of naming one. They are the document's own text. * - * Which pattern it was is not recoverable from the error, so the sentence lists - * the candidates instead of naming one. They are the document's own text. + * Everything else names no path. What failed is the working directory or + * something under it, and both are absolute paths the document did not write + * (§1.2). */ -function* search(options: Search): Operation> { - try { - return yield* glob(options); - } catch (error) { - if (error instanceof SyntaxError) { - const all = [...options.patterns, ...options.exclude]; - throw new GlobError( - `one of these patterns cannot be used: ${all.map((p) => `"${p}"`).join(", ")}.`, - ); - } - throw failed(error); +function failed(data: FilesFailureData | undefined, candidates: string[]): GlobError { + if (data?.phase === "target" && data.reason === "missing") { + return new GlobError("the working directory does not exist."); } -} - -/** - * Run a filesystem operation, replacing whatever it throws with a sanitized - * sentence. - * - * Nothing is passed through, including a `GlobError`: this component's own - * checks throw outside guarded calls, so an error surfacing from inside one came - * from the Api, and an error's class says nothing about whether its message is - * safe to show. - */ -function* guard(operation: Operation): Operation { - try { - return yield* operation; - } catch (error) { - throw failed(error); - } -} - -/** - * One sanitized sentence for a failed filesystem call. - * - * It names no path. What failed is the working directory or something under it, - * and both are absolute paths the document did not write (§1.2). - */ -function failed(error: unknown): GlobError { - return new GlobError(`cannot search the working directory: ${reason(error)}.`); -} - -// Code point order, not `localeCompare`: what a document branches on must not -// depend on the locale the host happens to be configured with. -function byCodePoint(left: string, right: string): number { - if (left < right) { - return -1; + if (data?.phase === "target" && data.reason === "not-directory") { + return new GlobError("the working directory is not a directory."); } - if (left > right) { - return 1; + if (data?.phase === "pattern") { + return new GlobError( + `one of these patterns cannot be used: ${candidates.map((p) => `"${p}"`).join(", ")}.`, + ); } - return 0; + return new GlobError(`cannot search the working directory: ${reason(data?.reason)}.`); } diff --git a/packages/core/src/components/TempDir.ts b/packages/core/src/components/TempDir.ts index ea08fb68..3c61f7c1 100644 --- a/packages/core/src/components/TempDir.ts +++ b/packages/core/src/components/TempDir.ts @@ -6,19 +6,24 @@ * finishes, fails, or is cancelled. Written self-closing it is an allocation: * the directory is retained at the invocation site so a later sibling can use * the path it renders, and removed with that scope. + * + * The directory comes from the installed `API.Files` provider, so this + * component neither creates nor removes anything itself. A provider that has no + * temporary directories to give — one whose whole filesystem is a database + * transaction — refuses the operation outright, and that refusal is fatal + * rather than a printed error: there is no directory to run inside, so the + * content must not run and the siblings after it must not proceed as though it + * had. */ -import { ensure, resource } from "effection"; import type { Operation } from "effection"; import { printErrors } from "../component-failures.ts"; -import { rm } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, parseFilesFailure } from "@executablemd/runtime"; import { ReplayGuard, StaleInputError } from "@executablemd/durable-streams"; -import { mkdtempSync, realpathSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; import { content, retain } from "../component-api.ts"; import { hasContent } from "../content-context.ts"; +import { temporaryDirectory } from "../files.ts"; +import { reason } from "./fs-error-phrases.ts"; export const props = { type: "object", @@ -26,33 +31,34 @@ export const props = { additionalProperties: false, }; +/** A temporary directory that could not be created. */ +export class TempDirError extends Error { + constructor(message: string) { + super(message); + this.name = "TempDirError"; + } +} + /** - * A directory this call created, named by its canonical path. - * - * Creation is synchronous so that nothing can suspend between it and the - * `ensure` that removes it. `until()` cannot cancel the promise it is waiting - * on, so an asynchronous `mkdtemp` halted mid-flight would go on to create a - * directory after the generator had already stopped — one nothing owns and - * nothing removes. Reading the directory does not suspend either, so the whole - * acquisition is a single uninterruptible step. + * A directory that lives as long as the acquiring scope. * - * `mkdtemp` names and creates at once, so the directory is never one an - * earlier run left behind. The path is then canonicalized: on macOS `tmpdir()` - * is a symlink (`/var/folders/…`) while a child process resolves it - * (`/private/var/…`), and canonicalizing is what makes the rendered path, - * `Env.cwd`, and a subprocess's own `cwd` the same string. - * - * `@effectionx/fs` has neither operation, so both come from Node. + * The provider owns creation and removal as one acquisition, so nothing can + * land between them and leave a directory nobody holds. What this adds is the + * unwrapping: an ordinary acquisition failure is a printed error naming no path + * — the directory is generated and the document never chose its name — while a + * provider that refuses the operation has already thrown past here. * * Exported for the lifetime tests, which drive acquisition directly. Not part * of the package's public surface — `mod.ts` does not re-export it. */ -export function useTemporaryDirectory(): Operation { - return resource(function* (provide) { - const created = mkdtempSync(join(tmpdir(), "xmd-tempdir-")); - yield* ensure(() => rm(created, { recursive: true, force: true })); - yield* provide(realpathSync(created)); - }); +export function* useTemporaryDirectory(): Operation { + const acquired = yield* temporaryDirectory(); + if (!acquired.ok) { + throw new TempDirError( + `cannot create a temporary directory: ${reason(parseFilesFailure(acquired.error)?.reason)}.`, + ); + } + return acquired.value; } /** diff --git a/packages/core/src/components/fs-error-phrases.ts b/packages/core/src/components/fs-error-phrases.ts index 2d3b59be..4a1506f7 100644 --- a/packages/core/src/components/fs-error-phrases.ts +++ b/packages/core/src/components/fs-error-phrases.ts @@ -1,64 +1,54 @@ /** - * The vocabulary a failed filesystem call may be reported in. + * The vocabulary a failed filesystem operation may be reported in. * - * Shared by the components that touch the filesystem — `` (§6.13) and + * Shared by the components that reach the filesystem — `` (§6.13) and * `` (§6.14) — because the constraint is the same for both. A platform * error message names the path it failed on: `ENOTDIR: not a directory, stat * '/private/var/…'`. That is the resolved path §1.2 keeps out of printed errors, * and it can be a path the document never wrote — a generated temporary, or a * file somewhere under a directory that was only ever named by a pattern. * - * So nothing from the caught error is reproduced. The errno code **selects** a - * phrase written here, and an unrecognized code selects the generic one. The - * code itself is never emitted, because whatever implements the Fs Api chooses - * it and can put a path, markup, or a newline there as easily as `ENOENT`. + * So nothing from the failure is reproduced. The provider has already reduced + * whatever its platform produced to a `FilesReason`, and that reason **selects** + * a phrase written here. An unrecognized reason selects the generic one, which + * is also what a provider reports when it could not classify the condition. */ +import type { FilesReason } from "@executablemd/runtime"; + /** * Every phrase, and the whole allowlist. * - * A `Map` rather than an object, because a lookup on an object literal answers - * for inherited keys — `REASONS["toString"]` would hand back a function whose - * source would then be interpolated into a printed error. + * A `Map` rather than an object literal, because a lookup on one answers for + * inherited keys — `PHRASES["toString"]` would hand back a function whose source + * would then be interpolated into a printed error. + * + * The reasons that are not here are the ones a component has a sentence of its + * own for: the lexical refusals, the containment escapes, a target that is a + * directory or a special file, and a pattern that cannot be compiled. */ -const REASONS: ReadonlyMap = new Map([ - ["ENOENT", "no such file or directory"], - ["ENOTDIR", "a component of the path is not a directory"], - ["EISDIR", "it is a directory"], - ["ENOTEMPTY", "the directory is not empty"], - ["EACCES", "permission denied"], - ["EPERM", "permission denied"], - ["EROFS", "the filesystem is read-only"], - ["ELOOP", "too many levels of symbolic links"], - ["ENAMETOOLONG", "the path is too long"], - ["ENOSPC", "no space left on the device"], - ["EDQUOT", "the disk quota is exhausted"], - ["EXDEV", "the destination is on a different filesystem"], - ["EBUSY", "the file is in use"], - ["EMFILE", "too many open files"], +const PHRASES: ReadonlyMap = new Map([ + ["missing", "no such file or directory"], + ["not-directory", "a component of the path is not a directory"], + ["directory", "it is a directory"], + ["directory-not-empty", "the directory is not empty"], + ["permission-denied", "permission denied"], + ["read-only", "the filesystem is read-only"], + ["too-many-symlinks", "too many levels of symbolic links"], + ["path-too-long", "the path is too long"], + ["no-space", "no space left on the device"], + ["quota-exhausted", "the disk quota is exhausted"], + ["cross-device", "the destination is on a different filesystem"], + ["busy", "the file is in use"], + ["too-many-open-files", "too many open files"], ]); const UNRECOGNIZED = "the filesystem operation failed"; -function errorCode(error: unknown): string | undefined { - if (typeof error !== "object" || error === null || !("code" in error)) { - return undefined; - } - const { code } = error; - return typeof code === "string" ? code : undefined; -} - -/** - * A phrase for a failure, chosen from `REASONS` or the generic one. - * - * `code` is attacker-supplied as far as a component is concerned — anything - * installed as Fs middleware can put a path, markup, or a newline in it — so it - * is used to select a phrase and never to build one. - */ -export function reason(error: unknown): string { - const code = errorCode(error); - if (code === undefined) { +/** A phrase for a failure, chosen from `PHRASES` or the generic one. */ +export function reason(value: FilesReason | undefined): string { + if (value === undefined) { return UNRECOGNIZED; } - return REASONS.get(code) ?? UNRECOGNIZED; + return PHRASES.get(value) ?? UNRECOGNIZED; } diff --git a/packages/core/src/errors.ts b/packages/core/src/errors.ts index aa9f508b..15807028 100644 --- a/packages/core/src/errors.ts +++ b/packages/core/src/errors.ts @@ -7,6 +7,8 @@ import { StaleInputError, TerminalDivergenceError, } from "@executablemd/durable-streams"; +import { asFilesFatal } from "@executablemd/runtime"; +import type { FilesFatalFailure } from "@executablemd/runtime"; import { InvocationTeardownError } from "./invocation.ts"; import type { ErrorSegment } from "./types.ts"; @@ -192,13 +194,13 @@ export type DurabilityFailure = | ContinuePastCloseDivergenceError; /** A failure that ends the execution rather than becoming a printed error. */ -export type FatalFailure = DocumentationError | DurabilityFailure; +export type FatalFailure = DocumentationError | DurabilityFailure | FilesFatalFailure; /** * The error that ends the execution, if this failure carries one. * * Expansion turns a failure into a printed error the document can render, which - * is right for anything the document itself got wrong. Two kinds are not that, + * is right for anything the document itself got wrong. Three kinds are not that, * and every generic catch in the engine rethrows them: * * - `DocumentationError` — the error mode has already decided this execution @@ -212,31 +214,54 @@ export type FatalFailure = DocumentationError | DurabilityFailure; * to a note. It would also bury *where* the journal stopped describing the * run: expansion that carried on would reach another durable operation, whose * own mismatch is then the one reported. + * - a **Files infrastructure failure** — no document filesystem provider is + * installed, one refused an operation outright, or one broke its own + * contract. None of those is a filesystem condition a document could have + * caused or could act on, and a provider that is not there must not become a + * printed comment that lets later siblings run as if the file work had + * happened. * * This looks through the three ways the engine and the platform aggregate * failures and returns the fatal error itself rather than the wrapper, which is - * the one worth reporting. The two kinds travel differently: a durability - * failure stays fatal through the complete cause graph, while a + * the one worth reporting. The kinds travel differently: durability and Files + * failures stay fatal through the complete cause graph, while a * `DocumentationError` remains fatal unless it crossed an explicit * `ContentError` recovery boundary — a component that recovered decided what * the document reports. An uncaught private content transport restores its * original `DocumentationError` explicitly at the function-component boundary, * not through this traversal. * - * **A durability failure outranks a documentation failure**, wherever each sits - * in the graph. A wrapper carries whatever failed together, in whatever order - * the platform happened to collect it, and one of those orders would otherwise - * report the document's failure and let the loop record an `error` outcome onto - * a journal already known not to describe this run. Precedence is therefore - * decided by kind, not by position: the graph is searched for a durability - * failure first, and only a graph without one reports a documentation failure. + * **Precedence is durability, then Files, then documentation**, wherever each + * sits in the graph. A wrapper carries whatever failed together, in whatever + * order the platform happened to collect it, and one of those orders would + * otherwise report the document's failure and let the loop record an `error` + * outcome onto a journal already known not to describe this run. Precedence is + * therefore decided by kind, not by position: each kind is searched for across + * the whole graph in turn, so nesting and aggregate member order cannot change + * the answer. * - * The two searches reach different parts of the same graph. A content failure - * ends the documentation search and not the durability one — see - * `isRecoveredContent` for why the asymmetry is the point. + * The searches reach different parts of the same graph. A content failure ends + * the documentation search and neither of the others — see `isRecoveredContent` + * for why the asymmetry is the point. */ export function fatalCause(error: unknown): FatalFailure | undefined { - return durabilityFailure(error) ?? documentationFailure(error); + return durabilityFailure(error) ?? filesFatalFailure(error) ?? documentationFailure(error); +} + +/** + * The Files infrastructure failure this one carries, if any. + * + * Recognized structurally rather than with `instanceof`: two copies of the + * runtime package can be loaded at once, and a provider failure constructed by + * the other one has to be found on the same terms as one constructed here. + * + * Like durability discovery, this walks the whole graph and treats no node as a + * leaf. A `ContentError` in the chain says a component recovered from failed + * content and reported a failure of its own; that decides which *document* + * failure is reported, and a missing provider is not a document failure. + */ +export function filesFatalFailure(error: unknown): FilesFatalFailure | undefined { + return firstCause(error, asFilesFatal); } /** @@ -260,8 +285,9 @@ export function documentationFailure(error: unknown): DocumentationError | undef * whether the document gets to read what happened. * * An `output` decision says yes — that is the whole difference between the mode - * a region installs and the mode documentation installs. A `throw` decision and - * a durability failure say no. + * a region installs and the mode documentation installs. A `throw` decision, a + * durability failure, and a Files infrastructure failure all say no: none of + * them is a decision about what the document reports. */ export function decidedByOutput(failure: FatalFailure): boolean { return failure instanceof DocumentationError && failure.mode === "output"; @@ -318,12 +344,13 @@ type OpaqueFailure = (error: object) => boolean; * replaced — the component's contextual printed error would be built and then * discarded in favour of the child's. * - * Durability discovery looks straight through it. Only the engine's own + * Durability and Files discovery look straight through it. Only the engine's own * projection failures are known to carry a documentation failure and nothing * else; this class is public, so an author constructs and subclasses it and may * put anything underneath — and a durability failure stays fatal however it is - * wrapped (§6.11). Recovery decides which failure the document *reports*, and a - * durability failure is not one of the things a document reports. + * wrapped (§6.11). Recovery decides which failure the document *reports*, and + * neither a durability failure nor a missing filesystem provider is one of the + * things a document reports. */ function isRecoveredContent(error: object): boolean { return error instanceof ContentError; diff --git a/packages/core/src/files.ts b/packages/core/src/files.ts new file mode 100644 index 00000000..8a92c3a7 --- /dev/null +++ b/packages/core/src/files.ts @@ -0,0 +1,112 @@ +/** + * The engine's one door to `API.Files`. + * + * Every document filesystem operation a component performs goes through here, + * because the contract has a part no component can enforce on its own: a + * provider is allowed to *fail*, and it is not allowed to *throw*. An ordinary + * filesystem condition comes back as `Err` with structural data; anything that + * throws is an installation or provider-contract failure and must end the + * execution rather than become something a document renders. + * + * Deciding that in one place is what makes the rule hold. A component that + * caught a throw would have to guess whether the thing it caught was safe to + * report, and an error's class is no evidence about its message — which is + * exactly the guess `` and `` already refuse to make about the + * platform. + * + * This lives in core rather than in the runtime package because the ordering + * below reaches durability failures, which belong to `@executablemd/durable- + * streams`. Core already depends on both; the runtime must not acquire a + * dependency on durable-streams to answer a question only the engine asks. + */ + +import { + Files, + FilesInvariantError, + parseFileWriteFailure, + parseFileWriteSuccess, +} from "@executablemd/runtime"; +import type { + FilePathInput, + FileWriteFailureData, + FileWriteInput, + FileWriteSuccess, + GlobInput, +} from "@executablemd/runtime"; +import type { Operation, Result } from "effection"; +import { durabilityFailure, filesFatalFailure } from "./errors.ts"; + +/** + * Perform one provider call, converting an illegal throw into a failure the + * engine already knows how to fence. + * + * The search order is the one thing here that is not obvious. A provider call + * can happen underneath work that has *already* failed — a durability failure + * unwinding through a component's teardown, or a Files failure from a nested + * operation — and the first failure is the one that describes what went wrong. + * Replacing it with a fresh invariant would report the symptom and lose the + * cause, and for a durability failure it would also lose the identity that + * #394's fail-stop records as "the first error". + * + * So: an existing durability failure is rethrown as it stands, then an existing + * Files infrastructure failure, and only something that is neither becomes a new + * `protocol` invariant. That last one carries no cause, message, errno text, or + * host value — a handler that threw an arbitrary object is precisely the case + * where nothing it produced can be trusted. + * + * Cancellation does not arrive here. Halting resumes a generator through + * `return()` rather than by throwing, so no `catch` in Effection converts a + * cancellation into a failure. + */ +export function* invokeFiles(call: Operation): Operation { + try { + return yield* call; + } catch (error) { + throw ( + durabilityFailure(error) ?? filesFatalFailure(error) ?? new FilesInvariantError("protocol") + ); + } +} + +export function checkFilePath(input: FilePathInput): Operation> { + return invokeFiles(Files.operations.checkFilePath(input)); +} + +export function readFileText(input: FilePathInput): Operation> { + return invokeFiles(Files.operations.readTextFile(input)); +} + +export function writeFileText(input: FileWriteInput): Operation> { + return invokeFiles(Files.operations.writeTextFile(input)); +} + +export function globFiles(input: GlobInput): Operation> { + return invokeFiles(Files.operations.globFiles(input)); +} + +export function temporaryDirectory(): Operation> { + return invokeFiles(Files.operations.temporaryDirectory()); +} + +/** + * What a write reported, or a fatal failure if it reported nothing readable. + * + * A write is the one operation whose result makes a claim about the world: the + * file was replaced, or it was not, or nobody can tell. Data that does not + * validate leaves no safe sentence to print — every candidate asserts one of + * those three — so a provider that cannot describe what it did is treated as + * one that may not have done it. `undefined` is not a possible return. + */ +export function writeReport(result: Result): FileWriteFailureData | undefined { + if (result.ok) { + if (parseFileWriteSuccess(result.value) === undefined) { + throw new FilesInvariantError("protocol"); + } + return undefined; + } + const failure = parseFileWriteFailure(result.error); + if (failure === undefined) { + throw new FilesInvariantError("protocol"); + } + return failure; +} diff --git a/packages/core/tests/component-registration.test.ts b/packages/core/tests/component-registration.test.ts index def79cff..a54b845f 100644 --- a/packages/core/tests/component-registration.test.ts +++ b/packages/core/tests/component-registration.test.ts @@ -12,7 +12,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, resource, scoped, spawn, until } from "effection"; import type { Operation } from "effection"; import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { mkdtemp, realpath } from "node:fs/promises"; @@ -107,6 +107,9 @@ function* thrown(body: () => Operation): Operation { function run(dir: string, componentDirs: string[] = [dir]): Operation { return scoped(function* () { + // `API.Files` has no host default, and `` is one of the components + // these cases resolve. + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(), componentDirs }), ); @@ -575,6 +578,7 @@ describe("Tier CR — selection is journaled", () => { function runOn(dir: string, stream: InMemoryStream): Operation { return scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); diff --git a/packages/core/tests/fatal-cause.test.ts b/packages/core/tests/fatal-cause.test.ts index ba375fa8..5a7aefd7 100644 --- a/packages/core/tests/fatal-cause.test.ts +++ b/packages/core/tests/fatal-cause.test.ts @@ -18,8 +18,21 @@ import { StaleInputError, TerminalDivergenceError, } from "@executablemd/durable-streams"; +import { + FILES_FATAL, + FilesInvariantError, + FilesOperationDeniedError, + FilesProviderUnavailableError, +} from "@executablemd/runtime"; import { InvocationTeardownError } from "../src/invocation.ts"; -import { ContentError, DocumentationError, durabilityFailure, fatalCause } from "../src/errors.ts"; +import { + ContentError, + decidedByOutput, + DocumentationError, + durabilityFailure, + fatalCause, + filesFatalFailure, +} from "../src/errors.ts"; import { Component } from "../src/component-api.ts"; import { expandSegments } from "../src/expand.ts"; import { renderSegments } from "../src/render.ts"; @@ -79,6 +92,39 @@ const DURABILITY_FAILURES: Array<() => Error> = [ () => new DurablePersistenceError("yield", new Error("journal unavailable")), ]; +/** + * One of each Files infrastructure failure. + * + * All three are structurally tagged and carry no cause, so what a failing + * assertion has to distinguish is the kind, not the class. + */ +const FILES_FAILURES: Array<() => Error> = [ + () => new FilesProviderUnavailableError(), + () => new FilesOperationDeniedError("temporary-directory"), + () => new FilesInvariantError("authority"), + () => new FilesInvariantError("savepoint"), + () => new FilesInvariantError("protocol"), + () => new FilesInvariantError("teardown"), +]; + +/** + * A Files failure as a **separately loaded copy** of the runtime package would + * build it. + * + * Two copies can be resolved at once — a repository component reaching its own + * runtime beside the engine's — and `instanceof` answers false across them. The + * structural tag is the whole mechanism, so this is built by hand rather than + * through the constructor: nothing about this object shares a class identity + * with the one core imported. + */ +function foreignFilesFatal(): Error { + const error = new Error("Files provider is not installed"); + error.name = "FilesProviderUnavailableError"; + return Object.assign(error, { + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }); +} + describe("Tier FA — Fatal error discovery", () => { it("FA1: an error that is its own cause terminates the search", function* () { const error = new Error("ordinary"); @@ -316,4 +362,149 @@ describe("Tier FA — Fatal error discovery", () => { expect(durabilityFailure(cyclic)).toBe(planted); expect(fatalCause(wrapper)).toBe(planted); }); + + // FA20–FA27: a Files infrastructure failure is the third fatal kind. It is + // not something the document did and not something it can act on, so it + // travels like a durability failure — through every wrapper, and through a + // recovery boundary — and ranks between the two existing kinds. + it("FA20: every Files infrastructure failure is discovered as fatal", function* () { + for (const make of FILES_FAILURES) { + const planted = make(); + expect(fatalCause(planted)).toBe(planted); + expect(filesFatalFailure(planted)).toBe(planted); + + const wrapped = new AggregateError([planted], "wrapped"); + expect(fatalCause(wrapped)).toBe(planted); + expect(filesFatalFailure(wrapped)).toBe(planted); + + // Not a durability failure: the two questions stay separate. + expect(durabilityFailure(planted)).toBeUndefined(); + } + }); + + it("FA21: a Files failure is found through every wrapper the engine builds", function* () { + const planted = new FilesProviderUnavailableError(); + + expect(fatalCause(new InvocationTeardownError([planted]))).toBe(planted); + expect(fatalCause(new AggregateError([new Error("other"), planted]))).toBe(planted); + expect(fatalCause(new Error("wrapper", { cause: planted }))).toBe(planted); + + const deep = new InvocationTeardownError([ + new AggregateError( + [new Error("noise"), new Error("component exploded", { cause: planted })], + "mixed", + ), + ]); + expect(fatalCause(deep)).toBe(planted); + }); + + it("FA22: a Files failure outranks a documentation failure in either order", function* () { + for (const make of FILES_FAILURES) { + const planted = make(); + const doc = documentation(); + + expect(fatalCause(new AggregateError([doc, planted], "mixed"))).toBe(planted); + expect(fatalCause(new AggregateError([planted, doc], "mixed"))).toBe(planted); + expect(fatalCause(new InvocationTeardownError([doc, planted]))).toBe(planted); + expect(fatalCause(new InvocationTeardownError([planted, doc]))).toBe(planted); + } + }); + + it("FA23: a durability failure outranks a Files failure in either order", function* () { + for (const make of DURABILITY_FAILURES) { + const durability = make(); + const files = new FilesInvariantError("protocol"); + + expect(fatalCause(new AggregateError([files, durability], "mixed"))).toBe(durability); + expect(fatalCause(new AggregateError([durability, files], "mixed"))).toBe(durability); + } + }); + + it("FA24: all three kinds at once resolve durability, then Files, then documentation", function* () { + const durability = stale(); + const files = new FilesInvariantError("savepoint"); + const doc = documentation(); + + // Every order of the three, and the answer never moves. + for (const members of [ + [durability, files, doc], + [doc, files, durability], + [files, doc, durability], + [durability, doc, files], + ]) { + expect(fatalCause(new AggregateError(members, "mixed"))).toBe(durability); + } + + expect(fatalCause(new AggregateError([doc, files], "mixed"))).toBe(files); + expect(fatalCause(new AggregateError([files, doc], "mixed"))).toBe(files); + + // Nesting cannot change it either: the shallowest member loses to the kind. + const nested = new AggregateError( + [doc, new InvocationTeardownError([new AggregateError([files], "inner")])], + "outer", + ); + expect(fatalCause(nested)).toBe(files); + }); + + it("FA25: a content failure does not hide a Files failure it carries", function* () { + for (const make of FILES_FAILURES) { + const planted = make(); + expect(filesFatalFailure(recovered(planted))).toBe(planted); + expect(fatalCause(recovered(planted))).toBe(planted); + expect(fatalCause(new AuthorContentError(planted))).toBe(planted); + } + + // And against the documentation failure a recovery boundary would otherwise + // let the search stop at. + const files = new FilesInvariantError("teardown"); + const contextual = new Error("component exploded", { + cause: recovered(new AggregateError([documentation(), files], "content")), + }); + expect(fatalCause(contextual)).toBe(files); + }); + + it("FA26: a cyclic graph carrying a Files failure terminates and still finds it", function* () { + const planted = new FilesProviderUnavailableError(); + const noise = new Error("noise"); + const teardown = new InvocationTeardownError([noise, planted]); + noise.cause = teardown; + + expect(fatalCause(teardown)).toBe(planted); + + const cyclic = recovered(); + const wrapper = new AggregateError([cyclic, planted], "mixed"); + cyclic.cause = wrapper; + expect(filesFatalFailure(cyclic)).toBe(planted); + }); + + it("FA27: a Files failure from a separately loaded runtime copy is recognized", function* () { + const foreign = foreignFilesFatal(); + + // The mechanism, stated as a fact about this object: no class identity. + expect(foreign instanceof FilesProviderUnavailableError).toBe(false); + + expect(filesFatalFailure(foreign)).toBe(foreign); + expect(fatalCause(new InvocationTeardownError([documentation(), foreign]))).toBe(foreign); + + // A tag that is not this one is not recognized, so recognition is the tag + // rather than the presence of a `data` member. + const impostor = Object.assign(new Error("looks similar"), { + data: { type: "some.other/v1", kind: "provider-unavailable" }, + }); + expect(filesFatalFailure(impostor)).toBeUndefined(); + expect(fatalCause(impostor)).toBeUndefined(); + }); + + it("FA28: only an output-mode documentation failure is decided by output", function* () { + const output = new DocumentationError({ type: "error", message: "wrong" }, "output"); + expect(decidedByOutput(output)).toBe(true); + expect(decidedByOutput(documentation())).toBe(false); + + for (const make of FILES_FAILURES) { + expect(decidedByOutput(make())).toBe(false); + } + for (const make of DURABILITY_FAILURES) { + expect(decidedByOutput(make())).toBe(false); + } + }); }); diff --git a/packages/core/tests/file-component.test.ts b/packages/core/tests/file-component.test.ts index f1c85a0e..4902f582 100644 --- a/packages/core/tests/file-component.test.ts +++ b/packages/core/tests/file-component.test.ts @@ -17,7 +17,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, race, resource, scoped, sleep, suspend, until } from "effection"; import type { Operation } from "effection"; import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { execute } from "../src/execute.ts"; @@ -62,11 +62,16 @@ function useFixture(): Operation { } /** - * Install the workspace as the contextual working directory. + * Install the workspace as the contextual working directory, and the host + * document filesystem provider beneath it. * * The document lives in the workspace too, but nothing depends on that: * `` resolves against `Env.cwd`, which this installs explicitly rather * than inheriting from the process. + * + * `API.Files` has no host default, so a suite driving `execute()` directly + * installs the provider the way an entrypoint does. Everything below then + * exercises the real host adapter rather than a stand-in. */ function* useWorkspaceCwd(fixture: Fixture): Operation { yield* API.Env.around( @@ -78,6 +83,7 @@ function* useWorkspaceCwd(fixture: Fixture): Operation { }, { at: "min" }, ); + yield* useHostFiles(); } function run(fixture: Fixture, source: string): Operation { diff --git a/packages/core/tests/files-fatal.test.ts b/packages/core/tests/files-fatal.test.ts new file mode 100644 index 00000000..093153ea --- /dev/null +++ b/packages/core/tests/files-fatal.test.ts @@ -0,0 +1,499 @@ +/** + * Tier FF — Files infrastructure failure (spec §§6.9, 6.11, 6.13, 6.14). + * + * Every other filesystem tier asks what a document reads when something goes + * wrong. This one asks the opposite question: when a document filesystem + * *provider* is missing, refuses an operation, or breaks its own contract, + * nothing is written into the document at all. The execution ends. + * + * The distinction matters because the two are easy to confuse from inside a + * component. "No such file" is something the document did and can fix; "no + * filesystem provider is installed" is not, and printing it as a comment would + * let every sibling after it run as though the file work had happened. + * + * These drive the real components through `execute()` and install providers + * that misbehave in exactly one way each. + */ + +import { describe, it, beforeAll } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, Err, Ok, resource, scoped, until } from "effection"; +import type { Operation, Result } from "effection"; +import { exists, rm, writeTextFile } from "@effectionx/fs"; +import { + API, + FILES_ERROR, + FILES_WRITE_SUCCESS, + Files, + FilesError, + FilesInvariantError, + FilesOperationDeniedError, + parseFilesFatal, + useHostFiles, +} from "@executablemd/runtime"; +import type { FilePathInput, FileWriteInput, FileWriteSuccess } from "@executablemd/runtime"; +import { InMemoryStream, StaleInputError } from "@executablemd/durable-streams"; +import { execute } from "../src/execute.ts"; +import { collect } from "../src/collect.ts"; +import { useTempFileCompiler } from "../src/temp-file-compiler.ts"; +import { fatalCause, filesFatalFailure } from "../src/errors.ts"; +import { invokeFiles } from "../src/files.ts"; +import { mkdtemp, realpath } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +function useFixture(): Operation { + return resource(function* (provide) { + const dir = yield* until(mkdtemp(join(tmpdir(), "ff-test-"))); + yield* ensure(() => rm(dir, { recursive: true, force: true })); + yield* provide(yield* until(realpath(dir))); + }); +} + +interface Outcome { + ok: boolean; + error: unknown; + output: string; +} + +/** + * Run `source` as a document, reporting both what it rendered and how the + * execution ended. + * + * Both halves are needed here: a fatal failure is distinguished from a printed + * error precisely by leaving nothing in the output, and by stopping what comes + * after it. + */ +function run(dir: string, source: string, install?: () => Operation): Operation { + return scoped(function* () { + yield* writeTextFile(join(dir, "doc.md"), source); + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd() { + return dir; + }, + }, + { at: "min" }, + ); + if (install) { + yield* install(); + } + const chunks: string[] = []; + const result = yield* scoped(function* () { + const execution = yield* execute({ + path: join(dir, "doc.md"), + stream: new InMemoryStream(), + componentDirs: [dir], + }); + try { + chunks.push(String(yield* collect(execution))); + } catch { + // Collection stops where the failure did; the execution's own outcome + // below is what says why. + } + return yield* execution; + }); + return { + ok: result.ok, + error: result.ok ? undefined : result.error, + output: chunks.join(""), + }; + }); +} + +/** + * Data as it arrives from outside the type system. + * + * A provider is a contextual handler: at run time it can return whatever it + * likes, and the contract this suite exercises is precisely the one the types + * cannot enforce. Round-tripping through JSON is how a malformed value is + * built here without asserting a type it does not have. + */ +function fromOutside(value: unknown): T { + return JSON.parse(JSON.stringify(value)); +} + +/** Install a provider that answers exactly one way, and nothing else. */ +function useFiles(handler: { + checkFilePath?: (input: FilePathInput) => Operation>; + readTextFile?: (input: FilePathInput) => Operation>; + writeTextFile?: (input: FileWriteInput) => Operation>; + globFiles?: (input: { + cwd: string; + include: string[]; + exclude: string[]; + }) => Operation>; + temporaryDirectory?: () => Operation>; +}): Operation { + return Files.around({ + ...(handler.checkFilePath === undefined + ? {} + : { + *checkFilePath([input], next) { + return yield* (handler.checkFilePath ?? next)(input); + }, + }), + ...(handler.readTextFile === undefined + ? {} + : { + *readTextFile([input], next) { + return yield* (handler.readTextFile ?? next)(input); + }, + }), + ...(handler.writeTextFile === undefined + ? {} + : { + *writeTextFile([input], next) { + return yield* (handler.writeTextFile ?? next)(input); + }, + }), + ...(handler.globFiles === undefined + ? {} + : { + *globFiles([input], next) { + return yield* (handler.globFiles ?? next)(input); + }, + }), + ...(handler.temporaryDirectory === undefined + ? {} + : { + *temporaryDirectory(_args, next) { + return yield* (handler.temporaryDirectory ?? next)(); + }, + }), + }); +} + +/** A `` write whose child leaves a marker on disk if it ever runs. */ +function writeDocument(dir: string): string { + return [ + '', + "```sh exec", + `touch ${join(dir, "child-ran.txt")}`, + "```", + "", + "", + "```sh exec", + `touch ${join(dir, "sibling-ran.txt")}`, + "```", + ].join("\n"); +} + +describe("Tier FF — Files infrastructure failure", () => { + beforeAll(() => useTempFileCompiler()); + + // FF1: no provider at all. The check a write performs before its children is + // the first Files call the document makes, so the failure lands before the + // children — and nothing after the component runs either. + it("FF1: an absent provider fails the execution before children or siblings", function* () { + const dir = yield* useFixture(); + const touched: string[] = []; + + const outcome = yield* run(dir, writeDocument(dir), function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + touched.push(path); + return yield* next(path, content); + }, + *rename([from, to], next) { + touched.push(from); + return yield* next(from, to); + }, + }); + }); + + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "provider-unavailable", + }); + // Nothing was written into the document, and nothing ran on either side. + expect(outcome.output).not.toContain("ERROR"); + expect(outcome.output).not.toContain("Files provider"); + expect(yield* exists(join(dir, "child-ran.txt"))).toBe(false); + expect(yield* exists(join(dir, "sibling-ran.txt"))).toBe(false); + // And the low-level host Api was never reached: absence does not fall back. + expect(touched).toEqual([]); + }); + + // FF2: the same for every other form. Each stops at its first Files call. + it("FF2: read, Glob and TempDir all stop at their first provider call", function* () { + const dir = yield* useFixture(); + + for (const source of [ + '\n\nAFTER', + '\n\nAFTER', + "INSIDE\n\nAFTER", + // The self-closing form acquires through `retain()`, which owns the + // resource at the invocation site — a different path to the same call, + // and one whose wrapper could otherwise launder the failure into an + // ordinary one. + '\n\nAFTER', + ]) { + const outcome = yield* run(dir, source); + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))?.kind).toBe("provider-unavailable"); + expect(outcome.output).not.toContain("AFTER"); + expect(outcome.output).not.toContain("INSIDE"); + } + }); + + // FF3: a provider that exists but refuses one operation. A logical filesystem + // owned by a transaction has no temporary directories to give, and inventing + // one would be worse than refusing: the document would run inside somewhere + // the run does not own. + it("FF3: a refused operation is fatal, with its own fixed diagnostic", function* () { + const dir = yield* useFixture(); + + const outcome = yield* run(dir, "INSIDE\n\nAFTER", function* () { + yield* useHostFiles(); + yield* useFiles({ + // deno-lint-ignore require-yield + *temporaryDirectory(): Operation> { + throw new FilesOperationDeniedError("temporary-directory"); + }, + }); + }); + + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "operation-denied", + operation: "temporary-directory", + }); + expect(String(outcome.error)).toContain("Files provider does not support temporary-directory"); + expect(outcome.output).not.toContain("INSIDE"); + expect(outcome.output).not.toContain("AFTER"); + }); + + // FF4: the ordering the whole two-stage write exists for. A check that fails + // means the children never expand and no second provider call is made — the + // provider-level statement of FL18b and FL18c. + it("FF4: a refused check expands no children and makes no later provider call", function* () { + const dir = yield* useFixture(); + const calls: string[] = []; + + const outcome = yield* run(dir, writeDocument(dir), function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *checkFilePath(): Operation> { + calls.push("checkFilePath"); + return Err( + new FilesError( + Object.freeze({ + type: FILES_ERROR, + operation: "check-file-path", + phase: "lexical", + reason: "lexical-escape", + }), + ), + ); + }, + // deno-lint-ignore require-yield + *writeTextFile(): Operation> { + calls.push("writeTextFile"); + throw new Error("the write must never be reached"); + }, + }); + }); + + expect(calls).toEqual(["checkFilePath"]); + // An ordinary refusal, so the document reads it and carries on. + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain("resolves outside the working directory"); + expect(yield* exists(join(dir, "child-ran.txt"))).toBe(false); + expect(yield* exists(join(dir, "sibling-ran.txt"))).toBe(true); + }); + + // FF5: a write outcome nobody can read. Every sentence a component could + // print makes a claim about whether the file was replaced, so a provider that + // cannot describe what it did is treated as one that may not have done it. + it("FF5: malformed write data and a malformed success are both fatal", function* () { + const dir = yield* useFixture(); + + const malformed: Array<() => Operation>> = [ + // A phase and a target that contradict each other. + // deno-lint-ignore require-yield + function* () { + return Err( + new FilesError( + Object.freeze({ + type: FILES_ERROR, + operation: "write", + phase: "temporary", + reason: "no-space", + target: "committed", + }), + ), + ); + }, + // A reason outside the vocabulary. + // deno-lint-ignore require-yield + function* () { + return Err( + new FilesError( + fromOutside({ + type: FILES_ERROR, + operation: "write", + phase: "commit", + reason: "the disk caught fire", + target: "commit-unknown", + }), + ), + ); + }, + // A success that does not describe a publication. + // deno-lint-ignore require-yield + function* () { + return Ok(fromOutside({ type: FILES_WRITE_SUCCESS, publication: "made-up" })); + }, + ]; + + for (const writeTextFile of malformed) { + const outcome = yield* run(dir, 'content\n\nAFTER', function* () { + yield* useHostFiles(); + yield* useFiles({ writeTextFile }); + }); + + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "protocol", + }); + expect(String(outcome.error)).toContain("Files provider invariant failed"); + // The category is control data, not text. + expect(String(outcome.error)).not.toContain("protocol"); + expect(outcome.output).not.toContain("AFTER"); + } + }); + + // FF6: malformed data on a *non-write* failure is not fatal. Nothing about a + // target is at stake, and the component already has a sentence for "the + // operation failed", so the document reads that and carries on. + it("FF6: a malformed non-write failure becomes the generic printed error", function* () { + const dir = yield* useFixture(); + + const outcome = yield* run(dir, '\n\nAFTER', function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + return Err( + Object.assign(new Error("read failed at /planted/absolute/path"), { + data: { type: FILES_ERROR, operation: "read", phase: "nowhere" }, + }), + ); + }, + }); + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain('cannot read "notes.md": the filesystem operation failed.'); + expect(outcome.output).not.toContain("/planted/absolute/path"); + expect(outcome.output).toContain("AFTER"); + }); + + // FF7: a handler that throws something arbitrary. Nothing it produced can be + // trusted, so it is replaced rather than wrapped — no cause, no message, no + // errno text, and no host value survives. + it("FF7: an arbitrary throw becomes a sanitized protocol invariant", function* () { + const dir = yield* useFixture(); + const planted = Object.assign(new Error("EACCES: denied, at '/planted/secret.txt'"), { + code: "EACCES", + }); + + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + throw planted; + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + expect(thrown).toBeInstanceOf(FilesInvariantError); + expect(thrown).not.toBe(planted); + expect(parseFilesFatal(thrown)?.kind).toBe("invariant"); + expect(thrown instanceof Error ? thrown.message : "").toBe("Files provider invariant failed"); + expect(thrown instanceof Error ? thrown.cause : "unset").toBeUndefined(); + expect(JSON.stringify(thrown instanceof Error ? { ...thrown } : {})).not.toContain("planted"); + }); + + // FF8: an already-meaningful failure is preserved by identity instead. The + // first failure is the one that describes what went wrong, and for a + // durability failure the identity is what the shared fail-stop records. + it("FF8: invokeFiles rethrows an existing durability or Files failure unchanged", function* () { + const dir = yield* useFixture(); + + const durability = new StaleInputError("the journal no longer describes this run"); + const files = new FilesInvariantError("authority"); + + for (const planted of [durability, files]) { + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + // Nested, and wrapped, which is how each of them actually arrives. + throw new AggregateError([new Error("noise"), planted], "teardown"); + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + expect(thrown).toBe(planted); + } + }); + + // FF9: precedence between the two, at the wrapper itself. A durability + // failure wins wherever it sits, because a Files invariant raised while one + // is already unwinding is the symptom rather than the cause. + it("FF9: a durability failure beneath a Files failure is the one preserved", function* () { + const dir = yield* useFixture(); + const durability = new StaleInputError("the journal no longer describes this run"); + + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + throw new AggregateError([new FilesInvariantError("teardown"), durability], "unwinding"); + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + expect(thrown).toBe(durability); + expect(filesFatalFailure(thrown)).toBeUndefined(); + }); + + // FF10: an ordinary failure is still an ordinary failure. The fatal rule is + // for a provider that is missing or wrong, not for everything that goes + // wrong beneath one. + it("FF10: an ordinary provider failure stays a printed error", function* () { + const dir = yield* useFixture(); + + const outcome = yield* run(dir, '\n\nAFTER', function* () { + yield* useHostFiles(); + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain('cannot read "absent.md": no such file.'); + expect(outcome.output).toContain("AFTER"); + }); +}); diff --git a/packages/core/tests/glob-component.test.ts b/packages/core/tests/glob-component.test.ts index a3a34512..cca795fe 100644 --- a/packages/core/tests/glob-component.test.ts +++ b/packages/core/tests/glob-component.test.ts @@ -18,7 +18,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, race, resource, scoped, sleep, suspend, until } from "effection"; import type { Operation } from "effection"; import { ensureDir, FsApi, rm, writeTextFile } from "@effectionx/fs"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { execute } from "../src/execute.ts"; @@ -92,6 +92,9 @@ function runWith( yield* writeTextFile(path, source); } yield* useCwd(fixture.workspace); + // `API.Files` has no host default, so a suite driving `execute()` directly + // installs the provider the way an entrypoint does. + yield* useHostFiles(); if (install) { yield* install(); } diff --git a/packages/core/tests/inline-root.test.ts b/packages/core/tests/inline-root.test.ts index 5f07b30b..25ed1329 100644 --- a/packages/core/tests/inline-root.test.ts +++ b/packages/core/tests/inline-root.test.ts @@ -23,7 +23,7 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, scoped, until } from "effection"; import type { Operation } from "effection"; import { InMemoryStream } from "@executablemd/durable-streams"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { useStubFs } from "@executablemd/runtime/test"; import { rm, writeTextFile } from "@effectionx/fs"; import { mkdtemp } from "node:fs/promises"; @@ -131,6 +131,7 @@ function* useWorkspace(files: Record): Operation { }, { at: "min" }, ); + yield* useHostFiles(); return root; } diff --git a/packages/core/tests/loop.test.ts b/packages/core/tests/loop.test.ts index 845e06f7..fdd2c82c 100644 --- a/packages/core/tests/loop.test.ts +++ b/packages/core/tests/loop.test.ts @@ -12,6 +12,7 @@ import type { SourceOrigin } from "../src/scanner.ts"; import { renderSegments } from "../src/render.ts"; import { DivergenceError, InMemoryStream, StaleInputError } from "@executablemd/durable-streams"; import type { DurableEvent, Json, Result } from "@executablemd/durable-streams"; +import { useHostFiles } from "@executablemd/runtime"; import { useEchoExec, useStubFs } from "@executablemd/runtime/test"; import { execute } from "../src/execute.ts"; import { collect } from "../src/collect.ts"; @@ -694,6 +695,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP35: every iteration journals its own eval entry", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": ["", "", "```js eval", "output('RAN');", "```", "", ""].join( "\n", @@ -713,6 +715,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP36: content skipped by writes no journal entry", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": [ "", @@ -746,6 +749,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP37: a binding accumulates across iterations", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": [ "```js eval", @@ -774,6 +778,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP38: a component in the loop body is imported once per iteration", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "components/Step.md": "step\n", "test.md": "", @@ -800,6 +805,7 @@ describe("Tier LOOP — document execution", () => { ].join("\n"); const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); @@ -841,6 +847,7 @@ describe("Tier LOOP — document execution", () => { it("LOOP40: a chosen from a binding stops the document's loop", function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": [ "```js eval", @@ -882,6 +889,7 @@ describe("Tier LOOP — execution records", () => { function runDoc(doc: string, files: Record = {}) { return scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); let output = ""; @@ -994,6 +1002,7 @@ describe("Tier LOOP — execution records", () => { reachedSecond.resolve(); } }; + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); @@ -1082,6 +1091,7 @@ describe("Tier LOOP — execution records", () => { enteredThird.resolve(); } }; + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); @@ -1105,6 +1115,7 @@ describe("Tier LOOP — execution records", () => { // The incomplete journal, read back and handed to a new execution. const resumed = yield* scoped(function* () { const stream = new InMemoryStream(interrupted); + yield* useHostFiles(); yield* useStubFs({ "test.md": DOC }); yield* useEchoExec(); const execution = yield* execute({ @@ -1184,6 +1195,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { ) { return scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); yield* collect(yield* execute({ path: "test.md", stream, props })); @@ -1201,6 +1213,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { ) { return scoped(function* () { const stream = new InMemoryStream(journal); + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); let failure: unknown; @@ -1303,6 +1316,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const cut = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": HELD }); yield* useEchoExec(); yield* collect(yield* execute({ path: "test.md", stream })); @@ -1328,6 +1342,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const run = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": "ab" }); yield* useEchoExec(); // No generic catch sits above the component, so the wrapper reaches the @@ -1367,6 +1382,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const run = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": "ab" }); yield* useEchoExec(); // Thrown from a component, so it travels through the generic catch that @@ -1429,6 +1445,7 @@ describe("Tier LOOP — replay validates the terminal record", () => { const complete = yield* scoped(function* () { const stream = new InMemoryStream(); + yield* useHostFiles(); yield* useStubFs({ "test.md": DIVERGING }); yield* useEchoExec(); yield* collect(yield* execute({ path: "test.md", stream })); @@ -1528,6 +1545,7 @@ describe("Tier BREAK — the projection boundary", () => { function runDoc(doc: string, files: Record) { return scoped(function* () { + yield* useHostFiles(); yield* useStubFs({ "test.md": doc, ...files }); yield* useEchoExec(); return asText( diff --git a/packages/core/tests/output-error-mode.test.ts b/packages/core/tests/output-error-mode.test.ts index fdcf1c0d..2160b407 100644 --- a/packages/core/tests/output-error-mode.test.ts +++ b/packages/core/tests/output-error-mode.test.ts @@ -18,7 +18,7 @@ import { ensure, scoped } from "effection"; import type { Operation } from "effection"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; -import { API } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { useStubFs } from "@executablemd/runtime/test"; import { forEach } from "@effectionx/stream-helpers"; import { execute } from "../src/execute.ts"; @@ -68,6 +68,8 @@ function run(files: Record, stream = new InMemoryStream()): Oper return scoped(function* () { yield* useStubFs(files); yield* useStagedExec(); + // `` reaches `API.Files`, which has no host default. + yield* useHostFiles(); const execution = yield* execute({ path: "doc.md", stream }); const chunks: string[] = []; diff --git a/packages/core/tests/temp-dir.test.ts b/packages/core/tests/temp-dir.test.ts index 7ffbc84e..2abb06ba 100644 --- a/packages/core/tests/temp-dir.test.ts +++ b/packages/core/tests/temp-dir.test.ts @@ -22,6 +22,7 @@ import { import type { Operation } from "effection"; import { when } from "@effectionx/converge"; import { cwd, exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { useHostFiles } from "@executablemd/runtime"; import { InMemoryStream, StaleInputError } from "@executablemd/durable-streams"; import type { Json } from "@executablemd/durable-streams"; import { execute } from "../src/execute.ts"; @@ -56,6 +57,7 @@ function writeDocument(dir: string, source: string): Operation { /** One bounded run, so the document scope closes before effects are read. */ function run(dir: string): Operation { return scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), @@ -114,6 +116,7 @@ describe("Tier TD — TempDir", () => { const stream = new InMemoryStream(); const output = yield* scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); @@ -381,6 +384,7 @@ describe("Tier TD — TempDir", () => { const stream = new InMemoryStream(); const first = String( yield* scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); @@ -401,6 +405,7 @@ describe("Tier TD — TempDir", () => { (event) => !(event.type === "close" && event.coroutineId === "root"), ); const outcome = yield* scoped(function* () { + yield* useHostFiles(); const execution = yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(partial), @@ -423,6 +428,7 @@ describe("Tier TD — TempDir", () => { // one behind — whether it arrives while the directory is in use or before // the acquiring task has run at all. it("TD15: a cancelled acquisition leaves no directory behind", function* () { + yield* useHostFiles(); const before = yield* temporaries(); // Halted while the directory is live: the path is observed first, so the @@ -467,6 +473,7 @@ describe("Tier TD — TempDir", () => { const stream = new InMemoryStream(); const first = String( yield* scoped(function* () { + yield* useHostFiles(); return yield* collect( yield* execute({ path: join(dir, "doc.md"), stream, componentDirs: [dir] }), ); @@ -483,6 +490,7 @@ describe("Tier TD — TempDir", () => { (event) => !(event.type === "close" && event.coroutineId === "root"), ); const outcome = yield* scoped(function* () { + yield* useHostFiles(); const execution = yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(partial), @@ -521,6 +529,7 @@ describe("Tier TD — TempDir", () => { ); const outcome = yield* scoped(function* () { + yield* useHostFiles(); const execution = yield* execute({ path: join(dir, "doc.md"), stream: new InMemoryStream(), diff --git a/packages/runtime/apis.ts b/packages/runtime/apis.ts index 2e02f311..e3c07474 100644 --- a/packages/runtime/apis.ts +++ b/packages/runtime/apis.ts @@ -30,10 +30,15 @@ * * - **Process** — subprocess lifecycle has its own cancellation semantics * (killing processes on scope teardown). Middleware targets exec only. - * - **Fs** — reading, writing, and inspecting files form a cohesive file-IO - * surface used together for component resolution, replay guards, and the - * `` component. Middleware installed here sees a document's own file - * access on the same terms as the engine's. + * - **Fs** — the low-level host file surface: reading, writing, and inspecting + * paths the engine itself resolves, for component lookup, replay guards, and + * the root document. It is the host adapter's own dependency, not the + * boundary a document's paths cross. + * - **Files** — document filesystem access, in whole semantic operations + * (`files.ts`). ``, ``, and `` speak only this Api, so + * the same document means the same thing whether its paths resolve in the + * caller's filesystem or in a run-owned logical one. Its terminal handler + * throws: an uninstalled provider must not silently reach the host. * - **Fetch** — HTTP has distinct timeout/body/abort semantics. Merging * with Fs or Process would blur cancellation boundaries. * - **Env** — the host itself: metadata (env vars, platform) plus the two @@ -84,6 +89,7 @@ import { exec as processExec } from "@effectionx/process"; import { race, sleep, until } from "effection"; import type { Operation } from "effection"; import { timeout as contextualTimeout } from "./config.ts"; +import { Files } from "./files.ts"; import { Service } from "./service.ts"; /** @@ -342,6 +348,7 @@ interface EnvHandler { export const API: { Process: Api; Fs: Api; + Files: typeof Files; Fetch: Api; Env: Api; Service: typeof Service; @@ -545,6 +552,7 @@ export const API: { ); }, }), + Files, Service, }; diff --git a/packages/runtime/files.ts b/packages/runtime/files.ts new file mode 100644 index 00000000..32197123 --- /dev/null +++ b/packages/runtime/files.ts @@ -0,0 +1,653 @@ +/** + * `API.Files` — the document filesystem boundary. + * + * A document names files with a path relative to the contextual working + * directory, and every one of those operations arrives here. What is on the + * other side is a provider's choice: `xmd run` installs a host adapter that + * resolves those paths in the caller's own filesystem, and `xmd workflow` + * installs one that resolves them in a logical filesystem owned by a database + * transaction. Neither is named in the components that call this Api, which is + * what lets one document mean the same thing under both. + * + * The operations are **semantic**, not primitive. `writeTextFile` is a whole + * replacement — admission, resolution, target classification, parent creation, + * and commit — rather than a sequence a caller assembles, because assembling it + * from outside is what would let a path admitted by one provider be used by + * another. `API.Fs` remains the low-level host surface a host adapter is built + * on; it is not this boundary. + * + * `checkFilePath` is the one exception, and it is deliberately weak: pure path + * arithmetic, no filesystem access, and nothing usable comes back — no path, no + * handle, no authority token. ``'s write form calls it to decide whether + * its children may expand at all, and the later `writeTextFile` repeats the + * same admission from the same authored path. A check that was skipped, + * replaced, or answered by another provider therefore authorizes nothing. + * + * ## Two kinds of failure + * + * An ordinary filesystem condition — missing, a directory, permission denied, + * no space — comes back as `Err(FilesError)` carrying frozen structural data. + * The consumer reads that data and selects a sentence from a fixed vocabulary; + * no message, errno code, resolved path, temporary path, or symlink target + * crosses this boundary. Cancellation is neither of these: it is not caught and + * never becomes a Result. + * + * A provider that is absent, that refuses an operation, or that breaks its own + * contract is not a filesystem condition. Those **throw**, with fixed + * diagnostics and no cause, and they end the execution rather than becoming + * something a document renders. A run whose Files provider is missing must not + * quietly reach the host instead. + * + * ## Why the data is structural + * + * Both the failures and the write outcome carry a plain frozen object under a + * stable `type` tag, and every consumer recognizes them by parsing that tag + * rather than with `instanceof`. Two copies of this package can be loaded at + * once — a repository component resolving its own runtime beside the engine's — + * and `instanceof` answers false across them, which would turn a provider + * failure into an unrecognized throw exactly when it matters most. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import type { Operation, Result } from "effection"; + +/** The stable discriminant on ordinary filesystem failure data. */ +export const FILES_ERROR = "executablemd.runtime.files-error/v1"; + +/** The stable discriminant on infrastructure failure data. */ +export const FILES_FATAL = "executablemd.runtime.files-fatal/v1"; + +/** The stable discriminant on a successful write's outcome data. */ +export const FILES_WRITE_SUCCESS = "executablemd.runtime.files-write-success/v1"; + +/** + * The vocabulary an ordinary failure is reported in. + * + * A provider maps whatever its platform produced onto one of these before + * returning. An unmapped condition becomes `operation-failed`, which is a real + * answer rather than a placeholder: the consumer has a sentence for it, and the + * unmapped value itself never crosses the boundary. + */ +export type FilesReason = + | "empty-path" + | "absolute-path" + | "lexical-escape" + | "resolved-escape" + | "missing" + | "directory" + | "special-file" + | "not-directory" + | "permission-denied" + | "read-only" + | "too-many-symlinks" + | "path-too-long" + | "no-space" + | "quota-exhausted" + | "cross-device" + | "busy" + | "too-many-open-files" + | "directory-not-empty" + | "invalid-pattern" + | "operation-failed"; + +const REASONS: readonly FilesReason[] = [ + "empty-path", + "absolute-path", + "lexical-escape", + "resolved-escape", + "missing", + "directory", + "special-file", + "not-directory", + "permission-denied", + "read-only", + "too-many-symlinks", + "path-too-long", + "no-space", + "quota-exhausted", + "cross-device", + "busy", + "too-many-open-files", + "directory-not-empty", + "invalid-pattern", + "operation-failed", +]; + +/** The operations whose failure carries no commit outcome. */ +export type FilesOperation = "check-file-path" | "read" | "glob" | "temporary-directory"; + +const OPERATIONS: readonly FilesOperation[] = [ + "check-file-path", + "read", + "glob", + "temporary-directory", +]; + +/** Where a non-write operation stopped. */ +export type FilesPhase = + | "lexical" + | "resolution" + | "target" + | "access" + | "pattern" + | "traversal" + | "acquire"; + +const PHASES: readonly FilesPhase[] = [ + "lexical", + "resolution", + "target", + "access", + "pattern", + "traversal", + "acquire", +]; + +/** Where a write stopped, which is what decides what may be said about the target. */ +export type FileWritePhase = + | "lexical" + | "resolution" + | "target" + | "parents" + | "temporary" + | "commit" + | "cleanup" + | "transaction"; + +/** + * What is known about the target afterwards. + * + * `commit-unknown` is an answer rather than a missing one: a commit that threw + * may have run or not, and no provider can tell which from where it stands. + * What still holds in that case is that the target is one complete version. + */ +export type FileWriteTarget = "unchanged" | "commit-unknown" | "committed" | "rolled-back"; + +export interface FilesFailureData { + readonly type: typeof FILES_ERROR; + readonly operation: FilesOperation; + readonly phase: FilesPhase; + readonly reason: FilesReason; +} + +export interface FileWriteFailureData { + readonly type: typeof FILES_ERROR; + readonly operation: "write"; + readonly phase: FileWritePhase; + readonly reason?: FilesReason; + readonly cleanup?: FilesReason; + readonly target: FileWriteTarget; +} + +export type FilesErrorData = FilesFailureData | FileWriteFailureData; + +/** + * The message every ordinary failure carries. + * + * Constant on purpose. A message is the part of an Error that gets printed by + * accident, and there is nothing safe to put in this one: the authored path + * belongs to the consumer that wrote it, and everything else belongs to the + * platform. + */ +export const FILES_ERROR_MESSAGE = "Files operation failed"; + +/** An ordinary filesystem failure. What it means is in `data`, never in the message. */ +export class FilesError extends Error { + readonly data: FilesErrorData; + + constructor(data: FilesErrorData) { + super(FILES_ERROR_MESSAGE); + this.name = "FilesError"; + this.data = data; + } +} + +export interface FileWriteSuccess { + readonly type: typeof FILES_WRITE_SUCCESS; + readonly publication: "host-committed" | "transaction-staged"; +} + +export interface FilePathInput { + readonly cwd: string; + readonly path: string; +} + +export interface FileWriteInput extends FilePathInput { + readonly content: string; +} + +export interface GlobInput { + readonly cwd: string; + readonly include: string[]; + readonly exclude: string[]; +} + +export interface FilesHandler { + /** + * Whether this authored path is admissible at all, decided from the path and + * `cwd` alone. No filesystem access, and nothing usable comes back. + */ + checkFilePath(input: FilePathInput): Operation>; + readTextFile(input: FilePathInput): Operation>; + writeTextFile(input: FileWriteInput): Operation>; + /** Sorted, deduplicated, POSIX-separated paths of the regular files that match. */ + globFiles(input: GlobInput): Operation>; + /** + * A directory that lives as long as the acquiring scope. A resource, so the + * caller holds it by acquisition rather than by remembering to remove it. + */ + temporaryDirectory(): Operation>; +} + +/** The operations a provider may refuse outright rather than fail at. */ +export type FilesDeniableOperation = "temporary-directory"; + +/** + * Which contract a provider broke. + * + * `authority` — the identity authorizing access is stale, foreign, or gone. + * `savepoint` — a nested transaction could not be rolled back or released. + * `protocol` — a handler threw, or returned data no consumer can trust. + * `teardown` — cleanup failed while the scope was already unwinding. + */ +export type FilesInvariantCategory = "authority" | "savepoint" | "protocol" | "teardown"; + +const INVARIANT_CATEGORIES: readonly FilesInvariantCategory[] = [ + "authority", + "savepoint", + "protocol", + "teardown", +]; + +/** + * Infrastructure failure data. + * + * Three kinds, each with fixed fields and nothing derived from the condition + * that produced it. A category is control data for a consumer deciding what to + * fence, not text: no diagnostic interpolates it. + */ +export type FilesFatalData = + | { readonly type: typeof FILES_FATAL; readonly kind: "provider-unavailable" } + | { + readonly type: typeof FILES_FATAL; + readonly kind: "operation-denied"; + readonly operation: FilesDeniableOperation; + } + | { + readonly type: typeof FILES_FATAL; + readonly kind: "invariant"; + readonly category: FilesInvariantCategory; + }; + +export const FILES_PROVIDER_UNAVAILABLE_MESSAGE = "Files provider is not installed"; +export const FILES_OPERATION_DENIED_MESSAGE = "Files provider does not support temporary-directory"; +export const FILES_INVARIANT_MESSAGE = "Files provider invariant failed"; + +/** No Files provider is installed, and there is no host to fall back to. */ +export class FilesProviderUnavailableError extends Error { + readonly data: FilesFatalData; + + constructor() { + super(FILES_PROVIDER_UNAVAILABLE_MESSAGE); + this.name = "FilesProviderUnavailableError"; + this.data = Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }); + } +} + +/** The installed provider does not implement this operation at all. */ +export class FilesOperationDeniedError extends Error { + readonly data: FilesFatalData; + + constructor(operation: FilesDeniableOperation) { + super(FILES_OPERATION_DENIED_MESSAGE); + this.name = "FilesOperationDeniedError"; + const denied = deniableOperation(operation); + if (denied === undefined) { + throw new FilesInvariantError("protocol"); + } + this.data = Object.freeze({ type: FILES_FATAL, kind: "operation-denied", operation: denied }); + } +} + +/** A provider broke its own contract. */ +export class FilesInvariantError extends Error { + readonly data: FilesFatalData; + + constructor(category: FilesInvariantCategory) { + super(FILES_INVARIANT_MESSAGE); + this.name = "FilesInvariantError"; + const parsed = invariantCategory(category); + if (parsed === undefined) { + throw new Error(FILES_INVARIANT_MESSAGE); + } + this.data = Object.freeze({ type: FILES_FATAL, kind: "invariant", category: parsed }); + } +} + +/** An infrastructure failure, recognized structurally rather than by class. */ +export interface FilesFatalFailure extends Error { + readonly data: FilesFatalData; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function dataOf(error: unknown): Record | undefined { + if (!(error instanceof Error) || !("data" in error)) { + return undefined; + } + const { data } = error; + return isRecord(data) ? data : undefined; +} + +function reasonOf(value: unknown): FilesReason | undefined { + return REASONS.find((reason) => reason === value); +} + +function operationOf(value: unknown): FilesOperation | undefined { + return OPERATIONS.find((operation) => operation === value); +} + +function phaseOf(value: unknown): FilesPhase | undefined { + return PHASES.find((phase) => phase === value); +} + +function invariantCategory(value: unknown): FilesInvariantCategory | undefined { + return INVARIANT_CATEGORIES.find((category) => category === value); +} + +function deniableOperation(value: unknown): FilesDeniableOperation | undefined { + return value === "temporary-directory" ? "temporary-directory" : undefined; +} + +/** + * The infrastructure failure data this Error carries, if it carries valid data. + * + * Every field is checked, and the member count with them: an object with extra + * keys is not the shape this contract describes, and accepting it would let a + * provider smuggle a path or a message through under a recognized tag. + */ +export function parseFilesFatal(error: unknown): FilesFatalData | undefined { + const data = dataOf(error); + if (data === undefined || data.type !== FILES_FATAL) { + return undefined; + } + const members = Object.keys(data).length; + if (data.kind === "provider-unavailable" && members === 2) { + return Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }); + } + if (data.kind === "operation-denied" && members === 3) { + const operation = deniableOperation(data.operation); + if (operation !== undefined) { + return Object.freeze({ type: FILES_FATAL, kind: "operation-denied", operation }); + } + } + if (data.kind === "invariant" && members === 3) { + const category = invariantCategory(data.category); + if (category !== undefined) { + return Object.freeze({ type: FILES_FATAL, kind: "invariant", category }); + } + } + return undefined; +} + +/** + * Whether this failure is a Files infrastructure failure. + * + * Structural, so a failure constructed by a separately loaded copy of this + * package is recognized on the same terms as one constructed here. + */ +export function isFilesFatal(error: unknown): error is FilesFatalFailure { + return parseFilesFatal(error) !== undefined; +} + +/** + * The infrastructure failure this one is, by identity. + * + * The original object comes back rather than a replacement, because a fail-stop + * that records "the first error" has to record the one that was thrown. + */ +export function asFilesFatal(error: unknown): FilesFatalFailure | undefined { + return isFilesFatal(error) ? error : undefined; +} + +/** + * What a write phase is allowed to say, which is the whole validity rule. + * + * The phase decides the target claim: nothing selects `committed` except a + * cleanup that failed after the commit returned, and nothing claims + * `commit-unknown` except a commit that threw. Keeping the table here rather + * than at each construction site makes an invalid combination unconstructable + * instead of merely unwritten. + */ +interface WritePhaseRule { + readonly target: FileWriteTarget; + readonly reason: "required" | "absent"; + readonly cleanup: "required" | "optional" | "absent"; +} + +const WRITE_PHASES: ReadonlyMap = new Map< + FileWritePhase, + WritePhaseRule +>([ + ["lexical", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["resolution", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["target", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["parents", { target: "unchanged", reason: "required", cleanup: "absent" }], + ["temporary", { target: "unchanged", reason: "required", cleanup: "optional" }], + ["commit", { target: "commit-unknown", reason: "required", cleanup: "optional" }], + ["cleanup", { target: "committed", reason: "absent", cleanup: "required" }], + ["transaction", { target: "rolled-back", reason: "required", cleanup: "absent" }], +]); + +function writePhaseOf(value: unknown): [FileWritePhase, WritePhaseRule] | undefined { + for (const entry of WRITE_PHASES) { + if (entry[0] === value) { + return entry; + } + } + return undefined; +} + +function violatesRule( + rule: WritePhaseRule, + reason: FilesReason | undefined, + cleanup: FilesReason | undefined, +): boolean { + if (rule.reason === "required" && reason === undefined) { + return true; + } + if (rule.reason === "absent" && reason !== undefined) { + return true; + } + if (rule.cleanup === "required" && cleanup === undefined) { + return true; + } + return rule.cleanup === "absent" && cleanup !== undefined; +} + +function writeData( + phase: FileWritePhase, + rule: WritePhaseRule, + reason: FilesReason | undefined, + cleanup: FilesReason | undefined, +): FileWriteFailureData { + return Object.freeze({ + type: FILES_ERROR, + operation: "write", + phase, + target: rule.target, + ...(reason === undefined ? {} : { reason }), + ...(cleanup === undefined ? {} : { cleanup }), + }); +} + +/** Build an ordinary non-write failure. */ +export function filesFailure(input: { + operation: FilesOperation; + phase: FilesPhase; + reason: FilesReason; +}): FilesError { + const operation = operationOf(input.operation); + const phase = phaseOf(input.phase); + const reason = reasonOf(input.reason); + if (operation === undefined || phase === undefined || reason === undefined) { + throw new FilesInvariantError("protocol"); + } + return new FilesError(Object.freeze({ type: FILES_ERROR, operation, phase, reason })); +} + +/** + * Build a write failure, refusing any combination a consumer could not read. + * + * A write's report is the only place a document learns what became of a file it + * asked to replace, so an invalid combination is a provider bug rather than a + * value to pass along and interpret later. + */ +export function fileWriteFailure(input: { + phase: FileWritePhase; + reason?: FilesReason; + cleanup?: FilesReason; +}): FilesError { + const found = writePhaseOf(input.phase); + if (found === undefined) { + throw new FilesInvariantError("protocol"); + } + const [phase, rule] = found; + + const reason = input.reason === undefined ? undefined : reasonOf(input.reason); + const cleanup = input.cleanup === undefined ? undefined : reasonOf(input.cleanup); + if ( + (input.reason !== undefined && reason === undefined) || + (input.cleanup !== undefined && cleanup === undefined) + ) { + throw new FilesInvariantError("protocol"); + } + if (violatesRule(rule, reason, cleanup)) { + throw new FilesInvariantError("protocol"); + } + + return new FilesError(writeData(phase, rule, reason, cleanup)); +} + +/** + * The non-write failure data this error carries, if it carries valid data. + * + * Malformed data is not fatal here — the consumer already has a sentence for + * "the operation failed" and nothing about a target is at stake — so this + * simply declines to recognize it. + */ +export function parseFilesFailure(error: unknown): FilesFailureData | undefined { + const data = dataOf(error); + if (data === undefined || data.type !== FILES_ERROR || Object.keys(data).length !== 4) { + return undefined; + } + const operation = operationOf(data.operation); + const phase = phaseOf(data.phase); + const reason = reasonOf(data.reason); + if (operation === undefined || phase === undefined || reason === undefined) { + return undefined; + } + return Object.freeze({ type: FILES_ERROR, operation, phase, reason }); +} + +/** + * The write failure data this error carries, if it carries valid data. + * + * Unlike a non-write failure, malformed data here has no safe reading: every + * sentence a consumer could print makes a claim about whether the file was + * replaced. A caller treats `undefined` from a write as a protocol invariant + * rather than inventing a commit state. + */ +export function parseFileWriteFailure(error: unknown): FileWriteFailureData | undefined { + const data = dataOf(error); + if (data === undefined || data.type !== FILES_ERROR || data.operation !== "write") { + return undefined; + } + const found = writePhaseOf(data.phase); + if (found === undefined) { + return undefined; + } + const [phase, rule] = found; + if (data.target !== rule.target) { + return undefined; + } + + const reason = data.reason === undefined ? undefined : reasonOf(data.reason); + const cleanup = data.cleanup === undefined ? undefined : reasonOf(data.cleanup); + if ( + (data.reason !== undefined && reason === undefined) || + (data.cleanup !== undefined && cleanup === undefined) || + violatesRule(rule, reason, cleanup) + ) { + return undefined; + } + + const members = 4 + (reason === undefined ? 0 : 1) + (cleanup === undefined ? 0 : 1); + if (Object.keys(data).length !== members) { + return undefined; + } + + return writeData(phase, rule, reason, cleanup); +} + +/** A successful write's outcome. */ +export function fileWriteSuccess(publication: FileWriteSuccess["publication"]): FileWriteSuccess { + const parsed = parseFileWriteSuccess({ type: FILES_WRITE_SUCCESS, publication }); + if (parsed === undefined) { + throw new FilesInvariantError("protocol"); + } + return parsed; +} + +/** + * The write outcome this value is, if it is a valid one. + * + * A malformed success is as untrustworthy as a malformed failure: a provider + * that cannot describe what it did may not have done it, so a caller treats + * `undefined` here as a protocol invariant too. + */ +export function parseFileWriteSuccess(value: unknown): FileWriteSuccess | undefined { + if (!isRecord(value) || value.type !== FILES_WRITE_SUCCESS || Object.keys(value).length !== 2) { + return undefined; + } + if (value.publication === "host-committed") { + return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "host-committed" }); + } + if (value.publication === "transaction-staged") { + return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "transaction-staged" }); + } + return undefined; +} + +/** + * The document filesystem Api. + * + * The terminal handler throws for every operation, including `checkFilePath`. + * A default that reached the host would make an uninstalled provider + * indistinguishable from an installed one, and the whole point of the boundary + * is that a workflow run cannot silently touch the caller's filesystem. + */ +export const Files: Api = createApi("executablemd.runtime.files", { + // deno-lint-ignore require-yield + *checkFilePath(_input: FilePathInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *readTextFile(_input: FilePathInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *writeTextFile(_input: FileWriteInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *globFiles(_input: GlobInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield + *temporaryDirectory(): Operation> { + throw new FilesProviderUnavailableError(); + }, +}); diff --git a/packages/runtime/host-files.ts b/packages/runtime/host-files.ts new file mode 100644 index 00000000..45ada6be --- /dev/null +++ b/packages/runtime/host-files.ts @@ -0,0 +1,545 @@ +/** + * The host `API.Files` provider — document filesystem access in the caller's + * own filesystem. + * + * This is what `xmd run` installs. A document's relative path is resolved + * against the contextual working directory and used as an ordinary host path, + * so a document can hand a file to a tool the caller already has. Everything + * below is built on the low-level `API.Fs`, which is deliberate: a host that + * already wraps `API.Fs` to observe or sandbox the engine's own file access + * keeps seeing a document's access on the same terms. + * + * ## What containment means here + * + * Access is confined to the contextual directory, judged against the filesystem + * as this adapter observes it. An empty path, an absolute path, and a lexical + * `..` escape are refused without touching the filesystem at all; a symlink + * leading out is refused once resolution can see it. + * + * That is sound **while the host pathname namespace is stable**, and every + * guarantee here is stated on that basis. It is not a sandbox. Another process + * can replace a directory, symlink, junction, or reparse point between the + * moment this adapter observes a path and the moment it uses one, and nothing + * available on the shipped runtimes closes that window without a native + * dependency. What is contained is the document's own children — the case a + * document controls — because resolution is deferred until after they run. + * + * ## Writes + * + * A write goes through a sibling temporary file and a rename. The rename is the + * commit point: everything before it can fail or be cancelled with the previous + * file untouched, and once it begins the target holds the complete old file or + * the complete new one, never a partial write. It is a commit rather than a + * transaction — a rename that returned is not undone by a later cancellation. + * The temporary also closes the one hole resolution cannot: a dangling symlink + * has nothing to resolve, and `rename` replaces the link rather than following + * it wherever it points. + * + * ## What crosses the boundary + * + * Nothing from a caught platform error. An errno code *selects* a + * `FilesReason`, and the reason is all the consumer receives — no message, no + * code, no resolved path, no temporary path, and no symlink target. A platform + * error names the path it failed on, and for a write that path can be a + * temporary the document never chose. + */ + +import { ensure, Err, Ok, resource, scoped } from "effection"; +import type { Operation, Result } from "effection"; +import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; +import { randomUUID } from "node:crypto"; +import { mkdtempSync, realpathSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { FsApi, rm } from "@effectionx/fs"; +import { API } from "./apis.ts"; +import { fileWriteFailure, fileWriteSuccess, filesFailure, FilesInvariantError } from "./files.ts"; +import type { + FilePathInput, + FilesHandler, + FilesOperation, + FilesPhase, + FilesReason, + FileWriteInput, + FileWritePhase, + FileWriteSuccess, + GlobInput, +} from "./files.ts"; + +/** + * A private step a host operation is about to take. + * + * Test-only. Production entrypoints install the adapter without one, so this is + * neither global state nor a capability: an observer can watch, and the point of + * watching is to replace part of the tree between an observation and the call + * that follows it, which is how the stable-namespace limitation is made + * observable rather than merely stated. + */ +export interface HostFilesEvent { + readonly operation: "read" | "write" | "glob"; + readonly phase: "target" | "access" | "parents" | "temporary" | "commit" | "cleanup" | "read-dir"; +} + +/** Synchronous, so nothing can run between the observation and the call it precedes. */ +export type HostFilesObserver = (event: HostFilesEvent) => void; + +export interface HostFilesOptions { + readonly observe?: HostFilesObserver; +} + +/** + * The errno codes this adapter recognizes, and the reason each selects. + * + * A `Map` rather than an object literal, because a lookup on one answers for + * inherited keys — `codes["toString"]` would hand back a function — and the code + * is chosen by whatever implements `API.Fs`. + */ +const REASON_BY_CODE: ReadonlyMap = new Map([ + ["ENOENT", "missing"], + ["ENOTDIR", "not-directory"], + ["EISDIR", "directory"], + ["ENOTEMPTY", "directory-not-empty"], + ["EACCES", "permission-denied"], + ["EPERM", "permission-denied"], + ["EROFS", "read-only"], + ["ELOOP", "too-many-symlinks"], + ["ENAMETOOLONG", "path-too-long"], + ["ENOSPC", "no-space"], + ["EDQUOT", "quota-exhausted"], + ["EXDEV", "cross-device"], + ["EBUSY", "busy"], + ["EMFILE", "too-many-open-files"], +]); + +/** + * The `errno` string a failed call carries, when it carries one. + * + * Read rather than asserted: `catch` gives back `unknown`, and what arrives + * there is only conventionally an `ErrnoException`. + */ +function errorCode(error: unknown): string | undefined { + if (typeof error !== "object" || error === null || !("code" in error)) { + return undefined; + } + const { code } = error; + return typeof code === "string" ? code : undefined; +} + +/** The reason a caught platform error selects, defaulting to the generic one. */ +function reasonOf(error: unknown): FilesReason { + const code = errorCode(error); + if (code === undefined) { + return "operation-failed"; + } + return REASON_BY_CODE.get(code) ?? "operation-failed"; +} + +/** + * Why an authored path is inadmissible, decided from arithmetic alone. + * + * `resolve` normalizes `..` lexically, so this holds against the contextual + * directory as given — canonicalizing it belongs to resolution and would only + * move the same comparison onto a different pair of strings. + */ +function inadmissible(input: FilePathInput): FilesReason | undefined { + if (input.path.length === 0) { + return "empty-path"; + } + if (isAbsolute(input.path)) { + return "absolute-path"; + } + if (!within(input.cwd, resolve(input.cwd, input.path))) { + return "lexical-escape"; + } + return undefined; +} + +/** + * Whether `path` names `base` or something inside it. + * + * The directory itself is contained — `.` is not an escape. That it is a + * directory is a question about the target, which the target check answers. + * + * Only a complete `..` segment leaves. A name that merely starts with two dots + * — `..notes.md` — is an ordinary file inside, and a prefix test would refuse it. + */ +function within(base: string, path: string): boolean { + const rel = relative(base, path); + if (isAbsolute(rel)) { + return false; + } + if (rel.length === 0) { + return true; + } + return rel !== ".." && !rel.startsWith(`..${sep}`); +} + +/** + * `path` with every symlink in its existing prefix resolved. + * + * `realpath` needs the whole path to exist, and a write commonly names one that + * does not yet, so the walk gives up one trailing segment at a time until + * something answers and then puts the segments back. The working directory + * always exists, so the loop terminates there at the latest. + */ +function* resolveExisting(path: string): Operation { + const trailing: string[] = []; + let current = path; + + while (true) { + const resolved = yield* API.Fs.operations.realpath(current); + if (resolved !== undefined) { + return trailing.length === 0 ? resolved : join(resolved, ...trailing); + } + const parent = dirname(current); + if (parent === current) { + return join(current, ...trailing); + } + trailing.unshift(basename(current)); + current = parent; + } +} + +/** What resolution produced, or why it could not. */ +type Destination = { readonly path: string } | { readonly reason: FilesReason }; + +/** + * The path as the filesystem currently has it. + * + * Resolves the part of the path that is already on disk — the file itself when + * it is there, the deepest existing ancestor when it is not — and re-checks the + * result, which is what catches a symlink pointing out. What comes back is that + * resolved path, so an internal symlink is followed to the file it names rather + * than replaced. + * + * Both sides of the comparison are canonical, so a working directory reached + * through a symlink — macOS's `/var` against `/private/var` — does not read as + * an escape. + */ +function* destination(input: FilePathInput): Operation { + try { + const base = (yield* API.Fs.operations.realpath(input.cwd)) ?? input.cwd; + const path = yield* resolveExisting(resolve(input.cwd, input.path)); + if (!within(base, path)) { + return { reason: "resolved-escape" }; + } + return { path }; + } catch (error) { + return { reason: reasonOf(error) }; + } +} + +function nonWriteFailure( + operation: FilesOperation, + phase: FilesPhase, + reason: FilesReason, +): Result { + return Err(filesFailure({ operation, phase, reason })); +} + +function writeFailure(input: { + phase: FileWritePhase; + reason?: FilesReason; + cleanup?: FilesReason; +}): Result { + return Err(fileWriteFailure(input)); +} + +function notify(observe: HostFilesObserver | undefined, event: HostFilesEvent): void { + observe?.(event); +} + +/** Code point order: what a document branches on must not depend on a locale. */ +function byCodePoint(left: string, right: string): number { + if (left < right) { + return -1; + } + if (left > right) { + return 1; + } + return 0; +} + +/** + * Build a host provider. + * + * Exported so a test can drive one operation directly; entrypoints install it + * with {@link useHostFiles}. + */ +export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { + const observe = options.observe; + + function* checkFilePath(input: FilePathInput): Operation> { + const reason = inadmissible(input); + if (reason !== undefined) { + return nonWriteFailure("check-file-path", "lexical", reason); + } + return Ok(undefined); + } + + function* readTextFile(input: FilePathInput): Operation> { + const lexical = inadmissible(input); + if (lexical !== undefined) { + return nonWriteFailure("read", "lexical", lexical); + } + + const target = yield* destination(input); + if ("reason" in target) { + return nonWriteFailure("read", "resolution", target.reason); + } + + notify(observe, { operation: "read", phase: "target" }); + try { + const info = yield* API.Fs.operations.stat(target.path); + if (!info.exists) { + return nonWriteFailure("read", "target", "missing"); + } + if (info.isDirectory) { + return nonWriteFailure("read", "target", "directory"); + } + if (!info.isFile) { + return nonWriteFailure("read", "target", "special-file"); + } + } catch (error) { + return nonWriteFailure("read", "target", reasonOf(error)); + } + + notify(observe, { operation: "read", phase: "access" }); + try { + return Ok(yield* API.Fs.operations.readTextFile(target.path)); + } catch (error) { + return nonWriteFailure("read", "access", reasonOf(error)); + } + } + + /** + * Replace the target with exactly `content`. + * + * Admission is repeated here from the authored path and contextual directory + * rather than carried over from `checkFilePath`. The check answers whether + * children may expand; a child can change what a path means, and a + * destination resolved before they ran would not be the one this write lands + * on. + * + * Removal of the temporary is registered before it is written rather than + * after. The write is where an interruption is most likely to land, and a + * cleanup installed on the far side of it would not run for the one failure it + * exists to handle. `remove` is forced, so registering it for a file that was + * never created — or one the rename has already consumed — is a no-op. + * + * Both halves are collected rather than thrown. A destructor that threw would + * replace the failure it is unwinding, and a write's own failure must not hide + * the fact that a temporary was left behind. + */ + function* writeTextFile(input: FileWriteInput): Operation> { + const lexical = inadmissible(input); + if (lexical !== undefined) { + return writeFailure({ phase: "lexical", reason: lexical }); + } + + const target = yield* destination(input); + if ("reason" in target) { + return writeFailure({ phase: "resolution", reason: target.reason }); + } + + notify(observe, { operation: "write", phase: "target" }); + try { + const info = yield* API.Fs.operations.stat(target.path); + if (info.exists && !info.isFile) { + return writeFailure({ + phase: "target", + reason: info.isDirectory ? "directory" : "special-file", + }); + } + } catch (error) { + return writeFailure({ phase: "target", reason: reasonOf(error) }); + } + + notify(observe, { operation: "write", phase: "parents" }); + try { + yield* API.Fs.operations.ensureDir(dirname(target.path)); + } catch (error) { + return writeFailure({ phase: "parents", reason: reasonOf(error) }); + } + + let failed: FilesReason | undefined; + let cleanup: FilesReason | undefined; + // Which step the write reached, which is what decides what may be said + // about the target: everything before the rename leaves the previous file + // in place, and a rename that threw may have run or not. + let step: FileWritePhase = "temporary"; + + yield* scoped(function* () { + const temporary = `${target.path}.xmd-${randomUUID().slice(0, 8)}.tmp`; + yield* ensure(function* () { + notify(observe, { operation: "write", phase: "cleanup" }); + try { + yield* API.Fs.operations.remove(temporary, { force: true }); + } catch (error) { + cleanup = reasonOf(error); + } + }); + try { + notify(observe, { operation: "write", phase: "temporary" }); + yield* API.Fs.operations.writeTextFile(temporary, input.content); + step = "commit"; + notify(observe, { operation: "write", phase: "commit" }); + yield* API.Fs.operations.rename(temporary, target.path); + } catch (error) { + failed = reasonOf(error); + } + }); + + if (failed !== undefined) { + return writeFailure({ phase: step, reason: failed, cleanup }); + } + if (cleanup !== undefined) { + return writeFailure({ phase: "cleanup", cleanup }); + } + return Ok(fileWriteSuccess("host-committed")); + } + + /** + * The regular files under `cwd` that `include` selects and `exclude` does not. + * + * Traversal is `API.Fs`'s: it reports directories and symbolic links too, and + * never follows one, which is what keeps the walk inside `cwd` and free of + * cycles. What this adds is the document-facing shape — regular files only, + * deduplicated, and sorted, so a document that branches on a listing branches + * the same way on every host. + */ + function* globFiles(input: GlobInput): Operation> { + try { + const info = yield* API.Fs.operations.stat(input.cwd); + if (!info.exists) { + return nonWriteFailure("glob", "target", "missing"); + } + if (!info.isDirectory) { + return nonWriteFailure("glob", "target", "not-directory"); + } + } catch (error) { + return nonWriteFailure("glob", "target", reasonOf(error)); + } + + try { + const matched = yield* traverse(input, observe); + const files = matched.filter((entry) => entry.isFile).map((entry) => entry.path); + return Ok([...new Set(files)].sort(byCodePoint)); + } catch (error) { + // The Api compiles patterns as it starts, so an unusable one — an + // unterminated character class — arrives as a `SyntaxError` from `RegExp` + // rather than as an errno. It is the one failure here a document can fix + // by editing what it wrote. + if (error instanceof SyntaxError) { + return nonWriteFailure("glob", "pattern", "invalid-pattern"); + } + return nonWriteFailure("glob", "traversal", reasonOf(error)); + } + } + + /** + * A directory this call created, named by its canonical path. + * + * Creation is synchronous so that nothing can suspend between it and the + * `ensure` that removes it. `until()` cannot cancel the promise it is waiting + * on, so an asynchronous `mkdtemp` halted mid-flight would go on to create a + * directory after the generator had already stopped — one nothing owns and + * nothing removes. Reading the canonical path does not suspend either, so the + * whole acquisition is a single uninterruptible step. + * + * `mkdtemp` names and creates at once, so the directory is never one an + * earlier run left behind. The path is then canonicalized: on macOS `tmpdir()` + * is a symlink (`/var/folders/…`) while a child process resolves it + * (`/private/var/…`), and canonicalizing is what makes the rendered path, the + * contextual directory, and a subprocess's own `cwd` the same string. + */ + function temporaryDirectory(): Operation> { + return resource(function* (provide) { + let created: string; + let canonical: string; + try { + created = mkdtempSync(join(tmpdir(), "xmd-tempdir-")); + canonical = realpathSync(created); + } catch (error) { + yield* provide(nonWriteFailure("temporary-directory", "acquire", reasonOf(error))); + return; + } + yield* ensure(() => discard(created)); + yield* provide(Ok(canonical)); + }); + } + + return { checkFilePath, readTextFile, writeTextFile, globFiles, temporaryDirectory }; +} + +/** + * Remove a temporary directory as its scope ends. + * + * A removal that fails during teardown is reported as an invariant rather than + * passed along: the failure it would otherwise carry names the generated + * directory, which the document never chose, and it can be unwinding a failure + * of its own that must not be replaced by platform text. + */ +function* discard(directory: string): Operation { + try { + yield* rm(directory, { recursive: true, force: true }); + } catch { + throw new FilesInvariantError("teardown"); + } +} + +/** + * Run the traversal, announcing each directory read when an observer is watching. + * + * The announcement is installed as `API.Fs` middleware for the duration of this + * one call rather than left in place, so an observer sees the reads this glob + * performs and nothing else. + */ +function traverse( + input: GlobInput, + observe: HostFilesObserver | undefined, +): Operation> { + const search = { patterns: input.include, root: input.cwd, exclude: input.exclude }; + if (observe === undefined) { + return API.Fs.operations.glob(search); + } + return scoped(function* () { + yield* FsApi.around({ + *readdirDirents([directory], next) { + observe({ operation: "glob", phase: "read-dir" }); + return yield* next(directory); + }, + }); + return yield* API.Fs.operations.glob(search); + }); +} + +/** + * Install the host provider beneath ordinary middleware. + * + * `at: "min"` is what lets a host wrap document filesystem access without + * replacing it — middleware installed later sees these operations and can + * delegate to them. + */ +export function useHostFiles(options: HostFilesOptions = {}): Operation { + const handler = hostFilesHandler(options); + return API.Files.around( + { + *checkFilePath([input]) { + return yield* handler.checkFilePath(input); + }, + *readTextFile([input]) { + return yield* handler.readTextFile(input); + }, + *writeTextFile([input]) { + return yield* handler.writeTextFile(input); + }, + *globFiles([input]) { + return yield* handler.globFiles(input); + }, + *temporaryDirectory() { + return yield* handler.temporaryDirectory(); + }, + }, + { at: "min" }, + ); +} diff --git a/packages/runtime/mod.ts b/packages/runtime/mod.ts index 8e468bee..4834d1ac 100644 --- a/packages/runtime/mod.ts +++ b/packages/runtime/mod.ts @@ -5,10 +5,12 @@ * `API` is available for middleware (`.around()`). * For normal calls, import operations directly. * - * Six domain APIs: + * Seven domain APIs: * - `API.Process` — subprocess execution (`exec`) - * - `API.Fs` — filesystem (`readTextFile`, `writeTextFile`, `stat`, `glob`, - * `realpath`, `ensureDir`, `rename`, `remove`) + * - `API.Fs` — the low-level host filesystem (`readTextFile`, `writeTextFile`, + * `stat`, `glob`, `realpath`, `ensureDir`, `rename`, `remove`) + * - `API.Files` — document filesystem access as whole semantic operations, + * with no host default. `useHostFiles()` installs the host provider. * - `API.Fetch` — HTTP requests (`fetch`) * - `API.Env` — the host: variables, platform info, the command that invokes * this xmd, and eval-block compilation @@ -64,3 +66,47 @@ export type { } from "./service.ts"; export { Config, timeout } from "./config.ts"; export type { ConfigApi } from "./config.ts"; +export { + asFilesFatal, + FILES_ERROR, + FILES_ERROR_MESSAGE, + FILES_FATAL, + FILES_INVARIANT_MESSAGE, + FILES_OPERATION_DENIED_MESSAGE, + FILES_PROVIDER_UNAVAILABLE_MESSAGE, + FILES_WRITE_SUCCESS, + Files, + FilesError, + FilesInvariantError, + FilesOperationDeniedError, + FilesProviderUnavailableError, + fileWriteFailure, + fileWriteSuccess, + filesFailure, + isFilesFatal, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, + parseFilesFatal, +} from "./files.ts"; +export type { + FilePathInput, + FilesDeniableOperation, + FilesErrorData, + FilesFailureData, + FilesFatalData, + FilesFatalFailure, + FilesHandler, + FilesInvariantCategory, + FilesOperation, + FilesPhase, + FilesReason, + FileWriteFailureData, + FileWriteInput, + FileWritePhase, + FileWriteSuccess, + FileWriteTarget, + GlobInput, +} from "./files.ts"; +export { hostFilesHandler, useHostFiles } from "./host-files.ts"; +export type { HostFilesEvent, HostFilesObserver, HostFilesOptions } from "./host-files.ts"; diff --git a/packages/runtime/tests/host-files.test.ts b/packages/runtime/tests/host-files.test.ts new file mode 100644 index 00000000..9d0ab7a6 --- /dev/null +++ b/packages/runtime/tests/host-files.test.ts @@ -0,0 +1,748 @@ +/** + * Tier HF — the host `API.Files` provider. + * + * These drive the provider directly rather than through a document, because + * what they assert is the contract a component cannot see: which phase a + * failure came from, that an ordinary condition is an `Err` and never a throw, + * that a missing provider throws and never falls back, and where the host + * guarantee actually stops. + * + * The replacement cases are the last of those. The host contract holds while + * the pathname namespace is stable, and the adapter's test-only observer is + * what makes "stable" falsifiable: it swaps part of the tree between the moment + * the adapter observes a path and the moment it uses one, synchronously, with + * no sleeping and no racing. What they prove is the documented weakness, not a + * containment claim. + * + * `mkdtemp`, `realpath`, `symlink`, `lstat`, and `readdir` have no + * `@effectionx/fs` equivalent; everything else goes through it. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, race, resource, scoped, sleep, spawn, suspend, until } from "effection"; +import type { Operation, Result } from "effection"; +import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { lstat, mkdir, mkdtemp, readdir, realpath, symlink } from "node:fs/promises"; +import { renameSync, symlinkSync, unlinkSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import process from "node:process"; +import { API } from "../apis.ts"; +import { hostFilesHandler, useHostFiles } from "../host-files.ts"; +import type { HostFilesEvent } from "../host-files.ts"; +import { + FILES_ERROR, + FILES_WRITE_SUCCESS, + Files, + FilesProviderUnavailableError, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, + parseFilesFatal, +} from "../files.ts"; +import type { FilesHandler } from "../files.ts"; + +/** A workspace, and a directory beside it that is deliberately out of reach. */ +interface Fixture { + root: string; + workspace: string; + outside: string; +} + +function useFixture(): Operation { + return resource(function* (provide) { + const root = yield* until(realpath(yield* until(mkdtemp(join(tmpdir(), "hf-test-"))))); + yield* ensure(() => rm(root, { recursive: true, force: true })); + const workspace = join(root, "workspace"); + const outside = join(root, "outside"); + yield* until(mkdir(workspace)); + yield* until(mkdir(outside)); + yield* provide({ root, workspace, outside }); + }); +} + +/** + * A directory symlink, spelled the way the running platform accepts one. + * + * A junction is what Windows gives an unprivileged process, and it is the + * reparse point the guarantee statement names; everywhere else it is an + * ordinary directory symlink. + */ +const DIRECTORY_LINK = process.platform === "win32" ? "junction" : "dir"; + +function linkDirectory(target: string, path: string): Operation { + return until(symlink(target, path, DIRECTORY_LINK)); +} + +function handler(observe?: (event: HostFilesEvent) => void): FilesHandler { + return hostFilesHandler(observe === undefined ? {} : { observe }); +} + +function failed(result: Result): unknown { + if (result.ok) { + throw new Error("expected a failure, got a value"); + } + return result.error; +} + +function value(result: Result): T { + if (!result.ok) { + throw result.error; + } + return result.value; +} + +/** Everything in a directory, sorted, so an assertion names a set. */ +function* entries(directory: string): Operation { + return (yield* until(readdir(directory))).sort(); +} + +/** A platform error shaped like the ones that leak: a code, and a path. */ +const PLANTED = "/planted/absolute/path/secret.txt"; + +function planted(code: string): Error { + return Object.assign(new Error(`${code}: operation failed, at '${PLANTED}'`), { code }); +} + +/** Every field a failure carries, as one string, for a leak assertion. */ +function inspected(error: unknown): string { + return JSON.stringify({ + message: error instanceof Error ? error.message : String(error), + data: error instanceof Error && "data" in error ? error.data : undefined, + }); +} + +describe("Tier HF — host Files provider", () => { + // HF1: the preliminary check is arithmetic. It has to be, because it runs + // before a write's children and its whole job is to cost nothing. + it("HF1: checkFilePath refuses inadmissible paths with no filesystem access", function* () { + const fixture = yield* useFixture(); + const touched: string[] = []; + + yield* scoped(function* () { + yield* API.Fs.around({ + *stat([path], next) { + touched.push(path); + return yield* next(path); + }, + *realpath([path], next) { + touched.push(path); + return yield* next(path); + }, + *readTextFile([path], next) { + touched.push(path); + return yield* next(path); + }, + }); + const files = handler(); + + for (const [path, reason] of [ + ["", "empty-path"], + [join(fixture.outside, "secret.txt"), "absolute-path"], + ["../outside/secret.txt", "lexical-escape"], + ]) { + const result = yield* files.checkFilePath({ cwd: fixture.workspace, path }); + expect(parseFilesFailure(failed(result))).toEqual({ + type: FILES_ERROR, + operation: "check-file-path", + phase: "lexical", + reason, + }); + } + + // And an admissible path answers with nothing usable. + const admitted = yield* files.checkFilePath({ cwd: fixture.workspace, path: "notes.md" }); + expect(admitted).toEqual({ ok: true, value: undefined }); + }); + + expect(touched).toEqual([]); + }); + + // HF2: the ordinary round trip, and the shape a success has. + it("HF2: a write commits and reads back, reporting a validated outcome", function* () { + const fixture = yield* useFixture(); + const files = handler(); + + const written = yield* files.writeTextFile({ + cwd: fixture.workspace, + path: "nested/notes.md", + content: "first", + }); + expect(parseFileWriteSuccess(value(written))).toEqual({ + type: FILES_WRITE_SUCCESS, + publication: "host-committed", + }); + + const read = yield* files.readTextFile({ cwd: fixture.workspace, path: "nested/notes.md" }); + expect(value(read)).toBe("first"); + // The commit consumed the temporary, so nothing is left beside the file. + expect(yield* entries(join(fixture.workspace, "nested"))).toEqual(["notes.md"]); + }); + + // HF3: the search's document-facing shape. Sorting and deduplication are the + // provider's, so a document that branches on a listing branches the same way + // wherever it runs. + it("HF3: globFiles returns sorted, deduplicated regular files", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "zebra.md"), "z"); + yield* writeTextFile(join(fixture.workspace, "Beta.md"), "b"); + yield* until(mkdir(join(fixture.workspace, "one"))); + yield* writeTextFile(join(fixture.workspace, "one/middle.md"), "m"); + yield* until(symlink(join(fixture.workspace, "Beta.md"), join(fixture.workspace, "link.md"))); + + const found = yield* handler().globFiles({ + cwd: fixture.workspace, + include: ["**/*.md", "**/*.md"], + exclude: [], + }); + + // A symbolic link is a link rather than a file, so it is not a result. + expect(value(found)).toEqual(["Beta.md", "one/middle.md", "zebra.md"]); + }); + + // HF4: every ordinary failure is a Result. Nothing throws, and nothing the + // platform said reaches the caller — a code that is not recognized selects + // the generic reason rather than being carried across. + it("HF4: platform failures become structured Errs, and leak nothing", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "existing"); + + const cases: Array<{ + code: string; + install: () => Operation; + expected: { phase: string; reason: string }; + }> = [ + { + code: "ELOOP", + expected: { phase: "resolution", reason: "too-many-symlinks" }, + *install() { + yield* API.Fs.around({ + *realpath([path], next) { + if (path.endsWith("notes.md")) { + throw planted("ELOOP"); + } + return yield* next(path); + }, + }); + }, + }, + { + code: "EACCES", + expected: { phase: "target", reason: "permission-denied" }, + *install() { + yield* API.Fs.around({ + *stat([path], next) { + if (path.endsWith("notes.md")) { + throw planted("EACCES"); + } + return yield* next(path); + }, + }); + }, + }, + { + code: "ENOTAREALCODE", + expected: { phase: "access", reason: "operation-failed" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *readTextFile() { + throw planted("ENOTAREALCODE"); + }, + }); + }, + }, + ]; + + for (const shape of cases) { + yield* scoped(function* () { + yield* shape.install(); + const result = yield* handler().readTextFile({ + cwd: fixture.workspace, + path: "notes.md", + }); + const error = failed(result); + expect(parseFilesFailure(error)).toEqual({ + type: FILES_ERROR, + operation: "read", + phase: shape.expected.phase, + reason: shape.expected.reason, + }); + const text = inspected(error); + expect(text).not.toContain(PLANTED); + expect(text).not.toContain(shape.code); + expect(text).not.toContain(fixture.workspace); + }); + } + }); + + // HF5: the write's phase table. Which phase a write stopped at is the only + // thing that decides what may be said about the target, so each one is + // produced from the outside and its whole structure checked. + it("HF5: each write phase reports a valid, phase-consistent outcome", function* () { + const fixture = yield* useFixture(); + + const cases: Array<{ + name: string; + install?: () => Operation; + path?: string; + prepare?: () => Operation; + expected: Record; + }> = [ + { + name: "lexical", + path: "../outside/planted.txt", + expected: { phase: "lexical", reason: "lexical-escape", target: "unchanged" }, + }, + { + name: "target", + path: "adirectory", + *prepare() { + yield* until(mkdir(join(fixture.workspace, "adirectory"))); + }, + expected: { phase: "target", reason: "directory", target: "unchanged" }, + }, + { + name: "parents", + expected: { phase: "parents", reason: "read-only", target: "unchanged" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *ensureDir() { + throw planted("EROFS"); + }, + }); + }, + }, + { + name: "temporary", + expected: { phase: "temporary", reason: "no-space", target: "unchanged" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *writeTextFile() { + throw planted("ENOSPC"); + }, + }); + }, + }, + { + name: "commit", + expected: { phase: "commit", reason: "cross-device", target: "commit-unknown" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *rename() { + throw planted("EXDEV"); + }, + }); + }, + }, + { + name: "cleanup", + expected: { phase: "cleanup", cleanup: "permission-denied", target: "committed" }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *remove() { + throw planted("EPERM"); + }, + }); + }, + }, + { + name: "commit and cleanup together", + expected: { + phase: "commit", + reason: "cross-device", + cleanup: "permission-denied", + target: "commit-unknown", + }, + *install() { + yield* API.Fs.around({ + // deno-lint-ignore require-yield + *rename() { + throw planted("EXDEV"); + }, + // deno-lint-ignore require-yield + *remove() { + throw planted("EPERM"); + }, + }); + }, + }, + ]; + + for (const shape of cases) { + if (shape.prepare) { + yield* shape.prepare(); + } + yield* scoped(function* () { + if (shape.install) { + yield* shape.install(); + } + const result = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: shape.path ?? `${shape.name.replace(/ /g, "-")}.txt`, + content: "replacement", + }); + const parsed = parseFileWriteFailure(failed(result)); + expect(parsed).toEqual({ + type: FILES_ERROR, + operation: "write", + ...shape.expected, + }); + expect(inspected(failed(result))).not.toContain(PLANTED); + }); + } + }); + + // HF6: containment, judged against what the adapter can observe. A symlink + // out is refused at resolution for both forms, and the destination is never + // named — reporting where a link pointed would perform the escape. + it("HF6: a symlink leaving the working directory is refused, naming nothing", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.outside, "secret.txt"), "SECRET"); + yield* until(symlink(join(fixture.outside, "secret.txt"), join(fixture.workspace, "escape"))); + const files = handler(); + + const read = yield* files.readTextFile({ cwd: fixture.workspace, path: "escape" }); + expect(parseFilesFailure(failed(read))?.reason).toBe("resolved-escape"); + expect(inspected(failed(read))).not.toContain("SECRET"); + expect(inspected(failed(read))).not.toContain(fixture.outside); + + const written = yield* files.writeTextFile({ + cwd: fixture.workspace, + path: "escape", + content: "planted", + }); + expect(parseFileWriteFailure(failed(written))).toEqual({ + type: FILES_ERROR, + operation: "write", + phase: "resolution", + reason: "resolved-escape", + target: "unchanged", + }); + expect(yield* readTextFile(join(fixture.outside, "secret.txt"))).toBe("SECRET"); + }); + + // HF7: an internal link is followed rather than replaced, which is what makes + // reading back through either name agree. + it("HF7: an internal symlink is followed for a write", function* () { + const fixture = yield* useFixture(); + yield* until(mkdir(join(fixture.workspace, "real"))); + yield* writeTextFile(join(fixture.workspace, "real/target.txt"), "original"); + yield* until( + symlink(join(fixture.workspace, "real/target.txt"), join(fixture.workspace, "link.txt")), + ); + + yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "link.txt", + content: "replaced", + }); + + expect(yield* readTextFile(join(fixture.workspace, "real/target.txt"))).toBe("replaced"); + expect((yield* until(lstat(join(fixture.workspace, "link.txt")))).isSymbolicLink()).toBe(true); + }); + + // HF8: a dangling final link is the one hole resolution cannot close — there + // is nothing to resolve — and the temporary plus rename is what closes it. + // The link is replaced rather than followed to wherever it pointed. + it("HF8: a write replaces a dangling symlink instead of following it", function* () { + const fixture = yield* useFixture(); + yield* until(symlink(join(fixture.outside, "absent.txt"), join(fixture.workspace, "dangling"))); + + const written = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "dangling", + content: "content", + }); + + expect(parseFileWriteSuccess(value(written))?.publication).toBe("host-committed"); + expect(yield* readTextFile(join(fixture.workspace, "dangling"))).toBe("content"); + expect((yield* until(lstat(join(fixture.workspace, "dangling")))).isSymbolicLink()).toBe(false); + expect(yield* exists(join(fixture.outside, "absent.txt"))).toBe(false); + }); + + // HF9: the observer's own contract. Production installs no observer, so this + // fixes what a test may rely on: the private phases, in order, once each. + it("HF9: the observer reports each private phase in order", function* () { + const fixture = yield* useFixture(); + const seen: string[] = []; + const files = handler((event) => seen.push(`${event.operation}.${event.phase}`)); + + yield* files.writeTextFile({ cwd: fixture.workspace, path: "a.txt", content: "a" }); + expect(seen).toEqual([ + "write.target", + "write.parents", + "write.temporary", + "write.commit", + "write.cleanup", + ]); + + seen.length = 0; + yield* files.readTextFile({ cwd: fixture.workspace, path: "a.txt" }); + expect(seen).toEqual(["read.target", "read.access"]); + + seen.length = 0; + yield* files.globFiles({ cwd: fixture.workspace, include: ["**/*"], exclude: [] }); + expect(seen).toEqual(["glob.read-dir"]); + }); + + // HF10: where the host guarantee stops, made observable. The parent is + // replaced by a link out of the workspace between the moment the adapter + // resolved the path and the moment it creates parents — synchronously, in one + // uninterrupted step, which is exactly what another process can do and what + // no shipped runtime lets this adapter prevent. + // + // What is asserted is the documented weakness. A test that expected refusal + // here would be asserting a containment claim `xmd run` does not make. + it("HF10: a parent replaced after resolution is observed by the later calls", function* () { + const fixture = yield* useFixture(); + yield* until(mkdir(join(fixture.workspace, "parent"))); + + let swapped = false; + const files = handler((event) => { + if (swapped || event.operation !== "write" || event.phase !== "parents") { + return; + } + swapped = true; + // Synchronous, so nothing runs between the observation and the use. + renameSync(join(fixture.workspace, "parent"), join(fixture.root, "moved")); + symlinkSync(fixture.outside, join(fixture.workspace, "parent"), DIRECTORY_LINK); + }); + + const written = yield* files.writeTextFile({ + cwd: fixture.workspace, + path: "parent/planted.txt", + content: "complete replacement", + }); + + expect(swapped).toBe(true); + // The write committed — through the replacement, which is the limitation. + expect(parseFileWriteSuccess(value(written))?.publication).toBe("host-committed"); + expect(yield* readTextFile(join(fixture.outside, "planted.txt"))).toBe("complete replacement"); + // Atomicity still holds: what landed is the whole file, never a fragment, + // and no temporary was left beside it. + expect(yield* entries(fixture.outside)).toEqual(["planted.txt"]); + }); + + // HF10b: the same seam on a read. A file swapped for a link out after + // resolution is read through, and the failure that would have been reported + // is not manufactured. + it("HF10b: a target replaced after resolution is read through", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "inside"); + yield* writeTextFile(join(fixture.outside, "secret.txt"), "SECRET"); + + let swapped = false; + const files = handler((event) => { + if (swapped || event.operation !== "read" || event.phase !== "access") { + return; + } + swapped = true; + unlinkSync(join(fixture.workspace, "notes.md")); + symlinkSync(join(fixture.outside, "secret.txt"), join(fixture.workspace, "notes.md")); + }); + + const read = yield* files.readTextFile({ cwd: fixture.workspace, path: "notes.md" }); + + expect(swapped).toBe(true); + expect(value(read)).toBe("SECRET"); + }); + + // HF11: rename is reached only as the commit phase, and a middleware fault on + // either side of it is the same event from where the adapter stands. Both + // report `commit-unknown`, and they have to: one of the two runs did commit. + it("HF11: a commit fault before and after next reports the same unknown outcome", function* () { + const fixture = yield* useFixture(); + + for (const side of ["before", "after"] as const) { + yield* writeTextFile(join(fixture.workspace, `${side}.md`), "first"); + const result = yield* scoped(function* () { + yield* API.Fs.around({ + *rename([from, to], next) { + if (side === "after") { + yield* next(from, to); + } + throw planted("EXDEV"); + }, + }); + return yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: `${side}.md`, + content: "second", + }); + }); + + expect(parseFileWriteFailure(failed(result))).toEqual({ + type: FILES_ERROR, + operation: "write", + phase: "commit", + reason: "cross-device", + target: "commit-unknown", + }); + } + + // The claim is honest about both: one run kept the previous file and one + // replaced it, and the report was the same. + expect(yield* readTextFile(join(fixture.workspace, "before.md"))).toBe("first"); + expect(yield* readTextFile(join(fixture.workspace, "after.md"))).toBe("second"); + }); + + // HF12: cancellation is not a Result. It resumes the generator rather than + // throwing, so nothing here can convert it into a write outcome — and the + // temporary is still removed, because its cleanup was registered before the + // step most likely to be interrupted. + it("HF12: cancellation produces no Result and leaves no temporary", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "first"); + let settled: unknown = "not settled"; + + yield* race([ + scoped(function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + yield* next(path, content); + yield* suspend(); + }, + }); + settled = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "notes.md", + content: "second", + }); + }), + sleep(250), + ]); + + expect(settled).toBe("not settled"); + expect(yield* readTextFile(join(fixture.workspace, "notes.md"))).toBe("first"); + expect(yield* entries(fixture.workspace)).toEqual(["notes.md"]); + }); + + // HF13: the temporary directory is a resource, so its lifetime is the + // acquiring scope's and a halt cannot land between creating it and owning its + // removal. + it("HF13: a temporary directory lives and dies with its acquiring scope", function* () { + const files = handler(); + let acquired = ""; + + yield* scoped(function* () { + acquired = value(yield* files.temporaryDirectory()); + expect(yield* exists(acquired)).toBe(true); + }); + expect(yield* exists(acquired)).toBe(false); + + // Halted before the acquiring task ran at all: an acquisition that + // suspended on a pending creation would finish afterwards and leave a + // directory nothing owns. + const early = yield* spawn(() => files.temporaryDirectory()); + yield* early.halt(); + yield* sleep(50); + const stale = (yield* entries(yield* until(realpath(tmpdir())))).filter((entry) => + entry.startsWith("xmd-tempdir-"), + ); + expect(stale.includes(acquired)).toBe(false); + }); + + // HF14: absence is fatal, for every operation. A default that reached the + // host would make an uninstalled provider indistinguishable from an installed + // one, which is the whole reason this Api has no host default. + it("HF14: with no provider installed every operation throws and touches nothing", function* () { + const fixture = yield* useFixture(); + const touched: string[] = []; + + yield* scoped(function* () { + yield* API.Fs.around({ + *stat([path], next) { + touched.push(path); + return yield* next(path); + }, + *realpath([path], next) { + touched.push(path); + return yield* next(path); + }, + *readTextFile([path], next) { + touched.push(path); + return yield* next(path); + }, + *writeTextFile([path, content], next) { + touched.push(path); + return yield* next(path, content); + }, + }); + + const calls: Array> = [ + Files.operations.checkFilePath({ cwd: fixture.workspace, path: "a.md" }), + Files.operations.readTextFile({ cwd: fixture.workspace, path: "a.md" }), + Files.operations.writeTextFile({ cwd: fixture.workspace, path: "a.md", content: "x" }), + Files.operations.globFiles({ cwd: fixture.workspace, include: ["*"], exclude: [] }), + Files.operations.temporaryDirectory(), + ]; + + for (const call of calls) { + let thrown: unknown; + try { + yield* call; + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(FilesProviderUnavailableError); + expect(parseFilesFatal(thrown)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "provider-unavailable", + }); + // Fixed diagnostics, and no cause to inspect. + expect(thrown instanceof Error ? thrown.message : "").toBe( + "Files provider is not installed", + ); + expect(thrown instanceof Error ? thrown.cause : "unset").toBeUndefined(); + } + }); + + expect(touched).toEqual([]); + }); + + // HF15: installation is what `useHostFiles` is for, and it goes beneath + // ordinary middleware so a host can still wrap document filesystem access. + it("HF15: useHostFiles installs beneath middleware that can wrap it", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "underneath"); + + const observed: string[] = []; + const read = yield* scoped(function* () { + yield* useHostFiles(); + yield* Files.around({ + *readTextFile([input], next) { + observed.push(input.path); + return yield* next(input); + }, + }); + return yield* Files.operations.readTextFile({ cwd: fixture.workspace, path: "notes.md" }); + }); + + expect(value(read)).toBe("underneath"); + expect(observed).toEqual(["notes.md"]); + }); + + // HF16: a junction is the Windows shape of the same limitation, and this is + // the row that names it. Elsewhere it is an ordinary directory symlink, so + // the case runs on every target rather than only where the reparse point + // exists. + it("HF16: a directory link is refused at resolution on every platform", function* () { + const fixture = yield* useFixture(); + yield* linkDirectory(fixture.outside, join(fixture.workspace, "escape")); + + const written = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "escape/planted.txt", + content: "planted", + }); + + expect(parseFileWriteFailure(failed(written))?.reason).toBe("resolved-escape"); + expect(yield* entries(fixture.outside)).toEqual([]); + }); +}); diff --git a/scripts/files-contract-probe.ts b/scripts/files-contract-probe.ts new file mode 100644 index 00000000..9ef6b667 --- /dev/null +++ b/scripts/files-contract-probe.ts @@ -0,0 +1,173 @@ +/** + * The host `API.Files` contract, as an executable a release target can run. + * + * `packages/runtime/tests/host-files.test.ts` proves the same contract from + * source. This exists because the shipped artifact is a compiled binary, and + * the host adapter reaches `node:path`, `node:fs`, and `node:os` — the modules + * whose behavior a `deno compile` graph, and the platform it was compiled for, + * can change. A source suite that passes says nothing about that. + * + * So this is deliberately small and self-verifying: it asserts the contract's + * observable claims and exits non-zero if any does not hold. It prints every + * claim it checked, because a probe that passes silently is indistinguishable + * from one that checked nothing. + * + * Usage: + * deno run --allow-all scripts/files-contract-probe.ts + * deno compile --allow-all --output scripts/files-contract-probe.ts + */ + +import { ensure, exit, main, scoped } from "effection"; +import type { Result } from "effection"; +import { mkdirSync, mkdtempSync, realpathSync, rmSync, statSync, symlinkSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import process from "node:process"; +import { + Files, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, + parseFilesFatal, + useHostFiles, +} from "@executablemd/runtime"; + +const checked: string[] = []; +const failures: string[] = []; + +function check(claim: string, held: boolean): void { + checked.push(claim); + if (!held) { + failures.push(claim); + } +} + +function reasonOf(result: Result): string | undefined { + return result.ok ? undefined : parseFilesFailure(result.error)?.reason; +} + +/** + * A write reports its own shape. Reading one with the non-write parser is how a + * probe silently checks nothing, so the two are separate here. + */ +function writeReasonOf(result: Result): string | undefined { + return result.ok ? undefined : parseFileWriteFailure(result.error)?.reason; +} + +function valueOf(result: Result): T | undefined { + return result.ok ? result.value : undefined; +} + +function exists(path: string): boolean { + try { + statSync(path); + return true; + } catch { + return false; + } +} + +/** A junction is what an unprivileged Windows process gets; elsewhere, a symlink. */ +const DIRECTORY_LINK = process.platform === "win32" ? "junction" : "dir"; + +await main(function* () { + const root = realpathSync(mkdtempSync(join(tmpdir(), "xmd-files-probe-"))); + yield* ensure(() => rmSync(root, { recursive: true, force: true })); + const workspace = join(root, "workspace"); + const outside = join(root, "outside"); + mkdirSync(workspace); + mkdirSync(outside); + + // Absence is checked before a provider exists, so the terminal handler is the + // one answering. + let absent: unknown; + try { + yield* Files.operations.readTextFile({ cwd: workspace, path: "notes.md" }); + } catch (error) { + absent = error; + } + check( + "an absent provider throws provider-unavailable", + parseFilesFatal(absent)?.kind === "provider-unavailable", + ); + + yield* useHostFiles(); + + check( + "an empty path is refused lexically", + reasonOf(yield* Files.operations.checkFilePath({ cwd: workspace, path: "" })) === "empty-path", + ); + check( + "an absolute path is refused lexically", + reasonOf( + yield* Files.operations.checkFilePath({ cwd: workspace, path: join(outside, "secret.txt") }), + ) === "absolute-path", + ); + check( + "a lexical escape is refused", + reasonOf( + yield* Files.operations.checkFilePath({ cwd: workspace, path: "../outside/secret.txt" }), + ) === "lexical-escape", + ); + check( + "an admissible path is admitted", + (yield* Files.operations.checkFilePath({ cwd: workspace, path: "notes.md" })).ok, + ); + + const written = yield* Files.operations.writeTextFile({ + cwd: workspace, + path: "nested/notes.md", + content: "probe content", + }); + check( + "a write commits to the host", + parseFileWriteSuccess(valueOf(written))?.publication === "host-committed", + ); + check( + "the write reads back", + valueOf(yield* Files.operations.readTextFile({ cwd: workspace, path: "nested/notes.md" })) === + "probe content", + ); + check("the commit left no temporary behind", !exists(join(workspace, "nested/notes.md.tmp"))); + + symlinkSync(outside, join(workspace, "escape"), DIRECTORY_LINK); + check( + "a directory link out of the working directory is refused", + writeReasonOf( + yield* Files.operations.writeTextFile({ + cwd: workspace, + path: "escape/planted.txt", + content: "planted", + }), + ) === "resolved-escape", + ); + check("nothing was written through the link", !exists(join(outside, "planted.txt"))); + + const found = yield* Files.operations.globFiles({ + cwd: workspace, + include: ["**/*.md"], + exclude: [], + }); + check( + "the search returns POSIX-relative files", + JSON.stringify(valueOf(found)) === '["nested/notes.md"]', + ); + + let temporary = ""; + yield* scoped(function* () { + temporary = valueOf(yield* Files.operations.temporaryDirectory()) ?? ""; + check("a temporary directory is acquired", temporary.length > 0 && exists(temporary)); + }); + check("a temporary directory is removed with its scope", !exists(temporary)); + + for (const claim of checked) { + console.log(`${failures.includes(claim) ? "FAIL" : "ok "} ${claim}`); + } + if (failures.length > 0) { + console.error(`files contract: ${failures.length} of ${checked.length} claims failed`); + yield* exit(1); + } + console.log( + `files contract: ${checked.length} claims hold on ${process.platform}/${process.arch}`, + ); +}); diff --git a/scripts/tests/filesystem-contract-workflow.test.ts b/scripts/tests/filesystem-contract-workflow.test.ts new file mode 100644 index 00000000..a65e599d --- /dev/null +++ b/scripts/tests/filesystem-contract-workflow.test.ts @@ -0,0 +1,152 @@ +/** + * The five-target filesystem contract job, held to the release matrix. + * + * The claim `xmd run` makes about containment is per-platform: path arithmetic + * and `realpath` are the host's, and Windows brings drive letters, UNC paths, + * junctions, and reparse points that POSIX does not. A job that covered four of + * the five triples would leave the fifth's claim asserted and unproven, and + * nothing about adding a release target would notice. + * + * So the matrix is held to `RELEASE_TARGETS` by set equality, the same way + * `publish-workflow-membership.test.ts` holds the release workflow. Adding a + * target without adding a row fails here. + */ + +import matter from "gray-matter"; +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Operation } from "effection"; +import { readTextFile } from "@effectionx/fs"; +import { RELEASE_TARGETS } from "../lib/release-targets.ts"; + +const CI_WORKFLOW = new URL("../../.github/workflows/ci.yml", import.meta.url); +const JOB = "filesystem-contract"; +const PROBE = "scripts/files-contract-probe.ts"; +const SUITE = "packages/runtime/tests/host-files.test.ts"; + +function object(value: unknown, label: string): Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new Error(`${label} is not an object`); + } + return Object.fromEntries(Object.entries(value)); +} + +function string(value: unknown, label: string): string { + if (typeof value !== "string") { + throw new Error(`${label} is not a string`); + } + return value; +} + +interface Row { + runner: string; + target: string; +} + +interface ContractJob { + runsOn: string; + rows: Row[]; + commands: string[]; +} + +function* job(): Operation { + const source = yield* readTextFile(CI_WORKFLOW); + const document = object(matter(`---\n${source}\n---`).data, "workflow"); + const jobs = object(document.jobs, "workflow.jobs"); + const contract = object(jobs[JOB], `workflow.jobs.${JOB}`); + const strategy = object(contract.strategy, `${JOB}.strategy`); + const matrix = object(strategy.matrix, `${JOB}.strategy.matrix`); + const include = matrix.include; + if (!Array.isArray(include)) { + throw new Error(`${JOB}.strategy.matrix.include is not an array`); + } + const steps = contract.steps; + if (!Array.isArray(steps)) { + throw new Error(`${JOB}.steps is not an array`); + } + + return { + runsOn: string(contract["runs-on"], `${JOB}.runs-on`), + rows: include.map((entry, index) => { + const row = object(entry, `${JOB}.strategy.matrix.include[${index}]`); + return { + runner: string(row.runner, `include[${index}].runner`), + target: string(row.target, `include[${index}].target`), + }; + }), + commands: steps.flatMap((entry, index) => { + const step = object(entry, `${JOB}.steps[${index}]`); + return "run" in step ? [string(step.run, `${JOB}.steps[${index}].run`)] : []; + }), + }; +} + +describe("the filesystem contract matrix", () => { + it("covers exactly the release targets", function* () { + const contract = yield* job(); + expect([...contract.rows.map((row) => row.target)].sort()).toEqual( + Object.keys(RELEASE_TARGETS).sort(), + ); + }); + + it("gives every target its own runner", function* () { + const contract = yield* job(); + const runners = contract.rows.map((row) => row.runner); + expect(new Set(runners).size).toEqual(runners.length); + expect(contract.runsOn).toEqual("${{ matrix.runner }}"); + }); + + it("names a runner whose platform matches its target", function* () { + const platforms: Record = { + "macos-15": { os: "darwin", arch: "arm64" }, + "macos-15-intel": { os: "darwin", arch: "x64" }, + "ubuntu-24.04": { os: "linux", arch: "x64" }, + "ubuntu-24.04-arm": { os: "linux", arch: "arm64" }, + "windows-2025": { os: "win32", arch: "x64" }, + }; + + const contract = yield* job(); + for (const row of contract.rows) { + const runner = platforms[row.runner]; + if (runner === undefined) { + throw new Error(`no platform recorded for runner "${row.runner}"`); + } + expect(runner).toEqual(RELEASE_TARGETS[row.target]); + } + }); + + // Every row must run all four: the source suite under each runtime the + // project ships, and the compiled probe. A row that only ran one of them + // would report "the contract holds" for a shape it never executed. + it("runs the contract under Deno, Node, Bun, and a compiled binary", function* () { + const commands = (yield* job()).commands.join("\n"); + + expect(commands).toContain(`deno test --allow-all --frozen ${SUITE}`); + expect(commands).toContain(`tsx --tsconfig tsconfig.node.json --test ${SUITE}`); + expect(commands).toContain(`bun test --timeout=300000 ${SUITE}`); + expect(commands).toContain(`deno compile`); + expect(commands).toContain(PROBE); + }); + + // The Deno steps come before either package manager's install, because each + // rewrites `node_modules` into its own layout and a Deno run afterwards + // resolves through links the other pruned (#279). + it("runs every Deno step before a package manager rewrites node_modules", function* () { + const commands = (yield* job()).commands; + const lastDeno = commands.findLastIndex((command) => command.includes("deno ")); + const firstInstall = commands.findIndex( + (command) => command.trim() === "pnpm install" || command.trim() === "bun install", + ); + + expect(lastDeno).toBeGreaterThan(-1); + expect(firstInstall).toBeGreaterThan(lastDeno); + }); + + it("does not duplicate the whole corpus on every row", function* () { + const commands = (yield* job()).commands.join("\n"); + + expect(commands).not.toContain("deno task test\n"); + expect(commands).not.toContain("pnpm test:node"); + expect(commands).not.toContain("bun run test:bun"); + }); +}); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 97836724..8ea228ca 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -116,12 +116,31 @@ paths relative to cwd, and the engine's own file access is written that way: component search directories (`["./components", "./"]`) are relative, and resolved paths in the journal (`"components/Greeting.md"`) are relative. -A component that resolves against the **contextual** working directory is the -exception, because a relative path would resolve against the process's -directory instead of the one it was given. `` (§6.13) resolves its -`path` prop against `Env.cwd` itself and hands the Fs Api the absolute result; -`` (§6.14) searches `Env.cwd` and returns relative paths. Nothing either -resolves reaches a printed error or the journal. +#### Document data and engine control plane + +Two kinds of filesystem access are separate boundaries, and the separation is +what lets one document mean the same thing in two environments. + +**Document data** — the files a document names in its own text — goes through +`API.Files`, a contextual Api of whole semantic operations. `` (§6.13), +`` (§6.14), and `` (§6.11) speak only that Api, hold no host +path, and never learn which provider answered. `xmd run` installs a host +provider that resolves those paths in the caller's filesystem; a workflow run +installs one whose paths name entries in a logical filesystem the run owns. +The Api has **no host default**: with no provider installed, every operation +fails the execution rather than reaching the host. + +**The engine's own control plane** — the root document, component search, +replay guards, the eval compiler, the diagnostic journal, and the test target +— reads host paths the caller selected, through the low-level `API.Fs`. Those +are not document-addressable, and they stay where they are. + +A path a document authored is always relative and always resolved by the +provider, against the contextual `Env.cwd` the component supplies with it. +Nothing the provider resolves reaches a printed error or the journal: what +crosses back is a reason from a fixed vocabulary (§6.13), never a resolved +path, a symlink target, a temporary name, an errno code, or a platform +message. #### The contextual working directory @@ -4493,6 +4512,26 @@ remove the directory through structured concurrency, and there is no `retain` prop and no retention after failure. A future execution-level inspection policy may keep scoped resources without changing this component's contract. +#### Where the directory comes from + +`` creates nothing itself. It asks the installed `API.Files` provider +(§1.2) for a temporary directory, and the provider owns creation, the canonical +path, and removal as one acquisition — so nothing can land between creating a +directory and owning its removal, and a cancellation arriving mid-acquisition +cannot leave one behind. + +A provider is allowed to have none to give. A run whose whole filesystem is a +database transaction has no temporary directories, and inventing a logical one +would put the content somewhere the run does not own — so that provider refuses +the operation outright, with the fixed diagnostic `Files provider does not +support temporary-directory`. The refusal is **fatal** on §6.13's terms: the +content does not run, nothing is rendered, and no later sibling expands. It is +never a printed error, because there is no directory for the document to work +in and carrying on would mean carrying on somewhere else. + +With no provider installed at all, `` fails the same way and for the +same reason (§6.13). + #### Why the path is canonical `` renders the directory's resolved path, not the one the host's @@ -4550,16 +4589,30 @@ something a document reports, and `ContentError` is a public type an author constructs and subclasses, so what one carries underneath is never taken as a guarantee that nothing fatal is inside it. -**A durability failure takes precedence over a documentation failure, whatever -the wrapper order.** A wrapper carries whatever failed together, in whatever -order the platform happened to collect it: an `AggregateError`'s members and an -`InvocationTeardownError`'s stage failures are both positional. Precedence is -therefore decided by kind rather than by position — the cause graph is searched -for a durability failure first, and only a graph without one reports a -documentation failure. Position-based discovery would let one ordering of the -same teardown report the document's failure instead, which `` would then -record as an ordinary `error` outcome onto a journal already known not to -describe the run. +**A Files infrastructure failure is fatal on the same terms.** A missing +document filesystem provider, an operation a provider refuses, and a provider +that broke its own contract (§6.13) are none of them things a document did or +can act on. Each is discovered through the same cycle-safe traversal, each +crosses a `ContentError` the way a durability failure does, and each is +recognized by its structural tag rather than by class — so a failure a +separately loaded copy of the runtime package constructed is found on the same +terms as one this copy did. + +**Precedence is decided by kind rather than by position.** A wrapper carries +whatever failed together, in whatever order the platform happened to collect +it: an `AggregateError`'s members and an `InvocationTeardownError`'s stage +failures are both positional. The cause graph is therefore searched for a +durability failure first, then for a Files infrastructure failure, and only a +graph with neither reports a documentation failure. Position-based discovery +would let one ordering of the same teardown report the document's failure +instead, which `` would then record as an ordinary `error` outcome onto a +journal already known not to describe the run. + +Whichever is selected comes back **by identity** — the object that was thrown, +not a replacement — because a fail-stop that records "the first error" has to +record the one that happened. Only an output-mode `DocumentationError` is a +decision a printing boundary may still act on; a durability failure and a Files +infrastructure failure never are. This is a limitation of the current durable model, not of the component. The `import_component` entry recording that `` resolved to core's @@ -4757,12 +4810,27 @@ reachable beneath it, carrying the same error segments the document reported — reporting the component's account costs nothing that a host inspecting the failure needs. +#### The provider boundary + +`` makes no filesystem call of its own. It calls `API.Files` (§1.2), a +contextual Api of whole semantic operations, and what "the filesystem" means +belongs to whichever provider is installed. `xmd run` installs a host provider; +a workflow run installs one whose paths name entries in a logical filesystem +the run owns. The component holds no host path, learns no resolved name, and +receives no handle or capability — the two forms it performs are `readTextFile` +and `writeTextFile`, and each is one call. + +What the component owns is **order**, and it owns it because ordering is what +containment depends on. + #### Containment Everything `` touches stays inside `Env.cwd`, checked in two stages that -answer different questions and therefore run at different times. +answer different questions and therefore run at different times. Both stages +are the provider's; the component decides when each happens. -The **lexical** stage is path arithmetic against `Env.cwd` and nothing else. +The **lexical** stage is `checkFilePath`: path arithmetic against `Env.cwd` and +nothing else. An empty path, an absolute path, and a `..` escape are all decided there, before any filesystem call — so the failure reveals nothing about what the path named. Only a complete `..` segment escapes: a name that merely begins @@ -4776,6 +4844,11 @@ For the write form this stage runs **before the children expand**. An unusable path costs nothing, and the printed error it produces is about the path rather than about whatever the children then did. +`checkFilePath` returns nothing usable — no path, no resolved name, no handle, +no capability. It answers one question, "may the children run?", and the answer +authorizes nothing else. A check that was skipped, replaced by middleware, or +answered by a different provider therefore cannot admit the write that follows. + A lexical check is not enough on its own, because a symlink inside the directory can point anywhere. The **resolving** stage takes the part of the path that already exists — the file itself when it is there, the deepest @@ -4784,24 +4857,37 @@ destination is still inside the directory is ordinary and is followed to the file it names; one that leaves is refused, before the content outside is read or changed. -For the write form this stage runs **after the children have finished**, and -immediately before the write. A child can change what a path means — replacing -a directory with a symlink out of the workspace — so a destination resolved -any earlier would not be the one the write lands on. - -Writes land through a sibling temporary file and a rename, which also closes -the one case resolution cannot: a **dangling** symlink has nothing to resolve, -and `rename` replaces the link rather than following it wherever it points. -Removal of the temporary is registered before it is written, so the write is -covered by it rather than the other way round, and removal is attempted on every -exit including cancellation. +For the write form this stage is inside `writeTextFile`, which runs **after the +children have finished**. A child can change what a path means — replacing a +directory with a symlink out of the workspace — so a destination resolved any +earlier would not be the one the write lands on. That one call **repeats +lexical admission** from the same authored path and contextual directory and +then owns every later step: resolution, target classification, parent creation, +and the commit. Nothing is handed between the two stages, which is why the +earlier check cannot be turned into authority for the later write. + +The read form is one call, `readTextFile`, and that call owns admission, +resolution, target classification, and the read together. + +A **host** provider lands writes through a sibling temporary file and a rename, +which also closes the one case resolution cannot: a dangling symlink has +nothing to resolve, and `rename` replaces the link rather than following it +wherever it points. Removal of the temporary is registered before it is +written, so the write is covered by it rather than the other way round, and +removal is attempted on every exit including cancellation. Reading a path that does not exist, or a directory, fails naming which it was. #### The commit point -The rename is the write's commit point, and the guarantees are stated around -it: +Every provider commits, but not every provider commits the same way, and the +difference is visible only when a write fails. A host provider's commit is a +rename; a transaction-bound provider's is a savepoint released into the +transaction that owns it. A successful write renders nothing under either, so +the distinction never reaches a document that succeeds. + +The rename is the **host** write's commit point, and the guarantees are stated +around it: - A failure or a cancellation **before** the rename leaves the previous target exactly as it was. Nothing has replaced it yet. @@ -4816,24 +4902,35 @@ write can be taken back. ##### What a failed write can say about the target -`rename` is an operation on the contextual Fs Api, and an `around` handler may +A failed write reports **where it stopped**, and where it stopped is what +decides what may be said about the target. The provider names the phase; the +component chooses the sentence. No other combination exists, and a provider +that reports one is a provider that broke its contract (below). + +A rename is an operation on the contextual Fs Api, and an `around` handler may do work on both sides of `next()`. So a rename that **throws** may have thrown -before the underlying rename ran, or after it succeeded, and the component -cannot tell which. The three outcomes it reports are exactly what it can -observe: +before the underlying rename ran, or after it succeeded, and no provider can +tell which. | Where the write stopped | What is reported | |---|---| +| Admission, resolution, target, or parent creation | no outcome sentence — nothing was attempted on the target | | Preparation — writing the temporary | `The previous file is unchanged.` | -| The rename threw | `Whether the replacement committed is unknown: the target holds either the complete previous content or the complete replacement, never a partial write.` | -| The rename returned | `The file was written.` | +| The commit threw | `Whether the replacement committed is unknown: the target holds either the complete previous content or the complete replacement, never a partial write.` | +| The commit returned, cleanup failed | `The file was written.` | +| A transaction rolled the change back | `The Workspace change was rolled back.` | -Only the first and third are conclusions. The middle one is the honest answer: -atomicity still holds, so it is one of two whole files, but which one is not -knowable from here. Reporting that the previous file survived would be a guess, -and wrong in exactly the case where a handler failed after committing. +The unknown row is the honest answer rather than a missing one: atomicity still +holds, so it is one of two whole files, but which one is not knowable from +there. Reporting that the previous file survived would be a guess, and wrong in +exactly the case where a handler failed after committing. -A failed cleanup is orthogonal and composes with any of the three, appending: +The last row belongs to a transaction-bound provider and cannot appear from a +host one; equally, no host rename or temporary-leftover wording appears from a +transaction-bound one. A rolled-back change is a conclusion — the write did not +happen, and nothing is left over. + +A failed cleanup is orthogonal to all of them and appends: ```text A temporary file beside it may remain. @@ -4849,17 +4946,68 @@ refusal exists to prevent. A platform error carries the path it failed on — `ENOTDIR: not a directory, stat '/private/var/…'` — so forwarding one would leak exactly what the rest of -this withholds. Every filesystem call is wrapped, and nothing from the error it -caught is reproduced: the errno code **selects** a phrase from a fixed -allowlist, and an unrecognized code selects `the filesystem operation failed`. -The code itself is never emitted. It is supplied by whatever implements the Fs -Api, so it can hold a path, a newline, or a comment terminator as easily as -`ENOENT` can. +this withholds. Nothing from one crosses the provider boundary. What a provider +returns is a **reason** drawn from a fixed vocabulary, and the reason *selects* +a phrase; an unrecognized reason, and a condition the provider could not +classify, both select `the filesystem operation failed`. + +The reasons are: + +```text +empty-path, absolute-path, lexical-escape, resolved-escape, missing, +directory, special-file, not-directory, permission-denied, read-only, +too-many-symlinks, path-too-long, no-space, quota-exhausted, cross-device, +busy, too-many-open-files, directory-not-empty, invalid-pattern, +operation-failed +``` + +A failure carries that reason and the phase it came from, as a plain frozen +object under a stable tag, and the component **parses** it before reading a +field. Data that does not validate is treated as absent rather than trusted: +for a read or a search that means the generic phrase, and for a write it is a +provider-contract failure (below), because every sentence a write could print +makes a claim about whether the file was replaced. The error's class carries no authority either. A `FileAccessError` arriving -from a wrapped call is replaced like any other, because a class says nothing -about whether a message is safe to show — trusting one would let an Fs -implementation choose the text of a printed error by choosing what to throw. +from a provider call is replaced like any other, because a class says nothing +about whether a message is safe to show — trusting one would let a provider +choose the text of a printed error by choosing what to throw. Recognition is by +structural tag rather than by `instanceof` for the same reason two copies of the +runtime package can be loaded at once, and `instanceof` answers false across +them. + +#### When the provider is the problem + +An ordinary filesystem condition is something the document did and can act on. +Three things are not, and none of them becomes a printed error: + +- **No provider is installed.** Every operation fails with the fixed diagnostic + `Files provider is not installed`. The write form reaches it at + `checkFilePath`, so it lands before the children; the read form, ``, + and `` reach it at their first call. Nothing falls back to the host. +- **The provider refuses the operation.** `Files provider does not support + temporary-directory` is the one such refusal (§6.11). +- **The provider broke its contract** — stale authority, a failed rollback, a + handler that threw, or result data that does not validate. All report + `Files provider invariant failed`, and which contract broke is structural + data for a consumer deciding what to fence rather than text: the category is + never interpolated into the message. + +All three **end the execution** (§6.11's rules for a durability failure apply +here too): no printed error, no `` output, no root or child `Close`, and +no later sibling runs. A missing provider is an installation fault, and a +document that carried on after one would run every step after the file work as +though the file work had happened. + +Precedence among fatal failures is by kind rather than position: a durability +failure first, then a Files infrastructure failure, then a documentation +failure, wherever each sits in the cause graph. + +Cancellation is none of these. Halting resumes a generator rather than throwing, +so no Result is manufactured and no printed error is created. A host provider's +cleanup still runs; a cleanup that fails while cancellation is unwinding is +reported as a sanitized teardown invariant rather than turning the cancellation +into a write outcome. #### When cleanup fails @@ -4894,19 +5042,31 @@ to remove. #### Threat model -Containment is judged against the filesystem as `` observes it. That is -sound while the filesystem is stable, and every guarantee above is stated on -that basis. +Containment is the installed provider's claim, and the two providers make +different ones. -It is not a sandbox. Nothing prevents another process from replacing a -directory with a symlink between the moment a path is validated and the moment -it is used. Resolving a write's destination immediately before writing narrows -that window and closes it for the case a document controls — its own children — -but check-then-use does not become atomic by being ordered more carefully. +**`xmd run`** resolves document paths in the caller's own filesystem, and +judges containment against that filesystem as it observes it. That is sound +**while the host pathname namespace is stable**, and every guarantee above is +stated on that basis. -Containment that does not depend on observed filesystem state — directory -handles, `openat`-style resolution, or platform-enforced sandboxing — is issue -#227. +It is not a sandbox. Nothing prevents another process from replacing a +directory, symlink, junction, or reparse point between the moment a path is +observed and the moment it is used. Resolving a write's destination immediately +before writing narrows that window and closes it for the case a document +controls — its own children — but check-then-use does not become atomic by being +ordered more carefully, and no capability the shipped runtimes expose closes it +without a native dependency. + +**A workflow run** resolves document paths in a logical filesystem the run +owns. A document path never becomes a host path there, so there is no host +namespace for another process to replace: lookup, symlink resolution, and +traversal are all the provider's own, and a symlink target that looks like a +host absolute path is an ordinary logical name. + +Neither claim covers a native command a document runs. A subprocess receives +the contextual working directory and the caller's filesystem, and containing +what it then does is not this component's boundary. #### Scope @@ -4968,9 +5128,22 @@ which is not an order at all. Finding nothing is a result. An empty array succeeds and the document carries on; it is not a failure and not a printed error. +#### Who searches + +`` validates the shape of what the document wrote — a pattern that is +empty, absolute, or begins by leaving cannot match anything a search produces — +and then makes exactly one `API.Files` call (§1.2). The provider compiles the +patterns, walks the tree, and returns the deduplicated, sorted, POSIX-relative +files. No directory path, no partial listing, and no host path crosses back. + +An absolute pattern is judged by the pattern's own grammar rather than the +running platform's: patterns match POSIX-relative paths everywhere, so a +leading `/` is absolute wherever the document runs, and so is a drive-letter +prefix. Deciding it from the host would make one document mean two things. + #### The pattern dialect -Patterns are the filesystem glob library's own, and `` adds no syntax: +Patterns are the provider's dialect, and `` adds no syntax: - `*` matches within one path segment; - `**` crosses segments, and `**/` matches no directories as readily as many — @@ -5013,10 +5186,9 @@ and a link to a directory is not descended into. That last rule is what keeps a search inside `Env.cwd` without judging any destination: traversal only ever follows real directories, so it cannot leave the -working directory and cannot cycle. The filesystem library exposes symlink -following, but nothing in it confines a resolved destination to the root or -detects a traversal cycle, so following one cannot be offered safely. A later -implementation may, if the library guarantees both. +working directory and cannot cycle. Following one cannot be offered safely +without confining a resolved destination to the root and detecting a traversal +cycle, and no provider guarantees both today. A later one may. #### Failures @@ -5039,9 +5211,9 @@ files". Only a whole leading `..` segment leaves: `..notes.md` is an ordinary name, and a `..` further along — `docs/../*.md` — is a path a search never produces, so it matches nothing for the ordinary reason. -A pattern that cannot be compiled arrives as a `RegExp` error about a translated -expression the author never wrote, and which pattern it was is not recoverable -from it. The candidates are listed rather than one being named; they are the +A pattern that cannot be compiled is reported by the provider as an +`invalid-pattern` failure, and which pattern it was does not survive that +boundary. The candidates are listed rather than one being named; they are the document's own text. #### Printed errors @@ -5051,20 +5223,24 @@ traversal failure names **no path at all**: what failed is a directory under `Env.cwd` that the document never wrote, and §1.2 keeps absolute paths out of printed errors. -As in §6.13, nothing from a caught platform error is reproduced. The errno code -**selects** a phrase from the fixed allowlist the filesystem components share, -and an unrecognized code selects `the filesystem operation failed`. The code -itself is never emitted, and the error's class carries no authority either — a -`GlobError` arriving from a wrapped call is replaced like any other, because a -class says nothing about whether a message is safe to show. +As in §6.13, nothing from a caught platform error is reproduced. The provider +returns a reason from the shared vocabulary, the reason **selects** a phrase, +and an unrecognized one selects `the filesystem operation failed`. The error's +class carries no authority either — a `GlobError` arriving from a provider call +is replaced like any other, because a class says nothing about whether a +message is safe to show. + +A provider that is absent or that broke its contract is not a search failure at +all. It ends the execution on §6.13's terms, and `` binds nothing. #### Threat model As with ``, the guarantee is about traversal rather than about the -filesystem being stable. `` never follows a symlink, so nothing it reads -is chosen by one; but a directory that is real when it is read could be replaced -afterwards, and this is not a sandbox. Containment that does not depend on -observed filesystem state is issue #227. +filesystem being stable. No provider follows a symlink, so nothing a search +reads is chosen by one. Under `xmd run` a directory that is real when it is read +could still be replaced afterwards: the host claim holds while the host pathname +namespace is stable, and this is not a sandbox. A workflow run's traversal walks +logical entries that no other process can replace. #### Scope @@ -6443,6 +6619,8 @@ visible warning blocks, gather into a separate error report). | TD14 | Ordinary failures are unchanged | A failing block inside a `` still renders a printed error and the following sibling still runs | | TD15 | Cancelled acquisition | Cancelling while the directory is live, and before the acquiring task runs, both leave nothing behind | | TD16 | Replayed component import | A nested component's journaled import is the other effect a `` can consume; it fails the execution the same way | +| TD18 | Provider-backed acquisition | Creation, the canonical path, and removal are one provider acquisition; with no provider installed the component fails the execution before rendering anything | +| TD19 | A refused operation | A provider that denies `temporary-directory` fails the execution with the fixed diagnostic, renders no content, and lets no later sibling run | | TD17 | Colocated document | `xmd test packages/core/src/components/TempDir.test.md` narrates the lifetime — ordinary cwd, live directory inside, removed and restored after, a captured directory live for a sibling, and the bare form's path — with no search path and no JavaScript | ### Tier PC — `` and `` @@ -6502,6 +6680,12 @@ visible warning blocks, gather into a separate error report). | FL24 | Regular file as a path component | `parent/child.txt` with `parent` a file fails for both forms without naming the resolved path | | FL25 | The working directory itself | `.` and a path normalizing to it are contained, and fail as a directory rather than as an escape | | FL26 | Adversarial error shapes | A `code` holding an absolute path, markup and a newline, an inherited key (`toString`), a planted path in both message and code, and an externally thrown `FileAccessError` all produce the generic phrase; nothing planted reaches the document and the printed error stays one line | +| FL28 | The check authorizes nothing | A provider whose `checkFilePath` refuses expands no children and receives no second call; the sibling after the component still runs | +| FL29 | The write repeats admission | The semantic write is one call that re-admits the authored path and owns resolution, target, parents, and the commit — proven by FL18's child swapping the parent between the two | +| FL30 | Every write phase | Admission, resolution, target, parents, temporary, commit, cleanup, and a rolled-back transaction each produce their own outcome sentence, and no other combination is constructable | +| FL31 | Provider absence | With no provider installed the write fails the execution before its children, writes nothing, renders nothing, reaches no low-level `API.Fs` call, and stops the sibling after it | +| FL32 | Malformed provider data | A write failure or success whose data does not validate is a provider-contract failure that ends the execution; a malformed non-write failure is the generic printed error and the document carries on | +| FL33 | A handler that throws | An arbitrary throw becomes a fixed `protocol` invariant with no cause, message, errno text, or host value; an existing durability or Files failure beneath it is rethrown by identity instead | | FL27 | Colocated document | `xmd test packages/core/src/components/File.test.md` covers both forms, `as` capture, nested parents, replacement, exact content for both authoring shapes, a leading-dots name, and isolation between temporary directories — with no search path and no JavaScript | ### Tier FA — Fatal error discovery @@ -6524,6 +6708,57 @@ visible warning blocks, gather into a separate error report). | FA17 | No resurrection | A `DocumentationError` a component recovered from is not reported as the outward failure, while the same one reached without crossing a content failure still is | | FA18 | Precedence behind a content failure | A durability failure beneath a recovered content failure outranks a documentation failure, in either wrapper order | | FA19 | Cycles through a content failure | A self-caused content failure and one whose cause points back at the wrapper holding it both terminate, and the durability failure is still found | +| FA20 | Every Files infrastructure failure | Provider-unavailable, operation-denied, and each invariant category is discovered as fatal, bare and wrapped, and none is a durability failure | +| FA21 | Discovery through every wrapper | A Files failure is found inside a teardown aggregate, an `AggregateError`, an ordinary `cause`, and all three at once | +| FA22 | Files outranks documentation | In either aggregate order and either teardown order, for every kind | +| FA23 | Durability outranks Files | In either order, for every durability kind | +| FA24 | All three at once | Every ordering of a durability, a Files, and a documentation failure reports the durability one; a graph with the last two reports the Files one; nesting changes neither | +| FA25 | A content failure hides no Files failure | Found beneath a `ContentError` set by subclass and by assignment, and preferred over a documentation failure the boundary would otherwise stop at | +| FA26 | Cycles carrying a Files failure | A cyclic teardown graph and a self-caused content failure both terminate and still find it | +| FA27 | A separately loaded runtime copy | A failure with no shared class identity is recognized by its structural tag; an object carrying a different tag is not | +| FA28 | Decided by output | Only an output-mode `DocumentationError` is; a `throw` decision, every durability kind, and every Files kind are not | + +### Tier HF — The host Files provider + +Driven directly rather than through a document: what these assert is the +contract a component cannot see. Every row runs on all five release targets +(`filesystem-contract` in CI), because path arithmetic and `realpath` are the +platform's. + +| # | Test | Verify | +|---|------|--------| +| HF1 | The check touches nothing | Empty, absolute, and lexically escaping paths are each refused with their own reason and no filesystem call at all; an admissible one answers `Ok(undefined)` | +| HF2 | The round trip | A write commits, reports `host-committed`, reads back, and leaves no temporary beside the file | +| HF3 | The search's shape | Sorted, deduplicated, POSIX-relative regular files; a symbolic link is not a result | +| HF4 | Platform failures are Results | `realpath`, `stat`, and the read each fail with the right phase and reason, and neither the message, the code, nor the workspace path survives | +| HF5 | Every write phase | Each of the eight phases produces data whose target claim matches it, and nothing planted survives | +| HF6 | Escapes | A link out is refused at resolution for both forms, the destination is not named, and the outside file is unchanged | +| HF7 | Internal links | Followed to the file they name; the link stays a link | +| HF8 | Dangling links | Replaced rather than followed, and nothing is created where the link pointed | +| HF9 | The observer contract | The private phases are announced in order, once each, for write, read, and search | +| HF10, HF10b | Where the guarantee stops | A parent replaced synchronously between resolution and use is written through, and a target replaced between resolution and access is read through — the documented weakness, with atomicity still holding | +| HF11 | The commit is one event | A fault before and after `next()` report the same unknown outcome, and one of the two runs really did commit | +| HF12 | Cancellation | No Result is produced and no temporary is left | +| HF13 | Temporary directories | Live and die with the acquiring scope; a halt before acquisition leaves nothing | +| HF14 | Absence | Every operation throws provider-unavailable with the fixed diagnostic and no cause, and no low-level call is made | +| HF15 | Installation | `useHostFiles()` installs beneath ordinary middleware, which can still wrap it | +| HF16 | Directory links on every platform | A junction on Windows and a directory symlink elsewhere are refused on the same terms | +| HF17 | The compiled artifact | `scripts/files-contract-probe.ts`, compiled and run on each of the five targets, asserts the contract's observable claims and prints every one it checked | + +### Tier FF — Files infrastructure failure + +| # | Test | Verify | +|---|------|--------| +| FF1 | Absence before children | A write with no provider fails the execution, expands no children, renders nothing, reaches no `API.Fs` call, and stops the following sibling | +| FF2 | Every other form | Read, ``, and `` each stop at their first provider call, and nothing after them expands | +| FF3 | A refused operation | Denial is fatal, carries its own fixed diagnostic, and renders no content | +| FF4 | The check authorizes nothing | A refused check makes exactly one provider call; the children never run and the document carries on | +| FF5 | Malformed write data | A phase and target that contradict each other, a reason outside the vocabulary, and an undescribable success are all fatal `protocol` invariants, and the category is not interpolated | +| FF6 | Malformed non-write data | Becomes the generic printed error, leaks nothing, and the document carries on | +| FF7 | An arbitrary throw | Replaced by a fixed invariant with no cause and no host value | +| FF8 | Identity is preserved | A nested durability failure and a nested Files failure are each rethrown as the same object | +| FF9 | Precedence at the wrapper | A durability failure beneath a Files invariant is the one preserved | +| FF10 | Ordinary failures | A missing file is still a printed error and the sibling still runs | ### Tier OM — The `output` error mode From 81a26c470c60dcfd3a5e41b121fd9d664027109b Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sat, 8 Aug 2026 23:46:40 -0400 Subject: [PATCH 2/8] =?UTF-8?q?=F0=9F=90=9B=20Keep=20core's=20error=20modu?= =?UTF-8?q?le=20out=20of=20the=20runtime's=20host=20graph?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `errors.ts` is in the graph a separately loaded copy of `printErrors` bundles, and importing the runtime's package root pulled the host Apis in with it — including a native addon no bundler can inline. The Files recognizer needs none of that, so it comes from the leaf module through a new `./files` subpath. The five-target job prepares with `deno install` rather than `deno task deps`: the task caches graphs this job does not need and reaches them by spawning a child, which does not survive the Windows runner's path handling. Its compile now carries the repository's isolation flags, and the rule that enforces them reads every compile in a workflow rather than letting the first invocation's flags answer for the rest. --- .github/workflows/ci.yml | 18 ++++++++++++++++-- packages/core/src/errors.ts | 9 +++++++-- packages/runtime/deno.json | 1 + packages/runtime/host-files.ts | 2 +- packages/runtime/package.json | 1 + scripts/files-contract-probe.ts | 6 ++++-- .../tests/publish-workflow-membership.test.ts | 19 +++++++++++++------ tsconfig.node.json | 3 +++ 8 files changed, 46 insertions(+), 13 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3ca07ce3..d79df1b2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -306,8 +306,22 @@ jobs: with: bun-version: 1.3.14 + # `deno install` rather than `deno task deps`: the task also caches the + # graphs a browser build and a release compile walk, and it reaches them + # by spawning a child — which does not survive the Windows runner's path + # handling. Nothing here builds the bundle or compiles the CLI, so the + # plain frozen install is the whole preparation this job needs. - name: Install dependencies - run: deno task deps + run: deno install --frozen + + # The compile below runs under `--node-modules-dir=none`, which resolves + # npm packages from the Deno cache rather than from `node_modules`. This + # caches the probe's graph in that mode without touching the layout the + # step above just created. + - name: Cache the probe's graph for a compile + run: > + deno install --entrypoint --node-modules-dir=none --frozen + scripts/files-contract-probe.ts # Deno first, and the compile with it: `pnpm install` and `bun install` # each rewrite `node_modules` into their own layout, so a Deno step after @@ -318,7 +332,7 @@ jobs: - name: Host contract as a compiled binary run: | set -eu - deno compile --allow-all --frozen \ + deno compile --node-modules-dir=none --cached-only --frozen --allow-all \ --output dist/files-contract-probe scripts/files-contract-probe.ts if [ -f dist/files-contract-probe.exe ]; then ./dist/files-contract-probe.exe diff --git a/packages/core/src/errors.ts b/packages/core/src/errors.ts index 15807028..488ba8aa 100644 --- a/packages/core/src/errors.ts +++ b/packages/core/src/errors.ts @@ -7,8 +7,13 @@ import { StaleInputError, TerminalDivergenceError, } from "@executablemd/durable-streams"; -import { asFilesFatal } from "@executablemd/runtime"; -import type { FilesFatalFailure } from "@executablemd/runtime"; +// The leaf module rather than the package root. This module is in the graph a +// separately loaded copy of `printErrors` bundles, and the runtime's root +// re-exports the host Apis — process, fetch, filesystem — one of which carries a +// native addon no bundler can inline. Recognizing a Files failure needs none of +// that: it is a structural tag and a parser. +import { asFilesFatal } from "@executablemd/runtime/files"; +import type { FilesFatalFailure } from "@executablemd/runtime/files"; import { InvocationTeardownError } from "./invocation.ts"; import type { ErrorSegment } from "./types.ts"; diff --git a/packages/runtime/deno.json b/packages/runtime/deno.json index d1adb361..555d3941 100644 --- a/packages/runtime/deno.json +++ b/packages/runtime/deno.json @@ -3,6 +3,7 @@ "version": "0.8.0", "exports": { ".": "./mod.ts", + "./files": "./files.ts", "./test": "./test/mod.ts" } } diff --git a/packages/runtime/host-files.ts b/packages/runtime/host-files.ts index 45ada6be..61b70a6a 100644 --- a/packages/runtime/host-files.ts +++ b/packages/runtime/host-files.ts @@ -423,7 +423,7 @@ export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { try { const matched = yield* traverse(input, observe); const files = matched.filter((entry) => entry.isFile).map((entry) => entry.path); - return Ok([...new Set(files)].sort(byCodePoint)); + return Ok([...new Set(files)].toSorted(byCodePoint)); } catch (error) { // The Api compiles patterns as it starts, so an unusable one — an // unterminated character class — arrives as a `SyntaxError` from `RegExp` diff --git a/packages/runtime/package.json b/packages/runtime/package.json index 371e4e4b..5282df66 100644 --- a/packages/runtime/package.json +++ b/packages/runtime/package.json @@ -5,6 +5,7 @@ "type": "module", "exports": { ".": "./mod.ts", + "./files": "./files.ts", "./test": "./test/mod.ts" }, "dependencies": { diff --git a/scripts/files-contract-probe.ts b/scripts/files-contract-probe.ts index 9ef6b667..8039e250 100644 --- a/scripts/files-contract-probe.ts +++ b/scripts/files-contract-probe.ts @@ -160,14 +160,16 @@ await main(function* () { }); check("a temporary directory is removed with its scope", !exists(temporary)); + // Diagnostics rather than a result, so they go to stderr: this probe's whole + // output is the account of what it checked. for (const claim of checked) { - console.log(`${failures.includes(claim) ? "FAIL" : "ok "} ${claim}`); + console.error(`${failures.includes(claim) ? "FAIL" : "ok "} ${claim}`); } if (failures.length > 0) { console.error(`files contract: ${failures.length} of ${checked.length} claims failed`); yield* exit(1); } - console.log( + console.error( `files contract: ${checked.length} claims hold on ${process.platform}/${process.arch}`, ); }); diff --git a/scripts/tests/publish-workflow-membership.test.ts b/scripts/tests/publish-workflow-membership.test.ts index 14bafc37..c306be80 100644 --- a/scripts/tests/publish-workflow-membership.test.ts +++ b/scripts/tests/publish-workflow-membership.test.ts @@ -94,13 +94,20 @@ describe("release.yml binary compilation", () => { if (!commands.includes("deno compile")) { continue; } + // Every compile in the file, not only the first: a workflow may compile + // more than one entrypoint, and reading to a single named one would let + // the first invocation's flags answer for all of them. + // // A folded shell command: read from `deno compile` to the entrypoint that - // ends it, so line breaks and continuations do not matter. - const invocation = commands.slice(commands.indexOf("deno compile")); - const flags = invocation.slice(0, invocation.indexOf("packages/cli/src/compiled.ts")); - for (const flag of ["--node-modules-dir=none", "--cached-only", "--frozen"]) { - if (!flags.includes(flag)) { - compiles.push(`${entry} compiles without ${flag}`); + // ends it — the first `.ts` path after the flags — so line breaks and + // continuations do not matter. + for (const invocation of commands.split("deno compile").slice(1)) { + const entrypoint = invocation.search(/\S+\.ts\b/); + const flags = entrypoint === -1 ? invocation : invocation.slice(0, entrypoint); + for (const flag of ["--node-modules-dir=none", "--cached-only", "--frozen"]) { + if (!flags.includes(flag)) { + compiles.push(`${entry} compiles without ${flag}`); + } } } } diff --git a/tsconfig.node.json b/tsconfig.node.json index ff774562..7fbaf1af 100644 --- a/tsconfig.node.json +++ b/tsconfig.node.json @@ -34,6 +34,9 @@ "@executablemd/runtime": [ "./packages/runtime/mod.ts" ], + "@executablemd/runtime/files": [ + "./packages/runtime/files.ts" + ], "@executablemd/runtime/test": [ "./packages/runtime/test/mod.ts" ], From 36b662bd6c4223ddaf4d5bd61cd93ba5f605ab8b Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sat, 8 Aug 2026 23:48:26 -0400 Subject: [PATCH 3/8] =?UTF-8?q?=F0=9F=90=9B=20Sort=20the=20search=20result?= =?UTF-8?q?s=20with=20the=20lib=20the=20Node=20typecheck=20targets?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `toSorted` needs es2023, which `tsconfig.node.json` does not select. The array is built from a Set on the line above, so nothing shared is being mutated. --- packages/runtime/host-files.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/runtime/host-files.ts b/packages/runtime/host-files.ts index 61b70a6a..45ada6be 100644 --- a/packages/runtime/host-files.ts +++ b/packages/runtime/host-files.ts @@ -423,7 +423,7 @@ export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { try { const matched = yield* traverse(input, observe); const files = matched.filter((entry) => entry.isFile).map((entry) => entry.path); - return Ok([...new Set(files)].toSorted(byCodePoint)); + return Ok([...new Set(files)].sort(byCodePoint)); } catch (error) { // The Api compiles patterns as it starts, so an unusable one — an // unterminated character class — arrives as a `SyntaxError` from `RegExp` From 8e21506c05344f400929ec85214a1a069173ce68 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 9 Aug 2026 00:02:42 -0400 Subject: [PATCH 4/8] =?UTF-8?q?=F0=9F=94=92=20Make=20Files=20recognition?= =?UTF-8?q?=20total=20and=20strict,=20and=20keep=20cancellation=20cleanup?= =?UTF-8?q?=20fatal?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Structural parsing runs from `fatalCause`, which every generic catch in expansion consults. A provider is free to hand back a Proxy that refuses to be inspected, and one of these parsers throwing would replace the failure being classified with a failure about classifying it. Every read is now total, and so is the cause traversal: a hostile wrapper narrows what discovery finds instead. Recognizing a Files fatal is a decision to let that exact object travel onward by identity, so it now requires the whole public contract — frozen data with no extra fields, the fixed diagnostic for its kind, and no cause. A candidate carrying a raw message or an errno chain is replaced by a fresh invariant rather than preserved. Durability recognition is unchanged: it stays #394's class-based mechanism, and only the Files boundary is crossed by a second loaded copy. A host cleanup that fails while cancellation is unwinding has no outcome to report beside it. It leaves the scope as a fixed teardown invariant instead of manufacturing a write result, carrying neither the platform's error nor the generated temporary's name. The loaded-copy claim is now proved by a real second copy: the Files module is bundled, imported as its own module, and its failures are recognized in both directions. --- architecture.md | 20 +- packages/core/src/errors.ts | 57 +++- packages/core/tests/files-fatal.test.ts | 220 +++++++++++++++- packages/core/tests/loaded-copy-files.test.ts | 146 +++++++++++ packages/runtime/files.ts | 244 ++++++++++++------ packages/runtime/host-files.ts | 16 +- packages/runtime/tests/host-files.test.ts | 109 +++++++- scripts/runtime-test-exclusions.ts | 6 + specs/executable-mdx-spec.md | 10 +- 9 files changed, 734 insertions(+), 94 deletions(-) create mode 100644 packages/core/tests/loaded-copy-files.test.ts diff --git a/architecture.md b/architecture.md index 6f2dbba2..8d921091 100644 --- a/architecture.md +++ b/architecture.md @@ -719,9 +719,23 @@ Both are discovered through one cycle-safe traversal of the whole cause graph, and precedence is decided by kind rather than by position: a durability failure first, then a Files infrastructure failure, then a documentation failure. The selected failure comes back by identity, because a fail-stop that records "the -first error" has to record the one that happened. Recognition is structural — -a stable tag on frozen data, never `instanceof` — so a failure a separately -loaded copy constructed is found on the same terms as one this copy did. +first error" has to record the one that happened. + +The two kinds are *recognized* by different mechanisms, and the difference is +deliberate. A durability failure is recognized exactly as it always has been — +by class, against the failure types durable-streams exports — and that is +unchanged. A **Files** infrastructure failure is recognized structurally: a +stable tag on frozen data, the fixed diagnostic for its kind, and no cause, +never `instanceof`. It has to be, because the Files boundary is the one a +separately loaded copy of a package reaches across, where `instanceof` answers +false. Recognizing one is also a decision to let that exact object travel +onward, so a candidate carrying anything else — a raw message, a cause chain, +an extra field — fails the contract and is replaced by a fresh invariant rather +than preserved. + +The traversal reads values it did not create, so every read in it is total: a +thrown Proxy or a failing accessor narrows what discovery finds instead of +replacing the failure being classified. ## The document filesystem boundary diff --git a/packages/core/src/errors.ts b/packages/core/src/errors.ts index 488ba8aa..f6463b89 100644 --- a/packages/core/src/errors.ts +++ b/packages/core/src/errors.ts @@ -387,14 +387,14 @@ function walkCauses( opaque: OpaqueFailure | undefined, seen: Set, ): T | undefined { - const selected = select(error); + const selected = attempt(() => select(error)); if (selected !== undefined) { return selected; } if (typeof error !== "object" || error === null || seen.has(error)) { return undefined; } - if (opaque !== undefined && opaque(error)) { + if (opaque !== undefined && attempt(() => opaque(error)) === true) { return undefined; } seen.add(error); @@ -407,16 +407,49 @@ function walkCauses( return undefined; } -/** The wrapper contracts a failure can aggregate other failures through. */ +/** + * The wrapper contracts a failure can aggregate other failures through. + * + * Every read here is of a value the engine did not create. A thrown object may + * be a Proxy, or carry an accessor that fails, and one of those refusing to + * answer must not become the failure this traversal was called to classify — + * every generic catch in expansion asks `fatalCause` first, so a throw here + * would replace the real failure with a failure about inspecting it. An + * unreadable wrapper simply aggregates nothing. + */ function causesOf(error: object): unknown[] { - if (error instanceof InvocationTeardownError) { - return error.causes; - } - if (error instanceof AggregateError) { - return error.errors; - } - if (error instanceof Error && error.cause !== undefined) { - return [error.cause]; + return ( + attempt(() => { + if (error instanceof InvocationTeardownError) { + return members(error.causes); + } + if (error instanceof AggregateError) { + return members(error.errors); + } + if (error instanceof Error && error.cause !== undefined) { + return [error.cause]; + } + return []; + }) ?? [] + ); +} + +/** A wrapper's members, when it really holds a list of them. */ +function members(value: unknown): unknown[] { + return Array.isArray(value) ? [...value] : []; +} + +/** + * Read a value that may refuse to be read. + * + * `undefined` means "this said nothing", which every caller here already treats + * as "not recognized" — so a hostile shape narrows what discovery finds rather + * than replacing what it was discovering. + */ +function attempt(read: () => T): T | undefined { + try { + return read(); + } catch { + return undefined; } - return []; } diff --git a/packages/core/tests/files-fatal.test.ts b/packages/core/tests/files-fatal.test.ts index 093153ea..f194bb2d 100644 --- a/packages/core/tests/files-fatal.test.ts +++ b/packages/core/tests/files-fatal.test.ts @@ -17,22 +17,28 @@ import { describe, it, beforeAll } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { ensure, Err, Ok, resource, scoped, until } from "effection"; +import { ensure, Err, Ok, resource, scoped, spawn, suspend, until, withResolvers } from "effection"; import type { Operation, Result } from "effection"; -import { exists, rm, writeTextFile } from "@effectionx/fs"; +import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; import { API, FILES_ERROR, + FILES_FATAL, FILES_WRITE_SUCCESS, Files, FilesError, FilesInvariantError, FilesOperationDeniedError, + hostFilesHandler, + parseFileWriteFailure, + parseFileWriteSuccess, + parseFilesFailure, parseFilesFatal, useHostFiles, } from "@executablemd/runtime"; import type { FilePathInput, FileWriteInput, FileWriteSuccess } from "@executablemd/runtime"; import { InMemoryStream, StaleInputError } from "@executablemd/durable-streams"; +import { InvocationTeardownError } from "../src/invocation.ts"; import { execute } from "../src/execute.ts"; import { collect } from "../src/collect.ts"; import { useTempFileCompiler } from "../src/temp-file-compiler.ts"; @@ -482,6 +488,216 @@ describe("Tier FF — Files infrastructure failure", () => { expect(filesFatalFailure(thrown)).toBeUndefined(); }); + // FF11: none of this parsing may throw. Every value it reads is a value a + // provider handed back, and a provider is as free to return a Proxy that + // refuses to be inspected as a plain record. These parsers run from + // `fatalCause`, which every generic catch in expansion consults — so one of + // them throwing would replace the failure being classified with a failure + // about classifying it, exactly when the engine is deciding whether the + // execution may continue. + it("FF11: hostile shapes are not recognized, and never throw", function* () { + const explode = () => { + throw new Error("EACCES: refused, at '/planted/secret.txt'"); + }; + + const hostile: Array<{ name: string; build: () => unknown }> = [ + { + name: "a throwing data accessor", + build: () => + Object.defineProperty(new Error("wrapper"), "data", { get: explode, configurable: true }), + }, + { + name: "data whose fields throw", + build: () => + Object.assign(new Error("wrapper"), { + data: new Proxy({}, { get: explode }), + }), + }, + { + name: "data whose key enumeration throws", + build: () => + Object.assign(new Error("wrapper"), { + data: new Proxy( + { type: FILES_FATAL, kind: "provider-unavailable" }, + { ownKeys: explode }, + ), + }), + }, + { + name: "an unreadable prototype", + build: () => new Proxy(new Error("wrapper"), { getPrototypeOf: explode }), + }, + { + name: "a throwing cause", + build: () => + Object.defineProperty(new Error("wrapper"), "cause", { + get: explode, + configurable: true, + }), + }, + { + name: "unreadable aggregate members", + build: () => + Object.defineProperty(new AggregateError([], "wrapper"), "errors", { + get: explode, + configurable: true, + }), + }, + { + name: "aggregate members that are not a list", + build: () => Object.assign(new AggregateError([], "wrapper"), { errors: 7 }), + }, + { + name: "unreadable teardown causes", + build: () => + Object.defineProperty(new InvocationTeardownError([]), "causes", { + get: explode, + configurable: true, + }), + }, + ]; + + for (const shape of hostile) { + const candidate = shape.build(); + // Every parser answers, and none of them recognizes the shape. + expect(parseFilesFatal(candidate)).toBeUndefined(); + expect(parseFilesFailure(candidate)).toBeUndefined(); + expect(parseFileWriteFailure(candidate)).toBeUndefined(); + expect(parseFileWriteSuccess(candidate)).toBeUndefined(); + expect(fatalCause(candidate)).toBeUndefined(); + expect(filesFatalFailure(candidate)).toBeUndefined(); + + // And a real failure underneath one is still found, so totality narrows + // what a hostile wrapper hides rather than what discovery reaches. + const planted = new FilesInvariantError("authority"); + expect(fatalCause(new AggregateError([candidate, planted], "mixed"))).toBe(planted); + } + }); + + // FF12: a candidate that carries the right tag but breaks the rest of the + // contract is not preserved. Recognition is a decision to let that exact + // object travel onward by identity, so an Error carrying a raw platform + // message and a cause chain would carry both past the boundary the reason + // vocabulary exists to hold. + it("FF12: a correctly tagged but unsafe failure is replaced, not preserved", function* () { + const dir = yield* useFixture(); + + const unsafe: Array<{ name: string; build: () => Error }> = [ + { + name: "a raw message", + build: () => + Object.assign(new Error("EACCES: denied, at '/planted/secret.txt'"), { + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + }, + { + name: "a cause chain", + build: () => + Object.assign( + new Error("Files provider is not installed", { + cause: new Error("ENOENT at /planted/secret.txt"), + }), + { data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }) }, + ), + }, + { + name: "mutable data", + build: () => + Object.assign(new Error("Files provider is not installed"), { + data: { type: FILES_FATAL, kind: "provider-unavailable" }, + }), + }, + { + name: "an extra data field", + build: () => + Object.assign(new Error("Files provider is not installed"), { + data: Object.freeze({ + type: FILES_FATAL, + kind: "provider-unavailable", + path: "/planted/secret.txt", + }), + }), + }, + ]; + + for (const shape of unsafe) { + const candidate = shape.build(); + expect(filesFatalFailure(candidate)).toBeUndefined(); + + let thrown: unknown; + yield* scoped(function* () { + yield* useFiles({ + // deno-lint-ignore require-yield + *readTextFile(): Operation> { + throw candidate; + }, + }); + try { + yield* invokeFiles(Files.operations.readTextFile({ cwd: dir, path: "notes.md" })); + } catch (error) { + thrown = error; + } + }); + + // Replaced rather than preserved, and nothing it carried came along. + expect(thrown).not.toBe(candidate); + expect(thrown).toBeInstanceOf(FilesInvariantError); + expect(parseFilesFatal(thrown)?.kind).toBe("invariant"); + expect(thrown instanceof Error ? thrown.message : "").toBe("Files provider invariant failed"); + expect(thrown instanceof Error ? thrown.cause : "unset").toBeUndefined(); + expect(JSON.stringify(thrown instanceof Error ? { ...thrown } : {})).not.toContain("planted"); + } + }); + + // FF13: a host cleanup that fails while cancellation is unwinding leaves the + // scope as an infrastructure failure, and the engine's own discovery is what + // has to find it — that is what makes it consumable by a coordinator deciding + // what to fence. + it("FF13: a cleanup failure during cancellation is discovered as fatal", function* () { + const dir = yield* useFixture(); + yield* writeTextFile(join(dir, "notes.md"), "first"); + const suspended = withResolvers(); + const files = hostFilesHandler(); + + const write = yield* spawn(function* () { + yield* scoped(function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + yield* next(path, content); + suspended.resolve(); + yield* suspend(); + }, + // deno-lint-ignore require-yield + *remove() { + throw Object.assign(new Error("EPERM: denied, at '/planted/secret.txt'"), { + code: "EPERM", + }); + }, + }); + yield* files.writeTextFile({ cwd: dir, path: "notes.md", content: "second" }); + }); + }); + + yield* suspended.operation; + let thrown: unknown; + try { + yield* write.halt(); + } catch (error) { + thrown = error; + } + + const selected = fatalCause(thrown); + expect(parseFilesFatal(selected)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "teardown", + }); + // Preserved by identity, which is what a shared fail-stop records. + expect(filesFatalFailure(thrown)).toBe(selected); + expect(String(selected)).not.toContain("planted"); + expect(yield* readTextFile(join(dir, "notes.md"))).toBe("first"); + }); + // FF10: an ordinary failure is still an ordinary failure. The fatal rule is // for a provider that is missing or wrong, not for everything that goes // wrong beneath one. diff --git a/packages/core/tests/loaded-copy-files.test.ts b/packages/core/tests/loaded-copy-files.test.ts new file mode 100644 index 00000000..90b5c960 --- /dev/null +++ b/packages/core/tests/loaded-copy-files.test.ts @@ -0,0 +1,146 @@ +/** + * Tier LC — a Files failure built by a genuinely separate copy of the runtime. + * + * `FA27` builds a correctly tagged object by hand, which proves the parser + * accepts the shape. It does not prove the case the shape exists for: two + * copies of `@executablemd/runtime` evaluated in one process, where a + * repository component reached its own dependency beside the engine's. In that + * world `instanceof` answers false, every class identity differs, and the tag + * is the only thing the two copies share. + * + * So this bundles `packages/runtime/files.ts` the way an installed dependency + * would arrive, imports the bundle as its own module, constructs a failure with + * *that* copy's constructor, and asks the engine's own discovery about it. + * + * Bundling is also the second half of a claim the smoke job makes: this module + * has to stay reachable without the runtime's host Apis, one of which carries a + * native addon no bundler can inline. A regression there fails here as a + * bundling error rather than as a mysterious CI break. + * + * Deno-only: `deno bundle` is Deno's. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, resource, until } from "effection"; +import type { Operation } from "effection"; +import { exec } from "@effectionx/process"; +import { rm } from "@effectionx/fs"; +import { + FilesInvariantError, + FilesProviderUnavailableError, + parseFilesFatal, +} from "@executablemd/runtime"; +import { InvocationTeardownError } from "../src/invocation.ts"; +import { DocumentationError, fatalCause, filesFatalFailure } from "../src/errors.ts"; +import { mkdtemp } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import process from "node:process"; + +const FILES_MODULE = fileURLToPath(new URL("../../runtime/files.ts", import.meta.url)); +const REPOSITORY = fileURLToPath(new URL("../../../", import.meta.url)); + +/** What a separately loaded copy of the module exposes to this test. */ +interface LoadedCopy { + FilesProviderUnavailableError: new () => Error; + FilesInvariantError: new (category: string) => Error; + parseFilesFatal: (error: unknown) => unknown; +} + +function isLoadedCopy(value: unknown): value is LoadedCopy { + if (typeof value !== "object" || value === null) { + return false; + } + const module: Record = Object.fromEntries(Object.entries(value)); + return ( + typeof module.FilesProviderUnavailableError === "function" && + typeof module.FilesInvariantError === "function" && + typeof module.parseFilesFatal === "function" + ); +} + +/** + * `packages/runtime/files.ts`, bundled and evaluated as its own module. + * + * The bundle is what makes the copy separate: importing the source path again + * would resolve to the module this test already holds, and share every class + * with it. + */ +function useSeparateCopy(): Operation { + return resource(function* (provide) { + const directory = yield* until(mkdtemp(join(tmpdir(), "lc-files-"))); + yield* ensure(() => rm(directory, { recursive: true, force: true })); + const bundle = join(directory, "files.js"); + + // `process.execPath` under Deno is the deno binary, so the driver stays + // typed against node:process rather than a runtime global. + const built = yield* exec(process.execPath, { + arguments: [ + "bundle", + "--frozen", + "--node-modules-dir=none", + FILES_MODULE, + "--output", + bundle, + ], + cwd: REPOSITORY, + }).join(); + if (built.code !== 0) { + throw new Error( + `could not bundle the runtime's Files module:\n${built.stdout}${built.stderr}`, + ); + } + + const loaded: unknown = yield* until(import(`file://${bundle}`)); + if (!isLoadedCopy(loaded)) { + throw new Error("the bundled copy does not expose the Files failure surface"); + } + yield* provide(loaded); + }); +} + +describe("Tier LC — a separately loaded runtime copy", () => { + it("LC1: a failure built by another copy is discovered as fatal, by identity", function* () { + const copy = yield* useSeparateCopy(); + const foreign = new copy.FilesProviderUnavailableError(); + + // The premise, stated as a fact about this object rather than assumed: + // nothing about it shares a class with the copy core imported. + expect(foreign instanceof FilesProviderUnavailableError).toBe(false); + expect(foreign.constructor).not.toBe(FilesProviderUnavailableError); + + // Recognized anyway, and returned as the object that was thrown. + expect(filesFatalFailure(foreign)).toBe(foreign); + expect(parseFilesFatal(foreign)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "provider-unavailable", + }); + + // Through every wrapper the engine builds, and ahead of a documentation + // failure sitting beside it. + const documentation = new DocumentationError({ type: "error", message: "wrong" }, "throw"); + expect(fatalCause(new InvocationTeardownError([documentation, foreign]))).toBe(foreign); + expect(fatalCause(new AggregateError([foreign, documentation], "mixed"))).toBe(foreign); + expect(fatalCause(new Error("wrapper", { cause: foreign }))).toBe(foreign); + }); + + it("LC2: recognition crosses in both directions", function* () { + const copy = yield* useSeparateCopy(); + + // This copy's failure, recognized by the other one's parser. + expect(copy.parseFilesFatal(new FilesInvariantError("savepoint"))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "savepoint", + }); + + // And the other's invariant, recognized here, category and all. + expect(parseFilesFatal(new copy.FilesInvariantError("authority"))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "authority", + }); + }); +}); diff --git a/packages/runtime/files.ts b/packages/runtime/files.ts index 32197123..8bd0ccda 100644 --- a/packages/runtime/files.ts +++ b/packages/runtime/files.ts @@ -329,18 +329,67 @@ export interface FilesFatalFailure extends Error { readonly data: FilesFatalData; } +/** + * Everything below reads values it did not create. + * + * A thrown value is whatever a provider threw, and a provider is as free to + * hand back a Proxy with a throwing `get` trap, an accessor that fails, or an + * object whose key enumeration explodes as it is to hand back a plain record. + * These parsers run from `fatalCause`, which every generic catch in the engine + * consults — so one of them throwing would replace the failure being classified + * with a failure *about classifying it*, at the exact moment the engine is + * deciding whether the execution may continue. + * + * So reading is total: every access that can fail goes through a helper that + * answers `undefined` instead, and each exported parser is additionally wrapped + * so that a shape nobody anticipated is simply not recognized. + */ +function attempt(read: () => T): T | undefined { + try { + return read(); + } catch { + return undefined; + } +} + function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null && !Array.isArray(value); } +/** Whether this is an Error, without trusting its prototype chain. */ +function isError(value: unknown): value is Error { + return attempt(() => value instanceof Error) === true; +} + +/** One property, read through a trap that may refuse or fail. */ +function property(target: object, name: string): unknown { + return attempt(() => Reflect.get(target, name)); +} + +/** How many own enumerable keys, when the object will say. */ +function keyCount(value: object): number | undefined { + return attempt(() => Object.keys(value).length); +} + +function isFrozen(value: object): boolean { + return attempt(() => Object.isFrozen(value)) === true; +} + function dataOf(error: unknown): Record | undefined { - if (!(error instanceof Error) || !("data" in error)) { + if (!isError(error)) { return undefined; } - const { data } = error; + const data = property(error, "data"); return isRecord(data) ? data : undefined; } +/** The one diagnostic each kind of infrastructure failure carries. */ +const FATAL_DIAGNOSTICS: ReadonlyMap = new Map([ + ["provider-unavailable", FILES_PROVIDER_UNAVAILABLE_MESSAGE], + ["operation-denied", FILES_OPERATION_DENIED_MESSAGE], + ["invariant", FILES_INVARIANT_MESSAGE], +]); + function reasonOf(value: unknown): FilesReason | undefined { return REASONS.find((reason) => reason === value); } @@ -364,42 +413,72 @@ function deniableOperation(value: unknown): FilesDeniableOperation | undefined { /** * The infrastructure failure data this Error carries, if it carries valid data. * - * Every field is checked, and the member count with them: an object with extra - * keys is not the shape this contract describes, and accepting it would let a - * provider smuggle a path or a message through under a recognized tag. + * Every field is checked, the member count with them, and that the object is + * frozen: extra keys are not the shape this contract describes, and a mutable + * one is not the shape a constructor here produces. Accepting either would let + * a provider smuggle a path or a message through under a recognized tag. */ export function parseFilesFatal(error: unknown): FilesFatalData | undefined { - const data = dataOf(error); - if (data === undefined || data.type !== FILES_FATAL) { - return undefined; - } - const members = Object.keys(data).length; - if (data.kind === "provider-unavailable" && members === 2) { - return Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }); - } - if (data.kind === "operation-denied" && members === 3) { - const operation = deniableOperation(data.operation); - if (operation !== undefined) { - return Object.freeze({ type: FILES_FATAL, kind: "operation-denied", operation }); + return attempt(() => { + const data = dataOf(error); + if (data === undefined || property(data, "type") !== FILES_FATAL || !isFrozen(data)) { + return undefined; } - } - if (data.kind === "invariant" && members === 3) { - const category = invariantCategory(data.category); - if (category !== undefined) { - return Object.freeze({ type: FILES_FATAL, kind: "invariant", category }); + const members = keyCount(data); + const kind = property(data, "kind"); + if (kind === "provider-unavailable" && members === 2) { + return Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }); } - } - return undefined; + if (kind === "operation-denied" && members === 3) { + const operation = deniableOperation(property(data, "operation")); + if (operation !== undefined) { + return Object.freeze({ type: FILES_FATAL, kind: "operation-denied", operation }); + } + } + if (kind === "invariant" && members === 3) { + const category = invariantCategory(property(data, "category")); + if (category !== undefined) { + return Object.freeze({ type: FILES_FATAL, kind: "invariant", category }); + } + } + return undefined; + }); } /** - * Whether this failure is a Files infrastructure failure. + * Whether this failure satisfies the whole public infrastructure-failure + * contract, not merely the tag. + * + * Recognition decides two different things at once, and the second is why this + * is stricter than `parseFilesFatal`. A recognized failure is **rethrown by + * identity** — the object that was thrown is the object a fail-stop records — + * so recognizing one is a decision to let that exact object travel onward. An + * Error that carries the right data but also a raw platform message, or a cause + * chain holding an errno and a path, would then carry all of that past the + * boundary the reason vocabulary exists to hold. * - * Structural, so a failure constructed by a separately loaded copy of this - * package is recognized on the same terms as one constructed here. + * So the diagnostic must be the fixed one for its kind, and there must be no + * cause. Anything else is a candidate that fails the contract: `invokeFiles` + * replaces it with a fresh invariant rather than preserving it. + * + * Structural throughout, so a failure constructed by a separately loaded copy + * of this package is recognized on exactly the same terms as one constructed + * here — `instanceof` answers false across two copies, which is the case this + * has to survive. */ export function isFilesFatal(error: unknown): error is FilesFatalFailure { - return parseFilesFatal(error) !== undefined; + return ( + attempt(() => { + const data = parseFilesFatal(error); + if (data === undefined || !isError(error)) { + return false; + } + if (property(error, "message") !== FATAL_DIAGNOSTICS.get(data.kind)) { + return false; + } + return property(error, "cause") === undefined; + }) === true + ); } /** @@ -539,17 +618,19 @@ export function fileWriteFailure(input: { * simply declines to recognize it. */ export function parseFilesFailure(error: unknown): FilesFailureData | undefined { - const data = dataOf(error); - if (data === undefined || data.type !== FILES_ERROR || Object.keys(data).length !== 4) { - return undefined; - } - const operation = operationOf(data.operation); - const phase = phaseOf(data.phase); - const reason = reasonOf(data.reason); - if (operation === undefined || phase === undefined || reason === undefined) { - return undefined; - } - return Object.freeze({ type: FILES_ERROR, operation, phase, reason }); + return attempt(() => { + const data = dataOf(error); + if (data === undefined || property(data, "type") !== FILES_ERROR || keyCount(data) !== 4) { + return undefined; + } + const operation = operationOf(property(data, "operation")); + const phase = phaseOf(property(data, "phase")); + const reason = reasonOf(property(data, "reason")); + if (operation === undefined || phase === undefined || reason === undefined) { + return undefined; + } + return Object.freeze({ type: FILES_ERROR, operation, phase, reason }); + }); } /** @@ -561,35 +642,43 @@ export function parseFilesFailure(error: unknown): FilesFailureData | undefined * rather than inventing a commit state. */ export function parseFileWriteFailure(error: unknown): FileWriteFailureData | undefined { - const data = dataOf(error); - if (data === undefined || data.type !== FILES_ERROR || data.operation !== "write") { - return undefined; - } - const found = writePhaseOf(data.phase); - if (found === undefined) { - return undefined; - } - const [phase, rule] = found; - if (data.target !== rule.target) { - return undefined; - } + return attempt(() => { + const data = dataOf(error); + if ( + data === undefined || + property(data, "type") !== FILES_ERROR || + property(data, "operation") !== "write" + ) { + return undefined; + } + const found = writePhaseOf(property(data, "phase")); + if (found === undefined) { + return undefined; + } + const [phase, rule] = found; + if (property(data, "target") !== rule.target) { + return undefined; + } - const reason = data.reason === undefined ? undefined : reasonOf(data.reason); - const cleanup = data.cleanup === undefined ? undefined : reasonOf(data.cleanup); - if ( - (data.reason !== undefined && reason === undefined) || - (data.cleanup !== undefined && cleanup === undefined) || - violatesRule(rule, reason, cleanup) - ) { - return undefined; - } + const declared = property(data, "reason"); + const declaredCleanup = property(data, "cleanup"); + const reason = declared === undefined ? undefined : reasonOf(declared); + const cleanup = declaredCleanup === undefined ? undefined : reasonOf(declaredCleanup); + if ( + (declared !== undefined && reason === undefined) || + (declaredCleanup !== undefined && cleanup === undefined) || + violatesRule(rule, reason, cleanup) + ) { + return undefined; + } - const members = 4 + (reason === undefined ? 0 : 1) + (cleanup === undefined ? 0 : 1); - if (Object.keys(data).length !== members) { - return undefined; - } + const members = 4 + (reason === undefined ? 0 : 1) + (cleanup === undefined ? 0 : 1); + if (keyCount(data) !== members) { + return undefined; + } - return writeData(phase, rule, reason, cleanup); + return writeData(phase, rule, reason, cleanup); + }); } /** A successful write's outcome. */ @@ -609,16 +698,23 @@ export function fileWriteSuccess(publication: FileWriteSuccess["publication"]): * `undefined` here as a protocol invariant too. */ export function parseFileWriteSuccess(value: unknown): FileWriteSuccess | undefined { - if (!isRecord(value) || value.type !== FILES_WRITE_SUCCESS || Object.keys(value).length !== 2) { + return attempt(() => { + if ( + !isRecord(value) || + property(value, "type") !== FILES_WRITE_SUCCESS || + keyCount(value) !== 2 + ) { + return undefined; + } + const publication = property(value, "publication"); + if (publication === "host-committed") { + return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "host-committed" }); + } + if (publication === "transaction-staged") { + return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "transaction-staged" }); + } return undefined; - } - if (value.publication === "host-committed") { - return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "host-committed" }); - } - if (value.publication === "transaction-staged") { - return Object.freeze({ type: FILES_WRITE_SUCCESS, publication: "transaction-staged" }); - } - return undefined; + }); } /** diff --git a/packages/runtime/host-files.ts b/packages/runtime/host-files.ts index 45ada6be..492c8bbc 100644 --- a/packages/runtime/host-files.ts +++ b/packages/runtime/host-files.ts @@ -367,6 +367,10 @@ export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { // about the target: everything before the rename leaves the previous file // in place, and a rename that threw may have run or not. let step: FileWritePhase = "temporary"; + // Whether the write reached its own end. Cleanup runs on every exit, and + // the two exits need different answers: one has a Result to compose with + // and the other does not. + let settled = false; yield* scoped(function* () { const temporary = `${target.path}.xmd-${randomUUID().slice(0, 8)}.tmp`; @@ -375,7 +379,16 @@ export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { try { yield* API.Fs.operations.remove(temporary, { force: true }); } catch (error) { - cleanup = reasonOf(error); + if (settled) { + cleanup = reasonOf(error); + return; + } + // Cancellation is unwinding, so there is no outcome to report this + // beside — and manufacturing one would turn a halt into a write + // result. It leaves the scope as an infrastructure failure instead, + // carrying neither the platform's error nor the generated temporary's + // name, and the engine's fatal discovery finds it there. + throw new FilesInvariantError("teardown"); } }); try { @@ -387,6 +400,7 @@ export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { } catch (error) { failed = reasonOf(error); } + settled = true; }); if (failed !== undefined) { diff --git a/packages/runtime/tests/host-files.test.ts b/packages/runtime/tests/host-files.test.ts index 9d0ab7a6..050c6cde 100644 --- a/packages/runtime/tests/host-files.test.ts +++ b/packages/runtime/tests/host-files.test.ts @@ -20,7 +20,17 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { ensure, race, resource, scoped, sleep, spawn, suspend, until } from "effection"; +import { + ensure, + race, + resource, + scoped, + sleep, + spawn, + suspend, + until, + withResolvers, +} from "effection"; import type { Operation, Result } from "effection"; import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; import { lstat, mkdir, mkdtemp, readdir, realpath, symlink } from "node:fs/promises"; @@ -105,6 +115,37 @@ function planted(code: string): Error { return Object.assign(new Error(`${code}: operation failed, at '${PLANTED}'`), { code }); } +/** + * The first Files infrastructure failure in a thrown graph. + * + * A destructor failure arrives wrapped in whatever Effection collected it with, + * so a test that only inspected the top of the graph would assert about the + * wrapper. Core owns the real traversal; this is the small local one a runtime + * test can have without importing the engine. + */ +function firstFilesFatal(error: unknown, seen = new Set()): unknown { + if (parseFilesFatal(error) !== undefined) { + return error; + } + if (typeof error !== "object" || error === null || seen.has(error)) { + return undefined; + } + seen.add(error); + const causes = + error instanceof AggregateError + ? error.errors + : error instanceof Error && error.cause !== undefined + ? [error.cause] + : []; + for (const cause of causes) { + const found = firstFilesFatal(cause, seen); + if (found !== undefined) { + return found; + } + } + return undefined; +} + /** Every field a failure carries, as one string, for a leak assertion. */ function inspected(error: unknown): string { return JSON.stringify({ @@ -623,6 +664,72 @@ describe("Tier HF — host Files provider", () => { expect(yield* entries(fixture.workspace)).toEqual(["notes.md"]); }); + // HF12b: cleanup that fails *while cancellation is unwinding* is the one exit + // with no outcome to report beside it. Manufacturing one would turn a halt + // into a write result, and forwarding the platform's error would name a + // temporary the document never chose — so a fixed teardown invariant leaves + // the scope instead, and the engine's fatal discovery finds it there. + // + // Deterministic rather than raced: the write suspends after the temporary is + // created, the test waits to observe that it did, and only then halts. + it("HF12b: a cleanup failure during cancellation escapes as a teardown invariant", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "first"); + const suspended = withResolvers(); + let settled: unknown = "not settled"; + + const write = yield* spawn(function* () { + yield* scoped(function* () { + yield* API.Fs.around({ + *writeTextFile([path, content], next) { + yield* next(path, content); + suspended.resolve(); + yield* suspend(); + }, + // deno-lint-ignore require-yield + *remove() { + throw planted("EPERM"); + }, + }); + settled = yield* handler().writeTextFile({ + cwd: fixture.workspace, + path: "notes.md", + content: "second", + }); + }); + }); + + yield* suspended.operation; + let thrown: unknown; + try { + yield* write.halt(); + } catch (error) { + thrown = error; + } + + // No Result was produced: the halt is not a write outcome. + expect(settled).toBe("not settled"); + // And the failure that did leave is the fixed infrastructure one. + const teardown = firstFilesFatal(thrown); + expect(parseFilesFatal(teardown)).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "teardown", + }); + expect(teardown instanceof Error ? teardown.message : "").toBe( + "Files provider invariant failed", + ); + const text = inspected(teardown); + expect(text).not.toContain(PLANTED); + expect(text).not.toContain(".tmp"); + expect(text).not.toContain("EPERM"); + // The category is structural control data for a consumer deciding what to + // fence — it belongs in `data`, and never in the message. + expect(teardown instanceof Error ? teardown.message : "").not.toContain("teardown"); + expect(teardown instanceof Error ? teardown.cause : "unset").toBeUndefined(); + expect(yield* readTextFile(join(fixture.workspace, "notes.md"))).toBe("first"); + }); + // HF13: the temporary directory is a resource, so its lifetime is the // acquiring scope's and a halt cannot land between creating it and owning its // removal. diff --git a/scripts/runtime-test-exclusions.ts b/scripts/runtime-test-exclusions.ts index 6f2d2333..9a63b7fd 100644 --- a/scripts/runtime-test-exclusions.ts +++ b/scripts/runtime-test-exclusions.ts @@ -114,6 +114,12 @@ const DENO_ONLY_TOOLING: RuntimeExclusion[] = [ "exercises Deno-private node:sqlite transaction identities and real SQLite savepoint failure behavior; node:sqlite remains behind --experimental-sqlite on Node 22", issue: "https://github.com/taras/executable.md/issues/365", }, + { + path: "packages/core/tests/loaded-copy-files.test.ts", + reason: + "builds the second copy of the runtime's Files module with `deno bundle`, which is Deno's; the structural recognition it proves is runtime-neutral and is also covered by fatal-cause.test.ts under all three", + issue: DERIVED_SCOPE, + }, ]; /** diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 8ea228ca..76825938 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4596,7 +4596,15 @@ can act on. Each is discovered through the same cycle-safe traversal, each crosses a `ContentError` the way a durability failure does, and each is recognized by its structural tag rather than by class — so a failure a separately loaded copy of the runtime package constructed is found on the same -terms as one this copy did. +terms as one this copy did. Durability failures keep their own recognition +unchanged; only Files failures are recognized structurally, because only that +boundary is crossed by a second loaded copy. + +Recognition is strict, because a recognized failure travels onward as the exact +object that was thrown. It must carry the fixed diagnostic for its kind, frozen +data with no extra fields, and no cause. A failure that carries the right tag +and anything else — a raw platform message, an errno beneath it — is not +preserved: it is replaced by a fresh invariant carrying none of it. **Precedence is decided by kind rather than by position.** A wrapper carries whatever failed together, in whatever order the platform happened to collect From 1c0faba06982006d29f07196ebad42a199cfdaab Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 9 Aug 2026 00:32:21 -0400 Subject: [PATCH 5/8] =?UTF-8?q?=F0=9F=94=92=20Rebuild=20every=20Files=20ou?= =?UTF-8?q?tcome=20from=20validated=20parts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Recognizing a Files fatal hands that exact object onward, so the contract now covers the whole Error: the fixed name and diagnostic for its kind, frozen data with exactly the kind's fields, no cause, and no other enumerable member — string or symbol. A path riding on `name`, on an extra property, or under a symbol key fails the contract, and `invokeFiles` replaces the candidate rather than preserving it. A `Result` is only conventionally a Result. The TypeScript signature is a claim about the provider, not a guarantee, so a component that read `ok`, `value`, or `error` first would be the thing that ran a hostile accessor — outside anything that sanitizes. The core wrappers now inspect the container totally and rebuild every outcome from validated parts: no provider-originated container, error object, or payload reaches ``, `` or ``, and a search result is copied rather than passed along. A container that will not say how it settled, and a success it cannot describe, are provider-contract failures. A malformed non-write failure is not: the vocabulary already has a sentence for it, so a fresh generic failure is substituted and the document carries on. FA24 now runs all six orderings of the three fatal kinds, through both wrappers. --- packages/core/src/components/File.ts | 5 +- packages/core/src/files.ts | 186 ++++++++++++++++--- packages/core/tests/fatal-cause.test.ts | 12 +- packages/core/tests/files-fatal.test.ts | 230 ++++++++++++++++++++++++ packages/runtime/files.ts | 60 ++++++- 5 files changed, 461 insertions(+), 32 deletions(-) diff --git a/packages/core/src/components/File.ts b/packages/core/src/components/File.ts index b0e7cb83..20568ff7 100644 --- a/packages/core/src/components/File.ts +++ b/packages/core/src/components/File.ts @@ -66,7 +66,7 @@ import type { FilesFailureData, FileWriteFailureData, FilesReason } from "@execu import { content } from "../component-api.ts"; import { hasContent } from "../content-context.ts"; import { ContentError } from "../errors.ts"; -import { checkFilePath, readFileText, writeFileText, writeReport } from "../files.ts"; +import { checkFilePath, readFileText, writeFileText } from "../files.ts"; import type { Json } from "../types.ts"; import { reason } from "./fs-error-phrases.ts"; @@ -100,8 +100,7 @@ export default printErrors(function* (props: Record): Operation(call: Operation): Operation { } } -export function checkFilePath(input: FilePathInput): Operation> { - return invokeFiles(Files.operations.checkFilePath(input)); +/** + * Read a member of a value the engine did not create. + * + * A `Result` is only conventionally a `Result`: what a provider actually + * returned is an arbitrary runtime value, and reading `ok` on a Proxy that + * refuses can throw. The type says otherwise, which is exactly why the check + * belongs here — the signature is a claim about the provider, not a guarantee. + */ +function property(target: unknown, name: string): unknown { + if (typeof target !== "object" || target === null) { + return undefined; + } + try { + return Reflect.get(target, name); + } catch { + return undefined; + } } -export function readFileText(input: FilePathInput): Operation> { - return invokeFiles(Files.operations.readTextFile(input)); +/** + * Whether this really is a settled `Result`, and which way it settled. + * + * `undefined` means it is neither, which is a provider-contract failure rather + * than an outcome: a value that will not say whether it succeeded cannot be + * reported as either. + */ +function settlement(result: unknown): boolean | undefined { + const ok = property(result, "ok"); + return typeof ok === "boolean" ? ok : undefined; +} + +/** + * The generic failure a non-write operation falls back to. + * + * Built fresh from this operation's own identity rather than from anything the + * provider returned, so the sentence a document reads is the one the vocabulary + * already has for "the filesystem operation failed" — and nothing the provider + * put in its place travels with it. + */ +function generic(operation: FilesOperation, phase: FilesPhase): FilesError { + return filesFailure({ operation, phase, reason: "operation-failed" }); +} + +/** + * One non-write operation, with its whole outcome rebuilt from validated parts. + * + * Nothing a provider returned reaches a component: not the container, not the + * error object, not the payload. A success is re-checked against the operation's + * own payload contract and a failure is re-constructed from parsed data, so by + * the time `` or `` reads `result.error` it is reading an object + * this module made. + * + * A malformed *failure* is not fatal — the vocabulary already has a sentence for + * an operation that failed for an unrecognized reason, and nothing about a + * target is at stake — so it becomes the generic one and the document carries + * on. A malformed *success* is fatal: a provider claiming an outcome it cannot + * describe has not established the outcome. + */ +function* outcome( + call: Operation, + contract: { + operation: FilesOperation; + phase: FilesPhase; + payload: (value: unknown) => { readonly value: T } | undefined; + }, +): Operation> { + const result = yield* invokeFiles(call); + const settled = settlement(result); + if (settled === undefined) { + throw new FilesInvariantError("protocol"); + } + if (settled) { + const payload = contract.payload(property(result, "value")); + if (payload === undefined) { + throw new FilesInvariantError("protocol"); + } + return Ok(payload.value); + } + const data = parseFilesFailure(property(result, "error")); + if (data === undefined || data.operation !== contract.operation) { + return Err(generic(contract.operation, contract.phase)); + } + return Err(filesFailure({ operation: data.operation, phase: data.phase, reason: data.reason })); +} + +function nothing(value: unknown): { readonly value: void } | undefined { + return value === undefined ? { value: undefined } : undefined; } -export function writeFileText(input: FileWriteInput): Operation> { - return invokeFiles(Files.operations.writeTextFile(input)); +function text(value: unknown): { readonly value: string } | undefined { + return typeof value === "string" ? { value } : undefined; +} + +/** + * A search result, copied out of whatever the provider handed back. + * + * The array itself is rebuilt rather than passed along: a provider could return + * something array-like whose elements are accessors, or one it goes on mutating + * after the fact, and a document binds this value. + */ +function paths(value: unknown): { readonly value: string[] } | undefined { + if (!Array.isArray(value)) { + return undefined; + } + const copied: string[] = []; + for (const entry of value) { + if (typeof entry !== "string") { + return undefined; + } + copied.push(entry); + } + return { value: copied }; +} + +export function checkFilePath(input: FilePathInput): Operation> { + return outcome(Files.operations.checkFilePath(input), { + operation: "check-file-path", + phase: "lexical", + payload: nothing, + }); +} + +export function readFileText(input: FilePathInput): Operation> { + return outcome(Files.operations.readTextFile(input), { + operation: "read", + phase: "access", + payload: text, + }); } export function globFiles(input: GlobInput): Operation> { - return invokeFiles(Files.operations.globFiles(input)); + return outcome(Files.operations.globFiles(input), { + operation: "glob", + phase: "traversal", + payload: paths, + }); } export function temporaryDirectory(): Operation> { - return invokeFiles(Files.operations.temporaryDirectory()); + return outcome(Files.operations.temporaryDirectory(), { + operation: "temporary-directory", + phase: "acquire", + payload: text, + }); } /** - * What a write reported, or a fatal failure if it reported nothing readable. + * What a write reported, or nothing when it succeeded. * - * A write is the one operation whose result makes a claim about the world: the - * file was replaced, or it was not, or nobody can tell. Data that does not - * validate leaves no safe sentence to print — every candidate asserts one of - * those three — so a provider that cannot describe what it did is treated as - * one that may not have done it. `undefined` is not a possible return. + * A write is the one operation whose outcome makes a claim about the world: the + * file was replaced, or it was not, or nobody can tell. Every sentence a + * consumer could print asserts one of those, so an outcome that does not + * validate leaves none of them available — and a provider that cannot describe + * what it did is treated as one that may not have done it. Both a malformed + * success and a malformed failure are therefore fatal, and so is a container + * that will not say which it is. + * + * The data that comes back is rebuilt from validated parts, like every other + * outcome here. */ -export function writeReport(result: Result): FileWriteFailureData | undefined { - if (result.ok) { - if (parseFileWriteSuccess(result.value) === undefined) { +export function* writeFileText(input: FileWriteInput): Operation { + const result = yield* invokeFiles(Files.operations.writeTextFile(input)); + const settled = settlement(result); + if (settled === undefined) { + throw new FilesInvariantError("protocol"); + } + if (settled) { + if (parseFileWriteSuccess(property(result, "value")) === undefined) { throw new FilesInvariantError("protocol"); } return undefined; } - const failure = parseFileWriteFailure(result.error); - if (failure === undefined) { + const data = parseFileWriteFailure(property(result, "error")); + if (data === undefined) { throw new FilesInvariantError("protocol"); } - return failure; + return parseFileWriteFailure( + fileWriteFailure({ phase: data.phase, reason: data.reason, cleanup: data.cleanup }), + ); } diff --git a/packages/core/tests/fatal-cause.test.ts b/packages/core/tests/fatal-cause.test.ts index 5a7aefd7..54cce018 100644 --- a/packages/core/tests/fatal-cause.test.ts +++ b/packages/core/tests/fatal-cause.test.ts @@ -425,14 +425,20 @@ describe("Tier FA — Fatal error discovery", () => { const files = new FilesInvariantError("savepoint"); const doc = documentation(); - // Every order of the three, and the answer never moves. + // All six orders of the three. Position is exactly what must not decide + // this, so a sample of the orderings would leave the claim partly untested — + // and the one ordering left out is the one that would ship the bug. for (const members of [ [durability, files, doc], - [doc, files, durability], - [files, doc, durability], [durability, doc, files], + [files, durability, doc], + [files, doc, durability], + [doc, durability, files], + [doc, files, durability], ]) { expect(fatalCause(new AggregateError(members, "mixed"))).toBe(durability); + // The other wrapper the engine builds, with the same members. + expect(fatalCause(new InvocationTeardownError(members))).toBe(durability); } expect(fatalCause(new AggregateError([doc, files], "mixed"))).toBe(files); diff --git a/packages/core/tests/files-fatal.test.ts b/packages/core/tests/files-fatal.test.ts index f194bb2d..2ea56a49 100644 --- a/packages/core/tests/files-fatal.test.ts +++ b/packages/core/tests/files-fatal.test.ts @@ -29,6 +29,7 @@ import { FilesError, FilesInvariantError, FilesOperationDeniedError, + FilesProviderUnavailableError, hostFilesHandler, parseFileWriteFailure, parseFileWriteSuccess, @@ -618,6 +619,69 @@ describe("Tier FF — Files infrastructure failure", () => { }), }), }, + { + name: "a path-bearing name", + build: () => + Object.assign(new Error("Files provider is not installed"), { + name: `FilesProviderUnavailableError: /planted/secret.txt`, + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + }, + { + name: "an extra Error-level path property", + build: () => + Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + path: "/planted/secret.txt", + }), + }, + { + name: "an enumerable symbol payload", + build: () => { + const error = Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }); + // Enumerable, so it survives a spread and an `Object.assign` copy — + // the two ways a consumer would carry a failure onward. + Object.defineProperty(error, Symbol.for("planted"), { + value: "/planted/secret.txt", + enumerable: true, + }); + return error; + }, + }, + { + name: "hostile key enumeration", + build: () => + new Proxy( + Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + { + ownKeys() { + throw new Error("refused, at '/planted/secret.txt'"); + }, + }, + ), + }, + { + name: "hostile descriptor access", + build: () => + new Proxy( + Object.assign(new Error("Files provider is not installed"), { + name: "FilesProviderUnavailableError", + data: Object.freeze({ type: FILES_FATAL, kind: "provider-unavailable" }), + }), + { + getOwnPropertyDescriptor() { + throw new Error("refused, at '/planted/secret.txt'"); + }, + }, + ), + }, ]; for (const shape of unsafe) { @@ -645,7 +709,24 @@ describe("Tier FF — Files infrastructure failure", () => { expect(parseFilesFatal(thrown)?.kind).toBe("invariant"); expect(thrown instanceof Error ? thrown.message : "").toBe("Files provider invariant failed"); expect(thrown instanceof Error ? thrown.cause : "unset").toBeUndefined(); + // Neither stringification nor enumerable output retains the planted value. + expect(String(thrown)).not.toContain("planted"); expect(JSON.stringify(thrown instanceof Error ? { ...thrown } : {})).not.toContain("planted"); + expect( + JSON.stringify( + Object.getOwnPropertySymbols(thrown instanceof Error ? thrown : {}).map(String), + ), + ).not.toContain("planted"); + } + + // And the real constructors, which have to stay recognizable: the contract + // above is what they produce, not a stricter shape nothing satisfies. + for (const genuine of [ + new FilesProviderUnavailableError(), + new FilesOperationDeniedError("temporary-directory"), + new FilesInvariantError("teardown"), + ]) { + expect(filesFatalFailure(genuine)).toBe(genuine); } }); @@ -698,6 +779,155 @@ describe("Tier FF — Files infrastructure failure", () => { expect(yield* readTextFile(join(dir, "notes.md"))).toBe("first"); }); + // FF14: the `Result` a provider returns is only conventionally a Result. The + // TypeScript signature is a claim about the provider, not a guarantee, so a + // component that read `ok`, `value`, or `error` first would be the thing that + // ran the hostile accessor — outside anything that sanitizes. Every outcome is + // therefore inspected and rebuilt at the boundary. + it("FF14: a hostile Result never reaches a component", function* () { + const dir = yield* useFixture(); + yield* writeTextFile(join(dir, "notes.md"), "existing"); + const explode = () => { + throw new Error("EACCES: refused, at '/planted/secret.txt'"); + }; + + const hostile: Array<{ name: string; build: () => unknown }> = [ + { + name: "a throwing ok", + build: () => ({ + get ok() { + return explode(); + }, + }), + }, + { name: "a non-boolean ok", build: () => ({ ok: "yes", value: "x" }) }, + { name: "no ok at all", build: () => ({ value: "x" }) }, + { + name: "a throwing value", + build: () => ({ + ok: true, + get value() { + return explode(); + }, + }), + }, + { + name: "a throwing error", + build: () => ({ + ok: false, + get error() { + return explode(); + }, + }), + }, + { name: "a proxied container", build: () => new Proxy({}, { get: explode }) }, + { name: "not an object at all", build: () => "committed" }, + ]; + + // Every form, through every operation a document can reach. + const documents: Array<{ name: string; source: string; method: string }> = [ + { name: "read", source: '\n\nAFTER', method: "readTextFile" }, + { + name: "write", + source: 'content\n\nAFTER', + method: "writeTextFile", + }, + { + name: "check", + source: 'content\n\nAFTER', + method: "checkFilePath", + }, + { + name: "glob", + source: '\n\nAFTER', + method: "globFiles", + }, + { + name: "tempdir", + source: "INSIDE\n\nAFTER", + method: "temporaryDirectory", + }, + ]; + + for (const document of documents) { + for (const shape of hostile) { + const outcome = yield* run(dir, document.source, function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *[document.method]() { + return fromOutside(shape.build()); + }, + }); + }); + + // A container that will not describe its own outcome is a provider + // contract failure, so the execution ends and later work stops. + expect(outcome.ok).toBe(false); + expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ + type: "executablemd.runtime.files-fatal/v1", + kind: "invariant", + category: "protocol", + }); + expect(outcome.output).not.toContain("AFTER"); + expect(outcome.output).not.toContain("INSIDE"); + expect(String(outcome.error)).not.toContain("planted"); + expect(String(outcome.error)).not.toContain("EACCES"); + } + } + }); + + // FF15: the one hostile shape that is *not* fatal. A non-write failure whose + // data does not validate has no claim about a target in it, and the vocabulary + // already has a sentence for an operation that failed for an unrecognized + // reason — so the document reads that and carries on. What it must not read is + // anything the provider put there. + it("FF15: malformed non-write failure data alone stays printable", function* () { + const dir = yield* useFixture(); + + const cases: Array<{ source: string; method: string; expected: string }> = [ + { + source: '\n\nAFTER', + method: "readTextFile", + expected: 'cannot read "notes.md": the filesystem operation failed.', + }, + { + source: '\n\nAFTER', + method: "globFiles", + expected: "cannot search the working directory: the filesystem operation failed.", + }, + { + source: "INSIDE\n\nAFTER", + method: "temporaryDirectory", + expected: "cannot create a temporary directory: the filesystem operation failed.", + }, + ]; + + for (const shape of cases) { + const outcome = yield* run(dir, shape.source, function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *[shape.method]() { + return Err( + Object.assign(new Error("EACCES: denied, at '/planted/secret.txt'"), { + data: { type: FILES_ERROR, operation: "read", phase: "nowhere" }, + path: "/planted/secret.txt", + }), + ); + }, + }); + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain(shape.expected); + expect(outcome.output).not.toContain("/planted/secret.txt"); + expect(outcome.output).not.toContain("EACCES"); + // The document carried on, which is the whole difference from FF14. + expect(outcome.output).toContain("AFTER"); + } + }); + // FF10: an ordinary failure is still an ordinary failure. The fatal rule is // for a provider that is missing or wrong, not for everything that goes // wrong beneath one. diff --git a/packages/runtime/files.ts b/packages/runtime/files.ts index 8bd0ccda..c3fbe1a0 100644 --- a/packages/runtime/files.ts +++ b/packages/runtime/files.ts @@ -390,6 +390,47 @@ const FATAL_DIAGNOSTICS: ReadonlyMap = new Map([ ["invariant", FILES_INVARIANT_MESSAGE], ]); +/** The one class name each kind of infrastructure failure carries. */ +const FATAL_NAMES: ReadonlyMap = new Map([ + ["provider-unavailable", "FilesProviderUnavailableError"], + ["operation-denied", "FilesOperationDeniedError"], + ["invariant", "FilesInvariantError"], +]); + +/** + * Everything a constructor here puts on the Error itself, and nothing else. + * + * `message` and `stack` are non-enumerable own properties of every Error, so + * what remains enumerable is exactly what a constructor assigned. Anything more + * is payload the contract does not describe — and since recognition hands the + * object onward by identity, payload travels with it. + */ +const FATAL_MEMBERS: readonly string[] = ["data", "name"]; + +/** + * Whether the Error carries only the members its constructor assigns. + * + * Symbols are checked as well as string keys: a symbol-keyed enumerable + * property survives spreading and appears in `Object.assign`'d copies, so + * leaving it unexamined would let a path ride along through exactly the + * mechanisms a consumer uses to inspect a failure. + */ +function hasOnlyContractMembers(error: Error): boolean { + const keys = attempt(() => [...Object.keys(error)].sort()); + if (keys === undefined || keys.length !== FATAL_MEMBERS.length) { + return false; + } + if (!keys.every((key, index) => key === FATAL_MEMBERS[index])) { + return false; + } + const payload = attempt(() => + Object.getOwnPropertySymbols(error).filter( + (symbol) => Object.getOwnPropertyDescriptor(error, symbol)?.enumerable === true, + ), + ); + return payload !== undefined && payload.length === 0; +} + function reasonOf(value: unknown): FilesReason | undefined { return REASONS.find((reason) => reason === value); } @@ -457,14 +498,17 @@ export function parseFilesFatal(error: unknown): FilesFatalData | undefined { * chain holding an errno and a path, would then carry all of that past the * boundary the reason vocabulary exists to hold. * - * So the diagnostic must be the fixed one for its kind, and there must be no - * cause. Anything else is a candidate that fails the contract: `invokeFiles` - * replaces it with a fresh invariant rather than preserving it. + * So the whole object has to match what a constructor here produces: the fixed + * name and diagnostic for its kind, frozen structural data with exactly the + * fields the kind describes, no cause, and no other enumerable member — string + * or symbol. Anything else is a candidate that fails the contract, and + * `invokeFiles` replaces it with a fresh invariant rather than preserving it. * * Structural throughout, so a failure constructed by a separately loaded copy * of this package is recognized on exactly the same terms as one constructed * here — `instanceof` answers false across two copies, which is the case this - * has to survive. + * has to survive. That is also why the `name` is checked rather than the class: + * a second copy's constructor is a different function producing the same name. */ export function isFilesFatal(error: unknown): error is FilesFatalFailure { return ( @@ -473,10 +517,16 @@ export function isFilesFatal(error: unknown): error is FilesFatalFailure { if (data === undefined || !isError(error)) { return false; } + if (property(error, "name") !== FATAL_NAMES.get(data.kind)) { + return false; + } if (property(error, "message") !== FATAL_DIAGNOSTICS.get(data.kind)) { return false; } - return property(error, "cause") === undefined; + if (property(error, "cause") !== undefined) { + return false; + } + return hasOnlyContractMembers(error); }) === true ); } From 99162d8c368b45a0669195b7e539874f43a10b72 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 9 Aug 2026 07:08:04 -0400 Subject: [PATCH 6/8] =?UTF-8?q?=F0=9F=94=92=20Tell=20an=20unreadable=20Res?= =?UTF-8?q?ult=20member=20apart=20from=20an=20absent=20one?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `undefined` was standing for three different answers: a member read fine and held undefined, a member was absent, and reading a member threw. Collapsing them let two containers through that never described their outcome — `checkFilePath` accepted a success whose `value` refused to be read, and a non-write failure whose `error` refused was downgraded to the printable generic. Presence and readability are now asked separately, and every operation says which it requires. A search array is copied by index through the same reader, so length and element traps are covered too and the iterator is never consulted. The one place the contract bends is `checkFilePath`: Effection spells a payload-free success as its shared `Unit`, `{ ok: true }` with no `value` member at all, so absence there is the ordinary success rather than a failure. What is refused is a `value` that cannot be read, and one that is present but is something other than undefined. The seam now passes live Proxies around real `Ok`/`Err` values through `API.Files`, so the boundary is the first thing to run a hostile trap — the previous JSON round-trip invoked the getter inside the provider handler and proved only the already-covered handler-throw path. --- packages/core/src/files.ts | 176 +++++++---- packages/core/tests/files-fatal.test.ts | 370 +++++++++++++++++++++--- 2 files changed, 449 insertions(+), 97 deletions(-) diff --git a/packages/core/src/files.ts b/packages/core/src/files.ts index 4048d118..7183df00 100644 --- a/packages/core/src/files.ts +++ b/packages/core/src/files.ts @@ -75,34 +75,57 @@ export function* invokeFiles(call: Operation): Operation { } /** - * Read a member of a value the engine did not create. + * Whether a member is there at all, or `undefined` when asking threw. + * + * Asked separately from reading it, because absence and unreadability are + * different answers and only one of them is ever legitimate. A `has` trap can + * refuse the question, which is why even this is total. + */ +function present(target: unknown, name: string): boolean | undefined { + if (typeof target !== "object" || target === null) { + return undefined; + } + try { + return Reflect.has(target, name); + } catch { + return undefined; + } +} + +/** + * A member of a value the engine did not create, or `undefined` when it is + * absent or reading it threw. * * A `Result` is only conventionally a `Result`: what a provider actually * returned is an arbitrary runtime value, and reading `ok` on a Proxy that * refuses can throw. The type says otherwise, which is exactly why the check * belongs here — the signature is a claim about the provider, not a guarantee. + * + * The value comes back boxed so that a member which really is `undefined` is + * distinguishable from one that could not be read. Collapsing those is how a + * container that never described its outcome gets mistaken for one that + * described an empty one. */ -function property(target: unknown, name: string): unknown { - if (typeof target !== "object" || target === null) { +function read(target: unknown, name: string): { readonly value: unknown } | undefined { + if (present(target, name) !== true) { return undefined; } try { - return Reflect.get(target, name); + return { value: Reflect.get(Object(target), name) }; } catch { return undefined; } } /** - * Whether this really is a settled `Result`, and which way it settled. + * How a `Result` settled, or `undefined` when it will not say. * - * `undefined` means it is neither, which is a provider-contract failure rather - * than an outcome: a value that will not say whether it succeeded cannot be - * reported as either. + * A value that does not describe its own outcome cannot be reported as either + * one, so `undefined` here is a provider-contract failure rather than a result. */ function settlement(result: unknown): boolean | undefined { - const ok = property(result, "ok"); - return typeof ok === "boolean" ? ok : undefined; + const ok = read(result, "ok"); + return typeof ok?.value === "boolean" ? ok.value : undefined; } /** @@ -118,19 +141,34 @@ function generic(operation: FilesOperation, phase: FilesPhase): FilesError { } /** - * One non-write operation, with its whole outcome rebuilt from validated parts. + * The failure half of a non-write outcome, rebuilt from validated parts. * - * Nothing a provider returned reaches a component: not the container, not the - * error object, not the payload. A success is re-checked against the operation's - * own payload contract and a failure is re-constructed from parsed data, so by - * the time `` or `` reads `result.error` it is reading an object - * this module made. + * The two unreadable cases are not the same as the unrecognized one. A + * container whose `error` is absent or refuses to be read never described what + * went wrong, and there is nothing to report — that is a provider-contract + * failure. An `error` that reads fine but carries data the vocabulary does not + * recognize *did* describe a failure, just not one this version knows: the + * generic sentence covers it and the document carries on. + */ +function failure(result: unknown, operation: FilesOperation, phase: FilesPhase): Result { + const reported = read(result, "error"); + if (reported === undefined) { + throw new FilesInvariantError("protocol"); + } + const data = parseFilesFailure(reported.value); + if (data === undefined || data.operation !== operation) { + return Err(generic(operation, phase)); + } + return Err(filesFailure({ operation: data.operation, phase: data.phase, reason: data.reason })); +} + +/** + * One non-write operation whose success carries a payload, with its whole + * outcome rebuilt from validated parts. * - * A malformed *failure* is not fatal — the vocabulary already has a sentence for - * an operation that failed for an unrecognized reason, and nothing about a - * target is at stake — so it becomes the generic one and the document carries - * on. A malformed *success* is fatal: a provider claiming an outcome it cannot - * describe has not established the outcome. + * Nothing a provider returned reaches a component: not the container, not the + * error object, not the payload. By the time `` or `` reads + * `result.error` it is reading an object this module made. */ function* outcome( call: Operation, @@ -145,22 +183,18 @@ function* outcome( if (settled === undefined) { throw new FilesInvariantError("protocol"); } - if (settled) { - const payload = contract.payload(property(result, "value")); - if (payload === undefined) { - throw new FilesInvariantError("protocol"); - } - return Ok(payload.value); + if (!settled) { + return failure(result, contract.operation, contract.phase); } - const data = parseFilesFailure(property(result, "error")); - if (data === undefined || data.operation !== contract.operation) { - return Err(generic(contract.operation, contract.phase)); + const carried = read(result, "value"); + if (carried === undefined) { + throw new FilesInvariantError("protocol"); } - return Err(filesFailure({ operation: data.operation, phase: data.phase, reason: data.reason })); -} - -function nothing(value: unknown): { readonly value: void } | undefined { - return value === undefined ? { value: undefined } : undefined; + const payload = contract.payload(carried.value); + if (payload === undefined) { + throw new FilesInvariantError("protocol"); + } + return Ok(payload.value); } function text(value: unknown): { readonly value: string } | undefined { @@ -170,30 +204,69 @@ function text(value: unknown): { readonly value: string } | undefined { /** * A search result, copied out of whatever the provider handed back. * - * The array itself is rebuilt rather than passed along: a provider could return - * something array-like whose elements are accessors, or one it goes on mutating - * after the fact, and a document binds this value. + * Every step of the copy is provider-controlled: iterator lookup, `length`, and + * each element are all trappable, so the walk is by index through the same + * total reader rather than by `for…of`. `undefined` from any of them is a + * refusal to describe the result, which the caller turns into one fixed + * invariant. + * + * The array itself is rebuilt rather than passed along. A provider could return + * something array-shaped whose elements are accessors, or one it goes on + * mutating afterwards — and a document binds this value. */ function paths(value: unknown): { readonly value: string[] } | undefined { if (!Array.isArray(value)) { return undefined; } + const length = read(value, "length"); + if (typeof length?.value !== "number" || !Number.isInteger(length.value) || length.value < 0) { + return undefined; + } const copied: string[] = []; - for (const entry of value) { - if (typeof entry !== "string") { + for (let index = 0; index < length.value; index++) { + const entry = read(value, String(index)); + if (typeof entry?.value !== "string") { return undefined; } - copied.push(entry); + copied.push(entry.value); } return { value: copied }; } -export function checkFilePath(input: FilePathInput): Operation> { - return outcome(Files.operations.checkFilePath(input), { - operation: "check-file-path", - phase: "lexical", - payload: nothing, - }); +/** + * Whether this authored path is admissible, with nothing usable coming back. + * + * The success carries no payload, and Effection spells that as its shared + * `Unit` — `{ ok: true }`, with no `value` member at all. So an **absent** + * `value` is the ordinary successful answer here, and this is the one operation + * where that is true. What is still refused is a `value` that cannot be read, + * and one that is present but is something other than `undefined`: the first + * means the container never described its outcome, and the second means it + * described one this operation does not have. + */ +export function* checkFilePath(input: FilePathInput): Operation> { + const result = yield* invokeFiles(Files.operations.checkFilePath(input)); + const settled = settlement(result); + if (settled === undefined) { + throw new FilesInvariantError("protocol"); + } + if (!settled) { + return failure(result, "check-file-path", "lexical"); + } + const carried = present(result, "value"); + if (carried === undefined) { + throw new FilesInvariantError("protocol"); + } + if (carried) { + // Present, so it has to be readable *and* be the absent payload. Asking + // only whether the value is `undefined` would let a member that refused to + // be read pass as one that read as nothing. + const value = read(result, "value"); + if (value === undefined || value.value !== undefined) { + throw new FilesInvariantError("protocol"); + } + } + return Ok(undefined); } export function readFileText(input: FilePathInput): Operation> { @@ -241,12 +314,17 @@ export function* writeFileText(input: FileWriteInput): Operation Operation): Oper }); } +function invariantCategory(error: unknown): string | undefined { + const data = parseFilesFatal(fatalCause(error)); + return data?.kind === "invariant" ? data.category : undefined; +} + +/** + * Whether anything a hostile provider planted reached the outside. + * + * The rendered document, the failure's own text, its cause chain, and its + * enumerable data are the places a value could surface. Only the *outcome* is + * inspected — never the hostile object — so nothing here runs a trap the + * boundary was supposed to run first. + */ +function leaked(outcome: Outcome): boolean { + const failure = outcome.error; + const shown = [ + outcome.output, + String(failure), + failure instanceof Error ? String(failure.cause) : "", + JSON.stringify(failure instanceof Error ? { ...failure } : {}), + ].join(" "); + return shown.includes("planted") || shown.includes("EACCES"); +} + /** * Data as it arrives from outside the type system. * @@ -782,99 +807,348 @@ describe("Tier FF — Files infrastructure failure", () => { // FF14: the `Result` a provider returns is only conventionally a Result. The // TypeScript signature is a claim about the provider, not a guarantee, so a // component that read `ok`, `value`, or `error` first would be the thing that - // ran the hostile accessor — outside anything that sanitizes. Every outcome is - // therefore inspected and rebuilt at the boundary. + // ran the hostile accessor — outside anything that sanitizes. + // + // Every shape below is a live Proxy **around a real `Ok`/`Err`**, so it keeps + // the declared type without a cast and, more importantly, its traps have never + // run when the provider hands it back. The first thing to touch them is the + // normalization boundary. Serializing one here instead would invoke the getter + // inside the test and prove only the handler-throw path FF7 already covers. it("FF14: a hostile Result never reaches a component", function* () { const dir = yield* useFixture(); yield* writeTextFile(join(dir, "notes.md"), "existing"); - const explode = () => { - throw new Error("EACCES: refused, at '/planted/secret.txt'"); - }; - const hostile: Array<{ name: string; build: () => unknown }> = [ + /** + * A Result whose named member is there and refuses to be read. + * + * The `has` trap matters as much as the `get` one: a payload-free success is + * Effection's `Unit`, which has no `value` member at all, so a `get` trap + * alone would never be consulted and the case would test nothing. + */ + function throwing(result: Result, member: string): Result { + return new Proxy(result, { + has(target, property) { + return property === member ? true : Reflect.has(target, property); + }, + get(target, property, receiver) { + if (property === member) { + throw new Error("EACCES: refused, at '/planted/secret.txt'"); + } + return Reflect.get(target, property, receiver); + }, + }); + } + + /** A Result whose named member is not there at all. */ + function absent(result: Result, member: string): Result { + return new Proxy(result, { + has(target, property) { + return property === member ? false : Reflect.has(target, property); + }, + get(target, property, receiver) { + return property === member ? undefined : Reflect.get(target, property, receiver); + }, + }); + } + + /** A Result that will not say how it settled. */ + function undecided(result: Result): Result { + return new Proxy(result, { + get(target, property, receiver) { + return property === "ok" ? "yes" : Reflect.get(target, property, receiver); + }, + }); + } + + const ordinary = new Error("EACCES: denied, at '/planted/secret.txt'"); + + /** + * Which settlement each shape actually bites. + * + * A trap on `value` is never consulted by a failure, and one on `error` is + * never consulted by a success — so pairing every shape with every + * settlement would demand a fatal outcome from combinations that + * legitimately take the ordinary path, and the test would measure the wrong + * thing. + */ + const shapes: Array<{ + name: string; + settlements: ReadonlyArray<"success" | "failure">; + skip?: string; + hostile: (result: Result) => Result; + }> = [ { name: "a throwing ok", - build: () => ({ - get ok() { - return explode(); - }, - }), + settlements: ["success", "failure"], + hostile: (result) => throwing(result, "ok"), }, - { name: "a non-boolean ok", build: () => ({ ok: "yes", value: "x" }) }, - { name: "no ok at all", build: () => ({ value: "x" }) }, + { + name: "an absent ok", + settlements: ["success", "failure"], + hostile: (result) => absent(result, "ok"), + }, + { name: "a non-boolean ok", settlements: ["success", "failure"], hostile: undecided }, { name: "a throwing value", - build: () => ({ - ok: true, - get value() { - return explode(); - }, - }), + settlements: ["success"], + hostile: (result) => throwing(result, "value"), + }, + { + name: "an absent value", + settlements: ["success"], + // `checkFilePath` succeeds with no payload, and Effection spells that as + // its shared `Unit` — `{ ok: true }`, with no `value` member at all. + // Absence is the ordinary success there, which FF14c asserts positively. + skip: "checkFilePath", + hostile: (result) => absent(result, "value"), }, { name: "a throwing error", - build: () => ({ - ok: false, - get error() { - return explode(); - }, - }), + settlements: ["failure"], + hostile: (result) => throwing(result, "error"), + }, + { + name: "an absent error", + settlements: ["failure"], + hostile: (result) => absent(result, "error"), }, - { name: "a proxied container", build: () => new Proxy({}, { get: explode }) }, - { name: "not an object at all", build: () => "committed" }, ]; - // Every form, through every operation a document can reach. - const documents: Array<{ name: string; source: string; method: string }> = [ - { name: "read", source: '\n\nAFTER', method: "readTextFile" }, + const operations: Array<{ + name: string; + source: string; + method: string; + ok: () => Result; + }> = [ + { + name: "read", + source: '\n\nAFTER', + method: "readTextFile", + ok: () => Ok("existing"), + }, { name: "write", source: 'content\n\nAFTER', method: "writeTextFile", + ok: () => Ok(fileWriteSuccess("host-committed")), }, { - name: "check", + name: "checkFilePath", source: 'content\n\nAFTER', method: "checkFilePath", + ok: () => Ok(undefined), }, { name: "glob", source: '\n\nAFTER', method: "globFiles", + ok: () => Ok(["notes.md"]), }, { name: "tempdir", source: "INSIDE\n\nAFTER", method: "temporaryDirectory", + ok: () => Ok(dir), }, ]; - for (const document of documents) { - for (const shape of hostile) { - const outcome = yield* run(dir, document.source, function* () { + for (const operation of operations) { + for (const shape of shapes) { + if (shape.skip === operation.name) { + continue; + } + for (const settlement of shape.settlements) { + const settled = settlement === "success" ? operation.ok : () => Err(ordinary); + const outcome = yield* run(dir, operation.source, function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *[operation.method]() { + return shape.hostile(settled()); + }, + }); + }); + + // A container that will not describe its own outcome is a provider + // contract failure: the execution ends and later work stops. + expect(outcome.ok).toBe(false); + expect(invariantCategory(outcome.error)).toBe("protocol"); + expect(outcome.output).not.toContain("AFTER"); + expect(outcome.output).not.toContain("INSIDE"); + // No child of the write ever expanded, either. + expect(yield* exists(join(dir, "out.txt"))).toBe(false); + expect(leaked(outcome)).toBe(false); + } + } + } + }); + + // FF14b: the array a search returns is provider-controlled all the way down — + // its length and every element. The copy walks it by index through the same + // total reader, so a trap that refuses becomes the same fixed invariant rather + // than an exception escaping into expansion. + it("FF14b: a hostile search array becomes a sanitized invariant", function* () { + const dir = yield* useFixture(); + const explode = () => { + throw new Error("EACCES: refused, at '/planted/secret.txt'"); + }; + + const arrays: Array<{ name: string; build: () => string[] }> = [ + { + name: "a throwing element", + build: () => + new Proxy(["notes.md"], { + get(target, property, receiver) { + return property === "0" ? explode() : Reflect.get(target, property, receiver); + }, + }), + }, + { + name: "a throwing length", + build: () => + new Proxy(["notes.md"], { + get(target, property, receiver) { + return property === "length" ? explode() : Reflect.get(target, property, receiver); + }, + }), + }, + { + name: "a refused element", + build: () => + new Proxy(["notes.md"], { + has(target, property) { + return property === "0" ? false : Reflect.has(target, property); + }, + }), + }, + { + name: "an element that is not a string", + build: () => + new Proxy(["notes.md"], { + get(target, property, receiver) { + return property === "0" ? 7 : Reflect.get(target, property, receiver); + }, + }), + }, + ]; + + for (const array of arrays) { + const outcome = yield* run( + dir, + '\n\nAFTER', + function* () { yield* useHostFiles(); yield* Files.around({ // deno-lint-ignore require-yield - *[document.method]() { - return fromOutside(shape.build()); + *globFiles() { + return Ok(array.build()); }, }); + }, + ); + + expect(outcome.ok).toBe(false); + expect(invariantCategory(outcome.error)).toBe("protocol"); + expect(outcome.output).not.toContain("AFTER"); + expect(leaked(outcome)).toBe(false); + } + + // The iterator is never consulted, because the walk is by index. A search + // whose only hostile trap is `Symbol.iterator` therefore copies cleanly — + // which is the derivation this kills: a `for…of` walk would have run it. + const untouched = yield* run( + dir, + '\n\nfound: {found}', + function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *globFiles() { + return Ok( + new Proxy(["b.md", "a.md"], { + get(target, property, receiver) { + return property === Symbol.iterator + ? explode() + : Reflect.get(target, property, receiver); + }, + }), + ); + }, + }); + }, + ); + expect(untouched.ok).toBe(true); + expect(untouched.output).toContain("found: b.md,a.md"); + + // And what comes back is the document's own array: mutating the provider's + // afterwards does not reach the bound value. + const source = ["b.md", "a.md"]; + const copied = yield* run( + dir, + '\n\nfound: {found}', + function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *globFiles() { + return Ok(source); + }, }); + }, + ); + source.push("planted.md"); + expect(copied.ok).toBe(true); + expect(copied.output).toContain("found: b.md,a.md"); + expect(copied.output).not.toContain("planted.md"); + }); + + // FF14c: a legitimate payload-free success is still a success. Effection + // spells one as its shared `Unit` — `{ ok: true }`, with no `value` member — + // so requiring the member to be present would reject the very thing the host + // adapter returns from `checkFilePath`. + it("FF14c: a payload-free success is accepted, however it is spelled", function* () { + const dir = yield* useFixture(); - // A container that will not describe its own outcome is a provider - // contract failure, so the execution ends and later work stops. - expect(outcome.ok).toBe(false); - expect(parseFilesFatal(fatalCause(outcome.error))).toEqual({ - type: "executablemd.runtime.files-fatal/v1", - kind: "invariant", - category: "protocol", + for (const admitted of [() => Ok(undefined), () => Ok(void 0)]) { + const outcome = yield* run(dir, 'content\n\nAFTER', function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *checkFilePath() { + return admitted(); + }, }); - expect(outcome.output).not.toContain("AFTER"); - expect(outcome.output).not.toContain("INSIDE"); - expect(String(outcome.error)).not.toContain("planted"); - expect(String(outcome.error)).not.toContain("EACCES"); - } + }); + + expect(outcome.ok).toBe(true); + expect(outcome.output).toContain("AFTER"); + expect(yield* readTextFile(join(dir, "out.txt"))).toBe("content"); } + + // A payload-free success that carries a payload anyway is not one. The + // payload arrives through a Proxy, so the declared type survives without a + // cast and the boundary is what first observes it. + const carrying = yield* run(dir, 'content\n\nAFTER', function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *checkFilePath() { + return new Proxy(Ok(undefined), { + has(target, property) { + return property === "value" ? true : Reflect.has(target, property); + }, + get(target, property, receiver) { + return property === "value" + ? "/planted/secret.txt" + : Reflect.get(target, property, receiver); + }, + }); + }, + }); + }); + expect(carrying.ok).toBe(false); + expect(invariantCategory(carrying.error)).toBe("protocol"); + expect(leaked(carrying)).toBe(false); }); // FF15: the one hostile shape that is *not* fatal. A non-write failure whose From f869b681098de2a3957e82582dd6ae184141add6 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 9 Aug 2026 07:31:26 -0400 Subject: [PATCH 7/8] =?UTF-8?q?=F0=9F=94=92=20Recognize=20a=20search=20res?= =?UTF-8?q?ult's=20array=20brand=20totally?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Array.isArray` is itself an operation on provider-controlled data: it throws on a revoked Proxy. Running it outside the total readers let a raw TypeError leave the boundary untagged, so a search whose result was revoked before it was returned surfaced the platform's message and let the document carry on. The brand check now answers instead of throwing, and a value whose array identity cannot be inspected is malformed success data like any other. The guard sits inside the payload contract rather than around the call site, so it stays the thing under test: restoring the unguarded call reds the regression. The specification catalog gains the rows for behavior already implemented — HF12b and FF11 through FF15 — and the provider-failure prose now separates an outcome that will not say what it is, which is a contract violation, from one that reads fine and reports a failure this version does not recognize, which is the generic sentence. --- packages/core/src/files.ts | 36 +++++++++++++++++++---- packages/core/tests/files-fatal.test.ts | 34 ++++++++++++++++++++++ specs/executable-mdx-spec.md | 38 +++++++++++++++++++++---- 3 files changed, 97 insertions(+), 11 deletions(-) diff --git a/packages/core/src/files.ts b/packages/core/src/files.ts index 7183df00..118b1f04 100644 --- a/packages/core/src/files.ts +++ b/packages/core/src/files.ts @@ -117,6 +117,22 @@ function read(target: unknown, name: string): { readonly value: unknown } | unde } } +/** + * Whether this really is an array, or `undefined` when asking threw. + * + * Recognizing the brand is itself an operation on provider-controlled data: + * `Array.isArray` throws on a revoked Proxy. A value whose array identity + * cannot be inspected has not described itself, which is the same answer as + * not being one. + */ +function isArray(value: unknown): boolean | undefined { + try { + return Array.isArray(value); + } catch { + return undefined; + } +} + /** * How a `Result` settled, or `undefined` when it will not say. * @@ -175,6 +191,14 @@ function* outcome( contract: { operation: FilesOperation; phase: FilesPhase; + /** + * Recognize this operation's payload, or answer `undefined`. + * + * Every contract here is **total**: it inspects provider-controlled data, + * so it answers rather than throws. Guarding the call site instead would + * make the guards inside each contract untestable, and an untestable guard + * is one nobody notices going missing. + */ payload: (value: unknown) => { readonly value: T } | undefined; }, ): Operation> { @@ -204,18 +228,18 @@ function text(value: unknown): { readonly value: string } | undefined { /** * A search result, copied out of whatever the provider handed back. * - * Every step of the copy is provider-controlled: iterator lookup, `length`, and - * each element are all trappable, so the walk is by index through the same - * total reader rather than by `for…of`. `undefined` from any of them is a - * refusal to describe the result, which the caller turns into one fixed - * invariant. + * Every step is provider-controlled — recognizing the array brand, reading + * `length`, and reading each element — so each goes through a total reader and + * the walk is by index rather than by `for…of`. Even `Array.isArray` can throw: + * a revoked Proxy refuses it. `undefined` from any step is a refusal to describe + * the result, which the caller turns into one fixed invariant. * * The array itself is rebuilt rather than passed along. A provider could return * something array-shaped whose elements are accessors, or one it goes on * mutating afterwards — and a document binds this value. */ function paths(value: unknown): { readonly value: string[] } | undefined { - if (!Array.isArray(value)) { + if (isArray(value) !== true) { return undefined; } const length = read(value, "length"); diff --git a/packages/core/tests/files-fatal.test.ts b/packages/core/tests/files-fatal.test.ts index d5069773..905d01e5 100644 --- a/packages/core/tests/files-fatal.test.ts +++ b/packages/core/tests/files-fatal.test.ts @@ -1053,6 +1053,40 @@ describe("Tier FF — Files infrastructure failure", () => { expect(leaked(outcome)).toBe(false); } + // A revoked Proxy is the shape that escapes everything else: recognizing it + // as an array is itself an operation on provider-controlled data, and + // `Array.isArray` throws on one. It is built valid, wrapped in `Ok` without + // being inspected, and revoked before the provider returns — so the first + // thing to touch it is the boundary, and the brand check is what it touches + // it with. + const revoked = yield* run( + dir, + '\n\nAFTER', + function* () { + yield* useHostFiles(); + yield* Files.around({ + // deno-lint-ignore require-yield + *globFiles() { + const { proxy, revoke } = Proxy.revocable(["notes.md"], {}); + const result = Ok(proxy); + revoke(); + return result; + }, + }); + }, + ); + + expect(revoked.ok).toBe(false); + expect(invariantCategory(revoked.error)).toBe("protocol"); + expect(revoked.output).not.toContain("AFTER"); + // The platform's own failure is replaced, not carried. + const selected = fatalCause(revoked.error); + expect(selected).not.toBeInstanceOf(TypeError); + expect(selected instanceof Error ? selected.cause : "unset").toBeUndefined(); + const shown = [revoked.output, String(selected), String(revoked.error)].join(" "); + expect(shown).not.toContain("IsArray"); + expect(shown).not.toContain("revoked"); + // The iterator is never consulted, because the walk is by index. A search // whose only hostile trap is `Symbol.iterator` therefore copies cleanly — // which is the derivation this kills: a `for…of` walk would have run it. diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 76825938..5a63a362 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4970,11 +4970,31 @@ operation-failed ``` A failure carries that reason and the phase it came from, as a plain frozen -object under a stable tag, and the component **parses** it before reading a -field. Data that does not validate is treated as absent rather than trusted: -for a read or a search that means the generic phrase, and for a write it is a -provider-contract failure (below), because every sentence a write could print -makes a claim about whether the file was replaced. +object under a stable tag, and it is **parsed** before any field is read. Data +that does not validate is treated as absent rather than trusted: for a read or a +search that means the generic phrase, and for a write it is a provider-contract +failure (below), because every sentence a write could print makes a claim about +whether the file was replaced. + +The outcome a provider returns is checked the same way, and the distinction +matters more than it looks. An outcome that **will not say** what it is — one +that does not report whether it succeeded, or whose success value or failure +cannot be read at all — has described nothing, and is a provider-contract +failure. An outcome that reads perfectly well but carries a failure the +vocabulary does not recognize *has* described something, just not in terms this +version knows: for a read or a search that is the generic sentence and the +document carries on. + +One operation qualifies that. Admitting a path succeeds with no value at all, so +an outcome that carries none is its ordinary success; what is refused there is a +value that cannot be read, and one that is present but is something other than +nothing. Every other operation's success carries a value and requires it to be +readable. + +Nothing a provider returned is passed onward. What reaches the component is +rebuilt from the fields that validated — including a search's list of paths, +which is copied, so what a document binds is not something the provider can +still change. The error's class carries no authority either. A `FileAccessError` arriving from a provider call is replaced like any other, because a class says nothing @@ -6747,6 +6767,7 @@ platform's. | HF10, HF10b | Where the guarantee stops | A parent replaced synchronously between resolution and use is written through, and a target replaced between resolution and access is read through — the documented weakness, with atomicity still holding | | HF11 | The commit is one event | A fault before and after `next()` report the same unknown outcome, and one of the two runs really did commit | | HF12 | Cancellation | No Result is produced and no temporary is left | +| HF12b | Cleanup failing as cancellation unwinds | There is no outcome to report it beside, so a fixed teardown invariant leaves the scope instead of a manufactured Result, carrying neither the platform's error nor the generated temporary's name | | HF13 | Temporary directories | Live and die with the acquiring scope; a halt before acquisition leaves nothing | | HF14 | Absence | Every operation throws provider-unavailable with the fixed diagnostic and no cause, and no low-level call is made | | HF15 | Installation | `useHostFiles()` installs beneath ordinary middleware, which can still wrap it | @@ -6767,6 +6788,13 @@ platform's. | FF8 | Identity is preserved | A nested durability failure and a nested Files failure are each rethrown as the same object | | FF9 | Precedence at the wrapper | A durability failure beneath a Files invariant is the one preserved | | FF10 | Ordinary failures | A missing file is still a printed error and the sibling still runs | +| FF11 | Hostile shapes are recognized, never fatal in themselves | A throwing `data` accessor, fields and key enumeration that refuse, an unreadable prototype, a throwing `cause`, unreadable or non-list aggregate members, and unreadable teardown causes are each unrecognized rather than thrown, and a real failure beneath one is still found | +| FF12 | An unsafe tagged candidate is replaced, not preserved | The right tag plus a raw message, a cause chain, mutable data, an extra data field, a path-bearing `name`, an extra Error-level property, an enumerable symbol payload, or hostile enumeration each fail the identity contract; the planted value survives neither stringification nor enumeration, and the real constructors stay recognizable | +| FF13 | Cancellation cleanup is discovered as fatal | A host cleanup that fails while cancellation unwinds is selected by the engine's own fatal discovery, by identity, with nothing of the platform's failure in it | +| FF14 | A hostile Result never reaches a component | An `ok`, `value` or `error` that throws, is absent, or is the wrong type is a fatal protocol violation for read, write, path admission, search and temporary-directory alike; no child expands, no later sibling runs, and nothing planted escapes | +| FF14b | Search results are recognized and copied totally | A refusing array brand, `length` or element is a fatal protocol violation; the walk never consults the iterator; and the array a document binds is its own copy | +| FF14c | A payload-free success is accepted however it is spelled | Effection's `Unit` carries no `value` member, so an absent one is the ordinary path-admission success — while a present but unreadable one, and a present one that is not `undefined`, are fatal | +| FF15 | Readable malformed failure data stays printable | A non-write failure whose data does not validate renders the generic sentence and the document carries on, with nothing the provider put there reaching it | ### Tier OM — The `output` error mode From ab552cc9388aae6326a14917a8c2ca3ac510ff0b Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 9 Aug 2026 08:11:44 -0400 Subject: [PATCH 8/8] =?UTF-8?q?=F0=9F=93=9D=20State=20the=20non-write=20fa?= =?UTF-8?q?ilure=20rule=20for=20every=20non-write=20operation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The provider-failure passages named only a read and a search, which left path admission and TempDir unstated even though they take the same path: readable data that does not validate selects the generic sentence there too. The FF14 row described a stricter rule than the boundary implements. It now matches FF14c and FF15: an unreadable settlement is a contract failure, so is an unreadable selected member, and an absent success value only where the operation carries one — path admission succeeds without one. A readable but unrecognized non-write failure stays printable. FF11 is renamed to what it asserts. Declining to recognize a hostile shape is not recognizing it as a valid structural failure. --- specs/executable-mdx-spec.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 5a63a362..5beb9e4b 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4971,10 +4971,10 @@ operation-failed A failure carries that reason and the phase it came from, as a plain frozen object under a stable tag, and it is **parsed** before any field is read. Data -that does not validate is treated as absent rather than trusted: for a read or a -search that means the generic phrase, and for a write it is a provider-contract -failure (below), because every sentence a write could print makes a claim about -whether the file was replaced. +that does not validate is treated as absent rather than trusted: for a non-write +operation that means the generic phrase, and for a write it is a +provider-contract failure (below), because every sentence a write could print +makes a claim about whether the file was replaced. The outcome a provider returns is checked the same way, and the distinction matters more than it looks. An outcome that **will not say** what it is — one @@ -4982,7 +4982,7 @@ that does not report whether it succeeded, or whose success value or failure cannot be read at all — has described nothing, and is a provider-contract failure. An outcome that reads perfectly well but carries a failure the vocabulary does not recognize *has* described something, just not in terms this -version knows: for a read or a search that is the generic sentence and the +version knows: for a non-write operation that is the generic sentence and the document carries on. One operation qualifies that. Admitting a path succeeds with no value at all, so @@ -6788,10 +6788,10 @@ platform's. | FF8 | Identity is preserved | A nested durability failure and a nested Files failure are each rethrown as the same object | | FF9 | Precedence at the wrapper | A durability failure beneath a Files invariant is the one preserved | | FF10 | Ordinary failures | A missing file is still a printed error and the sibling still runs | -| FF11 | Hostile shapes are recognized, never fatal in themselves | A throwing `data` accessor, fields and key enumeration that refuse, an unreadable prototype, a throwing `cause`, unreadable or non-list aggregate members, and unreadable teardown causes are each unrecognized rather than thrown, and a real failure beneath one is still found | +| FF11 | Hostile-shape inspection is total | A throwing `data` accessor, fields and key enumeration that refuse, an unreadable prototype, a throwing `cause`, unreadable or non-list aggregate members, and unreadable teardown causes are each declined rather than allowed to throw — none of them is a valid structural failure — and a real failure beneath one is still found | | FF12 | An unsafe tagged candidate is replaced, not preserved | The right tag plus a raw message, a cause chain, mutable data, an extra data field, a path-bearing `name`, an extra Error-level property, an enumerable symbol payload, or hostile enumeration each fail the identity contract; the planted value survives neither stringification nor enumeration, and the real constructors stay recognizable | | FF13 | Cancellation cleanup is discovered as fatal | A host cleanup that fails while cancellation unwinds is selected by the engine's own fatal discovery, by identity, with nothing of the platform's failure in it | -| FF14 | A hostile Result never reaches a component | An `ok`, `value` or `error` that throws, is absent, or is the wrong type is a fatal protocol violation for read, write, path admission, search and temporary-directory alike; no child expands, no later sibling runs, and nothing planted escapes | +| FF14 | A hostile outcome never reaches a component | A settlement that is absent, unreadable, or not a boolean is a provider-contract failure; so is a selected failure that is absent or unreadable, and a success value that is absent or unreadable for the operations that carry one — path admission's succeeds without a value (FF14c). A readable but invalid success payload is a contract failure too, while a readable but unrecognized non-write failure takes FF15's printable path. Where the outcome is a contract failure no child expands, no later sibling runs, and nothing planted escapes | | FF14b | Search results are recognized and copied totally | A refusing array brand, `length` or element is a fatal protocol violation; the walk never consults the iterator; and the array a document binds is its own copy | | FF14c | A payload-free success is accepted however it is spelled | Effection's `Unit` carries no `value` member, so an absent one is the ordinary path-admission success — while a present but unreadable one, and a present one that is not `undefined`, are fatal | | FF15 | Readable malformed failure data stays printable | A non-write failure whose data does not validate renders the generic sentence and the document carries on, with nothing the provider put there reaching it |