diff --git a/architecture.md b/architecture.md index 38c9cd71..ca3ee9dd 100644 --- a/architecture.md +++ b/architecture.md @@ -55,6 +55,9 @@ Existing documents and code get aligned to this section retroactively. | checkpoint | a completed journal boundary associated with the logical Workspace root visible after that effect | | history fork | a new workflow run that replays a compatible journal prefix and continues from its checkpoint and Workspace root under a new immutable document definition | | loaded copy | one independently evaluated instance of a package, such as the copy bundled into the binary or a separately installed dependency | +| authority | the power to decide what an execution or effect *is* — whether it happens, what it may replay from, and what it settles to — as distinct from the power to observe or refuse one | +| authoritative behavior | behavior that exercises authority; non-authoritative behavior may inspect, narrow, refuse or add a failure, but cannot bring an execution into being, substitute one, or rescue one | +| trusted host | the code that decides what an execution is for — a CLI entrypoint or a workflow runner — as distinct from the document, the components it expands, and the middleware packages composed around it | | `JournalProvenance` | a non-operational, equality-only witness that a live publication stream descends from the exact journal backend a provider selected for one workflow run; it grants no append, read, execution, publication or reconciliation capability, and is meaningful only because the provider retains the witness it established and later requires exact equality | ## Three axes @@ -1008,6 +1011,93 @@ hidden inside library objects that accumulate. One exception: metadata an author declares at module evaluation, about a value the author owns, may live on that value. +## Authoritative behavior + +**Authoritative** behavior decides what an execution *is*: whether it runs at +all, what history it may replay from, what options it runs under, and what it +settles to. **Non-authoritative** behavior observes, narrows, refuses, or adds — +it can stop something from happening and can make a success into a failure, but +it cannot bring an execution into being, substitute one, or rescue one. + +Public middleware is non-authoritative by construction. `Execution` and +`ReplayGuard` handlers compose lexically, and a handler installed further out +may answer without delegating — so anything they could decide would be decided +by registration order. They may refuse; they may not complete. + +### Capability-backed execution + +Canonical core is authoritative for document execution. It invokes the stable +`Execution` middleware through a private, per-invocation same-name Api whose +instance-owned default handler is the authoritative terminal. A stable name +shares the middleware context, so every public handler — including one installed +through another loaded copy's descriptor — composes exactly as it always did; +what a name does not share is the default handler, and that is where authority +sits. The exported `Execution.execute` default always refuses, so calling it +with a captured request settles nothing. + +`Execution.execute` middleware is handed an opaque `ExecutionRequest` and +returns nothing: it may +inspect the options, narrow them, register an additive completion policy, +install contextual behavior, refuse by throwing, and delegate. The document is +run afterwards, by the invocation that issued the request, under the options the +canonical terminal recorded. + +Terminal acceptance stores a detached, immutable structural snapshot of the +options: the containers are copied and frozen while the operational identities +inside them — the selected `DurableStream`, each modifier factory — are carried +across unchanged. The chain unwinds before the document runs, so an outer +handler that delegates and then edits what it delegated changes only its own +data. Replacing options before delegating, through `withOptions()`, remains the +supported path. + +Each invocation owns a child scope, held by one structured owner task, and +**settlement closes it** — on success, on failure, and on cancellation alike. +Contextual behavior an installation establishes is visible to the document and +to its teardown, isolated from a concurrent invocation, and absent from the next +execution in the same host scope. The final `Result` is published only after +that scope has finished tearing down, so a caller continuing on the completion +continues after cleanup, and a completed handle carries no live scope. + +A document outcome and an invocation-teardown failure are kept apart until the +scope has closed and then ranked, never replaced: a durability failure wins from +wherever it came and is returned by identity, then a Files infrastructure +failure on the same terms, then an existing document failure, and only a success +is converted by teardown. Every finalizer runs, exactly once, before the result +is observable. Once a handle exists, teardown contributes to its `Result` rather +than escaping as a thrown completion. Canonical +core constructs the one authoritative handle; it exposes the inner execution's +replay-safe output directly rather than bridging it through a second channel. + +The request is the capability, and it is **one-use**. It carries a private +reference to one invocation; a reconstructed look-alike, a superseded request, a +foreign invocation's request, and a second delegation are each refused with a +fresh, cause-free `ExecutionProtocolError`, before any journal read, expansion +or append. Reaching the end of the chain without consuming the request is the +same refusal. + +### Trusted host orchestration + +A **trusted host** is the code that decides what an execution is for — a CLI +entrypoint, a workflow runner — as distinct from the document, the components it +expands, and the middleware packages composed around it. + +A host attaches requirements to one execution through +`@executablemd/core/host`: `executeInstalled(options, installations)`, where an +installation carries `admissions` and an optional `install()`. Admissions are +copied and frozen **before** any installation runs, so what ends up +authoritative is fixed before any installed code, any middleware and any +document code exists. Each runs inside the execution's own journal read, on the +retained snapshot, in capture order, stopping at the first refusal — ahead of +root-history admission, `ReplayGuard`, terminal reuse, authored work and any +append. + +Admissions are refusal-only functions: they receive the retained history, return +nothing, and never receive a `next`. They travel as values the host holds and +passes — not through a context, an Api, a stable name, structural metadata or +module-scoped state — which is why a separately loaded package composes here by +handing over a closure rather than by agreeing on a name, and why no middleware +can read, transport or remove the collection. + ### The weak journal-provenance association Journal provenance is the one further exception, and it is deliberately narrow. diff --git a/packages/core/deno.json b/packages/core/deno.json index b61325a9..5420db1a 100644 --- a/packages/core/deno.json +++ b/packages/core/deno.json @@ -2,7 +2,8 @@ "name": "@executablemd/core", "version": "0.8.0", "exports": { - ".": "./mod.ts" + ".": "./mod.ts", + "./host": "./host.ts" }, "imports": { "@effectionx/context-api": "npm:@effectionx/context-api@0.6.0", diff --git a/packages/core/host.ts b/packages/core/host.ts new file mode 100644 index 00000000..2ec426cb --- /dev/null +++ b/packages/core/host.ts @@ -0,0 +1,28 @@ +/** + * @module + * + * The infrastructure boundary of document execution. + * + * Attaching an admission to an execution is a host act, not an authoring one: + * it decides what a retained history must satisfy before the document is + * allowed to replay from it. Keeping it behind its own entrypoint is what makes + * that visible at the import — nothing a document, a component or a middleware + * package reaches by importing `@executablemd/core` can require anything of a + * journal. + * + * The value crosses as a plain function the host holds and passes: + * + * ```ts + * import { executeInstalled } from "@executablemd/core/host"; + * + * const execution = yield* executeInstalled(options, [installation]); + * ``` + * + * That is also why a separately loaded package composes here. It hands the host + * a closure and the host hands it to canonical core; neither of them agrees on + * a name, looks anything up, or shares a registry, so there is nothing for a + * second copy to disagree about and nothing for anyone else to reach. + */ + +export { executeInstalled } from "./src/execute.ts"; +export type { ExecutionInstallation, JournalAdmission } from "./src/execute.ts"; diff --git a/packages/core/mod.ts b/packages/core/mod.ts index 45af4b8a..aeb30473 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -130,6 +130,10 @@ export type { ExecutionApi, DocumentExecution, } from "./src/execute.ts"; +// The narrowing surface a middleware handler needs, and nothing that completes +// an execution: `executeInstalled` and `JournalAdmission` are host boundary. +export { ExecutionProtocolError } from "./src/execution-request.ts"; +export type { CompletionFailure, ExecutionRequest } from "./src/execution-request.ts"; export { fileSource, formatDocumentReference, diff --git a/packages/core/package.json b/packages/core/package.json index cefd4073..f01785cb 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -4,7 +4,8 @@ "description": "Core engine that evaluates executable.md documents.", "type": "module", "exports": { - ".": "./mod.ts" + ".": "./mod.ts", + "./host": "./host.ts" }, "dependencies": { "@effectionx/context-api": "0.6.0", diff --git a/packages/core/src/agent/components.ts b/packages/core/src/agent/components.ts index f8fabbee..8ed8ebb0 100644 --- a/packages/core/src/agent/components.ts +++ b/packages/core/src/agent/components.ts @@ -1,29 +1,30 @@ /** * Agent component registration (specs/acp-client-spec.md). * - * Registers the agent words as ordinary function components for this scope, - * and decorates the Execution Api so prompt failures — and, when a root - * provider is configured, provider teardown failures — participate in the - * DocumentExecution completion. + * Registers the agent words as ordinary function components for this scope, and + * registers an additive completion policy so prompt failures — and, when a root + * provider is configured, provider teardown failures — turn an otherwise + * successful document into a failure. * - * Registration and execution decoration are separate concerns: the components - * are defaults a document can replace by writing its own file with one of these - * names, while the completion decoration belongs to the execution regardless of - * which implementation answered. + * Registration and completion policy are separate concerns: the components are + * defaults a document can replace by writing its own file with one of these + * names, while the policy belongs to the execution regardless of which + * implementation answered. * - * The root provider's lifetime is part of the execution: the middleware - * returns a bridged DocumentExecution whose owning spawned operation - * enters a scoped provider lifetime, runs the inner execution, forwards - * its output (the bridged output closes when the inner output closes), - * and resolves the completion only after provider cleanup has finished. - * Teardown failures therefore affect the final result without delaying - * rendered output. + * The root provider's lifetime is `Execution.document`. The provider scope + * surrounds the document's expansion and ends while the journal is still live, + * so cleanup has finished before the completion settles and rendered output is + * not delayed by it. A confirmed full replay never enters the provider at all. + * + * Completion precedence is first-failure: a document that already failed keeps + * its own failure, and prompt and teardown failures are added to a success + * rather than replacing anything. */ import { Err, scoped, spawn, withResolvers } from "effection"; import type { Operation, Result } from "effection"; import { Execution } from "../execute.ts"; -import type { DocumentExecution, ExecuteOptions } from "../execute.ts"; +import type { DocumentResult } from "../execute.ts"; import { registerComponents } from "../components/registration.ts"; import { CORE_ORIGIN } from "../components/registry.ts"; import { createReplayStream } from "../replay-stream.ts"; @@ -90,7 +91,7 @@ export function* installAgentComponents(options?: AgentComponentsOptions): Opera const rootProvider = options?.rootProvider; yield* Execution.around({ - *execute([executeOptions], next) { + *execute([request], next) { // Fresh per-execution prompt bookkeeping: an explicit sequence // records execution order in the journal, and per-location ordinals // keep durable identities stable through loops. @@ -117,7 +118,7 @@ export function* installAgentComponents(options?: AgentComponentsOptions): Opera // Confirmed full replay: durableRun returns the stored root result // without re-expanding, so no prompt would re-record. Restore the // journaled failures into this execution's collector instead. - const replayed = yield* readCompletedPrompts(executeOptions.stream); + const replayed = yield* readCompletedPrompts(request.options.stream); if (replayed) { for (const record of replayed) { const failure = promptFailureFromRecord(record); @@ -127,147 +128,89 @@ export function* installAgentComponents(options?: AgentComponentsOptions): Opera } } - // A confirmed full replay restores results from the journal, so it must - // never enter the root-provider lifetime — no availability check, setup, - // or prompt. Only a live run with a root provider bridges the provider. - if (!rootProvider || replayed) { - const inner = yield* next(executeOptions); - return decorateCompletion(inner, (result) => - combineCompletion(result, failures, undefined), - ); + // The provider's lifetime has to surround authored work and end while the + // journal is still live, which is what `Execution.document` is. Installed + // from here so it inherits this execution's replay decision — a confirmed + // full replay never enters the provider at all. + const teardown: TeardownSlot = {}; + if (rootProvider && !replayed) { + yield* Execution.around({ + *document([props], nextDocument) { + return yield* withRootProvider(rootProvider, teardown, () => nextDocument(props)); + }, + }); } - return yield* bridgeRootProvider(rootProvider, executeOptions, failures, next); + // Additive: prompt failures and a provider teardown failure turn a + // successful document into a failure. A document that already failed + // keeps its own failure — the completion policy adds, it does not replace. + request.addCompletionFailure(() => completionFailure(failures, teardown.error)); + yield* next(request); }, }); } -function* bridgeRootProvider( - rootProvider: { factory: AgentProviderFactory; options: AgentProviderOptions }, - executeOptions: ExecuteOptions, - failures: SequencedFailure[], - next: (options: ExecuteOptions) => Operation, -): Operation { - const channel = createReplayStream(); - const completion = withResolvers>(); - - yield* spawn(function* () { - let docResult: Result | undefined; - let teardown: Error | undefined; - let outputClosed = false; - let emitted = ""; - - try { - yield* scoped(function* () { - yield* rootProvider.factory(rootProvider.options); - const inner = yield* next(executeOptions); - const subscription = yield* inner.output; - let chunk = yield* subscription.next(); - while (!chunk.done) { - emitted += chunk.value; - yield* channel.send(chunk.value); - chunk = yield* subscription.next(); - } - yield* channel.close(chunk.value); - outputClosed = true; - docResult = yield* inner; - }); - } catch (error) { - const failure = error instanceof Error ? error : new Error(String(error)); - if (docResult === undefined) { - docResult = Err(failure); - } else { - // The inner execution completed; the throw came from dismantling - // the provider scope. - teardown = failure; - } - } - - if (!outputClosed) { - yield* channel.close(emitted); - } - completion.resolve( - combineCompletion( - docResult ?? Err(new Error("document execution did not complete")), - failures, - teardown, - ), - ); - }); - - return { - output: channel, - *[Symbol.iterator]() { - return yield* completion.operation; - }, - }; +interface TeardownSlot { + error?: Error; } /** - * Map an execution's completion: an `Ok` becomes `Err(failure())` when the - * policy reports one, after the inner completion — and therefore its - * closed output stream — settles. An existing `Err` passes through - * unchanged. (Local copy of the testing package's decorator — core cannot - * depend on testing.) + * Run the document inside the root provider's lifetime. + * + * A failure raised while dismantling the provider is recorded rather than + * thrown: the document already completed, and reporting the teardown as *its* + * failure would replace a result the document earned. It becomes an additive + * completion failure instead, which is the same precedence the bridged + * implementation had. */ -function decorateCompletion( - inner: DocumentExecution, - decorate: (result: Result) => Result, -): DocumentExecution { - return { - output: inner.output, - *[Symbol.iterator]() { - const result = yield* inner; - if (!result.ok) { - return result; - } - return decorate(result); - }, - }; +function* withRootProvider( + rootProvider: { factory: AgentProviderFactory; options: AgentProviderOptions }, + teardown: TeardownSlot, + body: () => Operation, +): Operation { + let completed: DocumentResult | undefined; + try { + return yield* scoped(function* () { + yield* rootProvider.factory(rootProvider.options); + completed = yield* body(); + return completed; + }); + } catch (error) { + if (completed === undefined) { + throw error; + } + teardown.error = error instanceof Error ? error : new Error(String(error)); + return completed; + } } /** - * Combine the document result, collected prompt failures, and provider - * teardown failure into the final completion. Primary failures come - * before teardown failures; existing AggregateError members are flattened - * rather than nested. + * The one failure a successful document earns from prompts and teardown. + * + * Flat, and primary first: the prompt failures in the order they were recorded, + * then whatever dismantling the provider raised. A reader looking for what went + * wrong first finds it first, and an `AggregateError` a member already is gets + * unpacked rather than nested. */ -function combineCompletion( - docResult: Result, +function completionFailure( failures: SequencedFailure[], teardown: Error | undefined, -): Result { +): Error | undefined { const promptErrors = [...failures] .sort((a, b) => a.sequence - b.sequence) .map((failure) => failure.error); const promptMessage = `${promptErrors.length} agent prompt(s) failed`; - if (!docResult.ok) { - if (!teardown) { - return docResult; - } - return Err( - new AggregateError( - [...flatten(docResult.error), ...flatten(teardown)], - "document execution and agent provider teardown failed", - ), - ); - } if (promptErrors.length > 0 && teardown) { - return Err( - new AggregateError( - [...promptErrors, ...flatten(teardown)], - `${promptMessage}; agent provider teardown failed`, - ), + return new AggregateError( + [...promptErrors, ...flatten(teardown)], + `${promptMessage}; agent provider teardown failed`, ); } if (promptErrors.length > 0) { - return Err(new AggregateError(promptErrors, promptMessage)); - } - if (teardown) { - return Err(teardown); + return new AggregateError(promptErrors, promptMessage); } - return docResult; + return teardown; } function flatten(error: Error): Error[] { diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index c8fd4a83..9a545bd0 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -10,7 +10,7 @@ * See DEC-005 in specs/decisions.md. */ -import { Err, Ok, scoped, spawn, withResolvers, until } from "effection"; +import { Err, Ok, ensure, scoped, spawn, withResolvers, until } from "effection"; import type { Operation, Result, Stream } from "effection"; import { type Api, createApi, type Operations } from "@effectionx/context-api"; import { @@ -29,6 +29,8 @@ import { exec, readTextFile, cwd } from "@executablemd/runtime"; import { cwd as processCwd } from "@effectionx/fs"; import type { Workflow, Json } from "@executablemd/durable-streams"; import { createReplayStream } from "./replay-stream.ts"; +import { ExecutionProtocolError, issueExecution } from "./execution-request.ts"; +import type { CompletionFailure, ExecutionRequest } from "./execution-request.ts"; import { createContext } from "effection"; import type { Context } from "effection"; import type { @@ -82,6 +84,7 @@ import { documentationError, documentationFailure, durabilityFailure, + filesFatalFailure, useSegmentCauses, } from "./errors.ts"; import { Component, importComponent } from "./component-api.ts"; @@ -612,12 +615,30 @@ function guardedJournal( stream: DurableStream, root: RootDocumentSource, coroutineId: CoroutineId, + admissions: readonly JournalAdmission[], ): DurableStream { const admitting: DurableStream = { *readAll(): Operation { - const retained = retainEvents(yield* stream.readAll()); + // Frozen before anyone is offered it, and offered to everyone. `readonly` + // is a compile-time claim: without this an admission could splice, reorder + // or empty the history in place, and every later admission, root-history + // validation and the replay itself would consume what it left behind. + // The events themselves are the retained graph's own, already sealed. + const retained: readonly DurableEvent[] = Object.freeze( + retainEvents(yield* stream.readAll()), + ); + // What the trusted host required of this history, on that exact snapshot, + // in the order it was captured and stopping at the first refusal. Ahead of + // root-history admission, ReplayGuard, terminal reuse, authored work and + // any append. + for (const admission of admissions) { + yield* admission(retained); + } admitRootHistory(retained, root, coroutineId); - return retained; + // The same objects the admissions were held to. `readAll` is declared + // mutable by the protocol, so this is a fresh array over the identical + // sealed events rather than a second reading of the backend. + return [...retained]; }, append: (event: DurableEvent) => stream.append(event), }; @@ -793,7 +814,7 @@ const silentFactory: ModifierFactory = (_params) => (_args, next) => * rendered text for a text root, the validated JSON for a value root. The pair * is journaled together so replay restores both; only `value` is public. */ -type DocumentResult = DocumentSuccess | DocumentFailureResult; +export type DocumentResult = DocumentSuccess | DocumentFailureResult; type DocumentSuccess = { status: "ok"; @@ -1260,7 +1281,11 @@ export interface DocumentExecution extends Operation> { * }, execution.output); * ``` */ -function* executeDocument(options: ExecuteOptions): Operation { +function* executeDocument( + options: ExecuteOptions, + admissions: readonly JournalAdmission[] = [], + completions: readonly CompletionFailure[] = [], +): Operation { const { stream, props = {}, @@ -1379,7 +1404,7 @@ function* executeDocument(options: ExecuteOptions): Operation // check happens inside the read that every phase downstream depends on // rather than in middleware anything could replace. const returned = yield* durableRun(() => Execution.operations.document(props), { - stream: guardedJournal(journal, root, ROOT_COROUTINE), + stream: guardedJournal(journal, root, ROOT_COROUTINE, admissions), }); // Taken rather than read, so the handoff belongs to the run that made it. const live = yield* takeLiveFailure(liveFailure); @@ -1400,7 +1425,7 @@ function* executeDocument(options: ExecuteOptions): Operation resolve(Err(failure instanceof Error ? failure : new Error(String(failure)))); return; } - resolve(Ok(result.value)); + resolve(settleCompletion(Ok(result.value), completions)); } catch (error) { // Close with everything already emitted — printed errors produced before // an abort stay visible to consumers of the close value. @@ -1418,14 +1443,39 @@ function* executeDocument(options: ExecuteOptions): Operation } /** - * Execution Api — a test-agnostic middleware surface around document - * execution. The default provider runs the document; extensions decorate the - * execution lifecycle with `Execution.around({ execute })` — observing - * options, wrapping the returned handle, or mapping its completion Result — - * without introducing another execution function. + * What an installation requires of the retained history, decided inside the + * execution's own journal read. + * + * Refusal-only: it throws or it returns. It is handed the exact retained + * snapshot and hands nothing back, so it cannot substitute a history, and it + * never receives a `next` to decline. + */ +export type JournalAdmission = (retained: readonly DurableEvent[]) => Operation; + +/** + * What a trusted host attaches to one execution. + * + * `install` runs contextual behavior the document inherits. `admissions` is + * copied by canonical execution *before* `install` runs, so nothing an + * installation does afterwards — including anything it composes — can add to, + * remove from or observe the collection that ends up authoritative. + */ +export interface ExecutionInstallation { + readonly admissions?: readonly JournalAdmission[]; + install?(): Operation; +} + +/** + * Execution Api — a policy surface around document execution. + * + * A handler is given an `ExecutionRequest`, not the execution. It may inspect + * the options, narrow or replace them with `withOptions()`, register an + * additive completion failure, install contextual behavior the document will + * inherit, refuse by throwing, and delegate. It returns nothing, and whatever + * it returns is ignored: only canonical execution completes a document. */ export interface ExecutionApi { - execute(options: ExecuteOptions): Operation; + execute(request: ExecutionRequest): Operation; /** * The document's expansion, as `durableRun` runs it. * @@ -1436,13 +1486,297 @@ export interface ExecutionApi { document(props: Record): Operation; } +/** + * The public Execution surface. + * + * Its `execute` default always refuses. A stable name composes replaceable + * policy across loaded copies, and this descriptor is the one everybody can + * reach — so it must not be a terminal that would settle any branded request + * handed to it. Canonical core dispatches through a private instance instead. + */ export const Execution: Api = createApi("Execution", { - *execute(options: ExecuteOptions): Operation { - return yield* executeDocument(options); + // deno-lint-ignore require-yield + *execute(_request: ExecutionRequest): Operation { + throw new ExecutionProtocolError("invoked execution outside canonical core"); }, *document(props: Record): Operation { return yield* documentWorkflow(props); }, }); -export const execute: Operations["execute"] = Execution.operations.execute; +/** + * Run one document execution, authoritatively, with the invocation owning its + * own lifetime. + * + * The order is the contract. Admissions are copied and frozen first, so what + * ends up authoritative is fixed before any installation, any middleware and + * any document code exists. Installations then run, then the chain is invoked + * with one opaque request, then the request must have reached the terminal + * exactly once, and only then does canonical core execute the document with the + * options the terminal recorded. + * + * One structured owner task holds the invocation scope. Everything an + * installation established, and every child the document spawned, lives inside + * it — and settlement closes it. That is the difference from attaching a + * resource to the caller: a resource would keep the invocation standing for as + * long as the caller's scope lasted, so a suspended authored child would still + * be running while the caller went on to other work. + * + * The handle canonical core returns is the authoritative one. It exposes the + * inner execution's replay-safe output directly — there is no second channel + * bridging identical chunks — and its completion is the owner's final result, + * published only after the invocation scope has finished tearing down. So a + * caller that continues on the completion continues after cleanup, and a + * completed handle carries no live scope to re-enter. + * + * Cancelling a live handle cancels the invocation. A consumer that is halted + * before settlement halts the owner on its way out, which is what closes the + * scope and the authored work inside it; the owner then settles the completion + * so no other observer is left waiting on a run that is over. Once settled, + * observing the handle again reads the recorded result and starts nothing. + * + * Failure before a handle exists keeps the existing pre-handle throwing + * behavior; once readiness is published, every later failure is a `Result`, + * reconciled with whatever the document itself produced. That reconciliation + * ranks by kind — durability, then Files infrastructure, then ordinary — and + * *within* a kind by occurrence, so the document's own failure precedes the + * teardown's and is returned by exact identity. + */ +function* runInvocation( + options: ExecuteOptions, + installations: readonly ExecutionInstallation[], + observed?: () => void, +): Operation { + const ready = withResolvers(); + const settled = withResolvers>(); + const state: { finished: boolean; document: Result | undefined } = { + finished: false, + document: undefined, + }; + + const finish = (result: Result): void => { + if (!state.finished) { + state.finished = true; + settled.resolve(result); + } + }; + + const owner = yield* spawn(function* () { + // Whatever ends this task — completion, failure, or a cancelled handle — + // leaves no observer waiting on a run that is over. A document that already + // produced an outcome keeps it: cancelling a handle during teardown ends the + // run, it does not erase what the run decided. + yield* ensure(() => { + finish( + state.document === undefined + ? Err(new Error("the document execution was cancelled")) + : reconcile(state.document, undefined), + ); + }); + + let published = false; + try { + yield* scoped(function* () { + const execution = yield* invoke(options, installations); + published = true; + ready.resolve(execution); + state.document = yield* execution; + }); + } catch (error) { + const teardown = error instanceof Error ? error : new Error(String(error)); + if (!published) { + // No handle exists, so this throws — but a durability or Files failure + // raised during cleanup is still the failure that gets reported. + ready.reject(fatalOf(teardown) ?? teardown); + return; + } + finish(reconcile(state.document, teardown)); + return; + } + finish(reconcile(state.document, undefined)); + }); + + const execution = yield* ready.operation; + return { + output: execution.output, + *[Symbol.iterator]() { + // Registered in the *consumer's* scope, so halting a consumer that is + // still waiting takes the invocation down with it. After settlement this + // does nothing, which is what lets a completed handle be read again. + yield* ensure(function* () { + if (!state.finished) { + yield* owner.halt(); + } + }); + // The callback this invocation was started with, captured by value at + // the boundary rather than reread from a record the caller still holds — + // so replacing that record afterwards changes nothing here. It exists so + // a test can cancel a consumer at the one moment that matters, after this + // observation is cancellable, without waiting a scheduler turn and calling + // that a proof. It carries nothing and decides nothing. + observed?.(); + return yield* settled.operation; + }, + }; +} + +/** The fatal failure this one carries, if it carries one. */ +function fatalOf(error: unknown): Error | undefined { + return durabilityFailure(error) ?? filesFatalFailure(error); +} + +/** + * The one result of a document whose invocation also failed to tear down. + * + * Both outcomes are kept until the scope has closed, and then ranked. A fatal + * failure is reported by identity from wherever it came — a durability failure + * first, then a Files infrastructure failure — because the engine's fences + * match on the exact object. Below that a document that already failed keeps + * its own failure, and only a success is converted by teardown. + */ +function reconcile(document: Result | undefined, teardown: Error | undefined): Result { + const failed = document !== undefined && !document.ok ? document.error : undefined; + + // Kind first, then occurrence *within* that kind. The document's outcome + // happened before the invocation was torn down, so when both carry the same + // kind of fatal failure the document's is the one reported — by identity, + // because the engine's fences match the exact object rather than a rebuilt + // one. + const durable = + (failed === undefined ? undefined : durabilityFailure(failed)) ?? + (teardown === undefined ? undefined : durabilityFailure(teardown)); + if (durable !== undefined) { + return Err(durable); + } + const files = + (failed === undefined ? undefined : filesFatalFailure(failed)) ?? + (teardown === undefined ? undefined : filesFatalFailure(teardown)); + if (files !== undefined) { + return Err(files); + } + + if (document !== undefined && !document.ok) { + return document; + } + if (teardown !== undefined) { + return Err(teardown); + } + return document ?? Err(new Error("the document execution did not complete")); +} + +function* invoke( + options: ExecuteOptions, + installations: readonly ExecutionInstallation[], +): Operation { + const admissions = Object.freeze( + installations.flatMap((installation) => [...(installation.admissions ?? [])]), + ); + + for (const installation of installations) { + if (installation.install) { + yield* installation.install(); + } + } + + const issued = issueExecution(options); + + // The terminal for this invocation and no other. + // + // A stable Api *name* shares the middleware context, so every public handler + // installed anywhere — including through another loaded copy's descriptor — + // composes around this call exactly as it composes around the public + // descriptor's. What a name does not share is the default handler: each + // `createApi()` instance owns its own, and this one is closed over this + // invocation. So the public chain terminates in a continuation no middleware + // can reach, replace, or reorder, and a request another invocation issued is + // refused here rather than settling somebody else's execution. + const invocationExecution = createApi<{ + execute(request: ExecutionRequest): Operation; + }>("Execution", { + // deno-lint-ignore require-yield + *execute(request: ExecutionRequest): Operation { + issued.consume(request); + }, + }); + + // Whatever a handler returns is not an execution, so it is not read. + yield* invocationExecution.operations.execute(issued.request); + + return yield* executeDocument(issued.settle(), admissions, issued.completions()); +} + +/** + * What one invocation may be watched by, and nothing more. + * + * Non-authoritative by construction: the callback takes no arguments, returns + * nothing, and is read at exactly one point. Nothing here can change what an + * execution does, what it settles to, or how it is torn down. + * + * The record is the caller's; what the invocation keeps is the function it held + * at the moment the invocation started. + */ +export interface InvocationObservers { + /** Called once a consumer of the returned handle has become cancellable. */ + observed?: () => void; +} + +/** The ordinary entrypoint: one execution, nothing installed around it. */ +export function execute(options: ExecuteOptions): Operation { + return runInvocation(options, []); +} + +/** + * The same invocation, watched. + * + * Package-internal and test-only: neither `@executablemd/core` nor + * `@executablemd/core/host` exports it, and the observers it takes belong to + * this call alone rather than to a slot every execution shares. + */ +export function executeObserved( + options: ExecuteOptions, + installations: readonly ExecutionInstallation[], + observers: InvocationObservers, +): Operation { + // The callback is read here, once, and passed on as a value. What the caller + // does to its own record afterwards is its own business. + return runInvocation(options, [...installations], observers.observed); +} + +/** + * The trusted-host entrypoint. + * + * Reached through `@executablemd/core/host`, because attaching an admission is + * infrastructure rather than authoring: the value crosses as a function the + * host holds and passes, so a separately loaded workflow package composes by + * handing its closure over rather than by agreeing on a name. + */ +export function executeInstalled( + options: ExecuteOptions, + installations: readonly ExecutionInstallation[], +): Operation { + return runInvocation(options, [...installations]); +} + +/** + * Apply every additive completion policy, in registration order. + * + * Additive means one direction only: the first policy that reports a failure + * turns a success into that failure, and nothing after it can turn a failure + * back into a success or replace it with a different one. + */ +function settleCompletion( + result: Result, + completions: readonly CompletionFailure[], +): Result { + let settled = result; + for (const completion of completions) { + if (!settled.ok) { + return settled; + } + const failure = completion(); + if (failure !== undefined) { + settled = Err(failure); + } + } + return settled; +} diff --git a/packages/core/src/execution-request.ts b/packages/core/src/execution-request.ts new file mode 100644 index 00000000..10e16229 --- /dev/null +++ b/packages/core/src/execution-request.ts @@ -0,0 +1,241 @@ +/** + * The capability-backed request one document execution is asked through. + * + * `Execution.execute` used to hand middleware the finished `DocumentExecution` + * and take back whatever it returned. That made every handler authoritative by + * construction: one could answer without delegating and its invented completion + * *was* the execution, and one could wrap the returned handle and rewrite what + * the document had already settled. + * + * Middleware is policy, so it gets a request rather than the execution. A + * handler may read the options, narrow or replace them, register an additive + * completion failure, refuse by throwing, install contextual behavior the + * document will inherit, and delegate. What it cannot do is *complete* the + * execution: the only thing that runs a document is canonical core, reached + * after the middleware chain unwinds, and it runs the options the terminal + * recorded. + * + * The capability is the request itself. It carries a private reference to one + * invocation and may be consumed exactly once, so a reconstructed look-alike, a + * stale request left over from `withOptions()`, a second delegation and a + * replayed request are all refusals rather than second executions. + */ + +import type { ExecuteOptions } from "./execute.ts"; +import type { Json } from "./types.ts"; + +/** + * A protocol violation by whoever is composed around an execution. + * + * Fresh and cause-free every time. What went wrong is the shape of the call, + * not anything a caller supplied, and attaching their value would carry it into + * logs and rendered output. + */ +export class ExecutionProtocolError extends Error { + override name = "ExecutionProtocolError"; + + constructor(problem: string) { + super( + `Execution middleware ${problem}. A handler may inspect, transform, refuse or ` + + "delegate a request; only canonical execution completes one.", + ); + } +} + +/** An additive completion policy: it may fail a success, never rescue a failure. */ +export type CompletionFailure = () => Error | undefined; + +/** + * What a handler is given. + * + * Opaque on purpose: the members below are the whole of what a handler may do + * with it, and none of them reaches the admissions the invocation captured. + */ +export interface ExecutionRequest { + /** The options as they stand, after every handler that has run so far. */ + readonly options: ExecuteOptions; + /** The same invocation, asked with different options. Supersedes this one. */ + withOptions(options: ExecuteOptions): ExecutionRequest; + /** Register a completion policy. Additive: it can fail a success, not rescue one. */ + addCompletionFailure(failure: CompletionFailure): void; +} + +/** + * One execution's private state. + * + * `generation` is what makes a superseded request stale: `withOptions()` mints + * the next generation, and only the newest may be consumed. Without it a + * handler could transform the options and then delegate the request it started + * from, running the document under options a later handler believed it had + * replaced. + */ +class Invocation { + generation = 0; + consumed = false; + settled: ExecuteOptions | undefined; + readonly completions: CompletionFailure[] = []; +} + +class CanonicalRequest implements ExecutionRequest { + readonly #invocation: Invocation; + readonly #generation: number; + readonly options: ExecuteOptions; + + constructor(invocation: Invocation, options: ExecuteOptions, generation: number) { + this.#invocation = invocation; + this.#generation = generation; + this.options = options; + Object.freeze(this); + } + + withOptions(options: ExecuteOptions): ExecutionRequest { + this.#invocation.generation += 1; + return new CanonicalRequest(this.#invocation, options, this.#invocation.generation); + } + + addCompletionFailure(failure: CompletionFailure): void { + this.#invocation.completions.push(failure); + } + + /** + * Take this request's options on behalf of `invocation`, once. + * + * The expected invocation is supplied by the caller rather than read off the + * request, which is the whole point: a request another invocation issued is + * *also* a canonical request, and accepting it would let one execution's + * terminal settle another's. + * + * Every check runs before anything is written, so a rejected delegation + * consumes neither invocation and both remain usable afterwards. + */ + static consume(request: unknown, invocation: Invocation): void { + // `#invocation in request` recognizes a value this class constructed + // without reading anything off it and without a registry to consult. It is + // also total: `in` on a primitive, on null, or on a proxy whose traps throw + // is guarded here, so nothing native or planted escapes. + if (!isCanonical(request)) { + throw new ExecutionProtocolError("delegated a request canonical execution did not issue"); + } + if (request.#invocation !== invocation) { + throw new ExecutionProtocolError("delegated a request another execution issued"); + } + if (invocation.consumed) { + throw new ExecutionProtocolError("delegated an execution request more than once"); + } + if (request.#generation !== invocation.generation) { + throw new ExecutionProtocolError("delegated a request that a later withOptions() superseded"); + } + // Snapshotted before the invocation is marked consumed, so a failure to + // detach leaves nothing consumed either. + const settled = detachOptions(request.options); + invocation.consumed = true; + invocation.settled = settled; + } + + /** Whether this class built `value`, answered without trusting it. */ + static own(value: unknown): value is CanonicalRequest { + if (typeof value !== "object" || value === null) { + return false; + } + try { + return #invocation in value; + } catch { + // A revoked proxy, or one whose `has` trap refuses. Not one of ours. + return false; + } + } +} + +/** + * The options this execution runs under, detached from the caller. + * + * The chain unwinds before canonical core runs the document, so an outer + * handler that delegates and *then* edits what it delegated would otherwise + * change what executes. What the terminal accepts has to stop being the + * caller's to change. + * + * Structure is copied and frozen; identity is preserved where identity is the + * point. The stream is the operational object a provider selected and a witness + * is held against — cloning it would break provenance — and a modifier factory + * is a function whose identity a registry compares. Props are journal data, so + * they are copied all the way down. An absent optional setting stays absent, so + * the defaults it feeds are unchanged. + */ +function detachOptions(options: ExecuteOptions): ExecuteOptions { + const { props, componentDirs, modifiers, ...rest } = options; + return frozen({ + ...rest, + ...(props === undefined ? {} : { props: detachProps(props) }), + ...(componentDirs === undefined ? {} : { componentDirs: frozen([...componentDirs]) }), + ...(modifiers === undefined ? {} : { modifiers: frozen({ ...modifiers }) }), + }); +} + +/** Frozen at runtime, unchanged to the type system. */ +function frozen(value: T): T { + Object.freeze(value); + return value; +} + +/** + * A frozen copy of one JSON object. + * + * `Object.fromEntries` rather than assignment, because a `__proto__` key + * assigned onto a fresh object rewrites its prototype on some runtimes instead + * of becoming a member. + */ +function detachProps(props: Record): Record { + return frozen( + Object.fromEntries(Object.entries(props).map(([key, value]) => [key, detachJson(value)])), + ); +} + +function detachJson(value: Json): Json { + if (Array.isArray(value)) { + return frozen(value.map((member) => detachJson(member))); + } + if (typeof value === "object" && value !== null) { + return detachProps(value); + } + return value; +} + +function isCanonical(value: unknown): value is CanonicalRequest { + return CanonicalRequest.own(value); +} + +/** One execution's request and what the invocation reads back from it. */ +export interface IssuedExecution { + readonly request: ExecutionRequest; + /** + * Settle this invocation on `request`, or refuse it. + * + * Called only by the invocation's own private terminal — the default handler + * of the same-name Api instance canonical core built for this call. + */ + consume(request: unknown): void; + /** The options the terminal recorded, or a refusal when it was never reached. */ + settle(): ExecuteOptions; + /** Every completion policy registered, in registration order. */ + completions(): readonly CompletionFailure[]; +} + +export function issueExecution(options: ExecuteOptions): IssuedExecution { + const invocation = new Invocation(); + return { + request: new CanonicalRequest(invocation, options, invocation.generation), + consume(request: unknown): void { + CanonicalRequest.consume(request, invocation); + }, + settle(): ExecuteOptions { + const settled = invocation.settled; + if (!invocation.consumed || settled === undefined) { + throw new ExecutionProtocolError("returned without delegating the execution request"); + } + return settled; + }, + completions(): readonly CompletionFailure[] { + return Object.freeze([...invocation.completions]); + }, + }; +} diff --git a/packages/core/tests/execution-protocol.test.ts b/packages/core/tests/execution-protocol.test.ts new file mode 100644 index 00000000..88e7d613 --- /dev/null +++ b/packages/core/tests/execution-protocol.test.ts @@ -0,0 +1,1390 @@ +/** + * Tier EP — the capability-backed execution protocol. + * + * `Execution.execute` middleware is policy. It is handed a request, not an + * execution: it may read the options, narrow them, register an additive + * completion failure, install contextual behavior, refuse, and delegate. What + * it may not do is *complete* an execution — and the tests below are mostly + * about what happens when something tries. + * + * Every refusal is checked the same way: the journal is a real `InMemoryStream` + * and it must be untouched. A protocol failure that still managed to read the + * history, expand the document, or append a Yield or Close would be a refusal + * in name only. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { + createContext, + ensure, + resource, + scoped, + spawn, + suspend, + until, + withResolvers, +} from "effection"; +import { rm, writeTextFile } from "@effectionx/fs"; +import { mkdtemp } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import type { Operation } from "effection"; +import { createApi } from "@effectionx/context-api"; +import type { Api } from "@effectionx/context-api"; +import { DurablePersistenceError, InMemoryStream } from "@executablemd/durable-streams"; +import { FilesInvariantError } from "@executablemd/runtime"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import type { Result } from "effection"; +import { + collect, + execute, + Execution, + ExecutionProtocolError, + inlineSource, + registerComponents, +} from "../mod.ts"; +import type { ExecutionRequest, ModifierFactory } from "../mod.ts"; +import { executeInstalled } from "../host.ts"; +import { executeObserved } from "../src/execute.ts"; +import type { InvocationObservers } from "../src/execute.ts"; +import type { ExecutionInstallation, JournalAdmission } from "../host.ts"; + +const DOC = "# Hello\n"; + +/** A document that fails on its own terms, under error mode. */ +const FAILING_DOC = ["", "", "```bash exec", "exit 3", "```", "", "", ""].join( + "\n", +); + +/** A journal that reports what a refused execution managed to do to it. */ +function watched(): { stream: InMemoryStream; reads: number } { + const stream = new InMemoryStream(); + const watcher = { stream, reads: 0 }; + const readAll = stream.readAll.bind(stream); + stream.readAll = function* (): Operation { + watcher.reads += 1; + return yield* readAll(); + }; + return watcher; +} + +/** + * A directory holding one Markdown component, for this test only. + * + * `mkdtemp` is the one step `@effectionx/fs` has no equivalent for; the write + * and the removal go through it. Cleanup is an `ensure`, so a cancelled test + * cannot strand the directory. + */ +function useComponentFixture(): Operation { + return resource(function* (provide) { + const dir = yield* until(mkdtemp(join(tmpdir(), "xmd-ep26-"))); + yield* ensure(function* () { + yield* rm(dir, { recursive: true, force: true }); + }); + yield* writeTextFile(join(dir, "Accepted.md"), "ACCEPTED-COMPONENT\n"); + yield* provide(dir); + }); +} + +function* raised(operation: Operation): Operation { + try { + yield* operation; + return undefined; + } catch (error) { + return error; + } +} + +/** `` — records that authored work ran. */ +function useMark(expanded: string[]): Operation { + return registerComponents([ + { + name: "Mark", + origin: "tier-ep", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn() { + expanded.push("expanded"); + return ""; + }, + }, + ]); +} + +describe("Tier EP — the execution protocol", () => { + it("EP1: an ordinary execute() with nothing installed still runs the document", function* () { + const output = yield* scoped(function* () { + return yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + expect(String(output)).toContain("Hello"); + }); + + it("EP2: an option transformation reaches the document", function* () { + const output = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + yield* next(request.withOptions({ ...request.options, ...inlineSource("# Replaced\n") })); + }, + }); + return yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + expect(String(output)).toContain("Replaced"); + expect(String(output)).not.toContain("Hello"); + }); + + it("EP3: a handler that answers without delegating is refused, and nothing runs", function* () { + const journal = watched(); + const expanded: string[] = []; + + const failure = yield* scoped(function* () { + yield* useMark(expanded); + yield* Execution.around({ + // deno-lint-ignore require-yield + *execute() { + // Returning a synthetic execution instead of delegating. + return undefined; + }, + }); + return yield* raised(execute({ ...inlineSource("\n"), stream: journal.stream })); + }); + + expect(failure).toBeInstanceOf(ExecutionProtocolError); + expect(journal.reads).toEqual(0); + expect(expanded).toEqual([]); + expect(journal.stream.snapshot()).toEqual([]); + }); + + it("EP4: a substitute return after delegating is ignored", function* () { + // Typed as returning something, which the canonical surface no longer + // permits — so this is written the only way it can still happen: through a + // same-named descriptor whose own types allow it. + const loose: Api<{ execute(request: ExecutionRequest): Operation }> = createApi( + "Execution", + { + // deno-lint-ignore require-yield + *execute(_request: ExecutionRequest): Operation { + return undefined; + }, + }, + ); + + const output = yield* scoped(function* () { + yield* loose.around({ + *execute([request], next) { + yield* next(request); + // Whatever a handler hands back is not an execution. + return { output: undefined, [Symbol.iterator]: () => undefined }; + }, + }); + return yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + expect(String(output)).toContain("Hello"); + }); + + it("EP5: a refusal before delegating performs no read, expansion or append", function* () { + const journal = watched(); + const expanded: string[] = []; + + const failure = yield* scoped(function* () { + yield* useMark(expanded); + yield* Execution.around({ + // deno-lint-ignore require-yield + *execute() { + throw new Error("this policy says no"); + }, + }); + return yield* raised(execute({ ...inlineSource("\n"), stream: journal.stream })); + }); + + expect(failure).toBeInstanceOf(Error); + expect(String(failure)).toContain("this policy says no"); + expect(journal.reads).toEqual(0); + expect(expanded).toEqual([]); + expect(journal.stream.snapshot()).toEqual([]); + }); + + it("EP6: delegating twice fails", function* () { + const failure = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + yield* next(request); + yield* next(request); + }, + }); + return yield* raised(execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + expect(failure).toBeInstanceOf(ExecutionProtocolError); + expect(String(failure)).toContain("more than once"); + }); + + it("EP7: reusing a request a previous execution consumed fails", function* () { + const captured: ExecutionRequest[] = []; + yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + captured.push(request); + yield* next(request); + }, + }); + yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + + const stale = captured[0]; + expect(stale).toBeDefined(); + const failure = yield* scoped(function* () { + yield* Execution.around({ + *execute([,], next) { + // Delegating the *previous* execution's request. + yield* next(stale!); + }, + }); + return yield* raised(execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + expect(failure).toBeInstanceOf(ExecutionProtocolError); + }); + + it("EP8: delegating a reconstructed look-alike fails", function* () { + const journal = watched(); + const failure = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + const lookalike: ExecutionRequest = { + options: request.options, + withOptions: (options) => ({ ...lookalike, options }), + addCompletionFailure: () => {}, + }; + yield* next(lookalike); + }, + }); + return yield* raised(execute({ ...inlineSource(DOC), stream: journal.stream })); + }); + expect(failure).toBeInstanceOf(ExecutionProtocolError); + // The look-alike itself is what is refused — not merely "nothing reached + // the terminal", which a silently ignored look-alike would also produce. + expect(String(failure)).toContain("did not issue"); + expect(journal.reads).toEqual(0); + expect(journal.stream.snapshot()).toEqual([]); + }); + + it("EP9: delegating a request a later withOptions() superseded fails", function* () { + const failure = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + request.withOptions({ ...request.options, ...inlineSource("# Other\n") }); + // The superseded request, not the one just derived. + yield* next(request); + }, + }); + return yield* raised(execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + expect(failure).toBeInstanceOf(ExecutionProtocolError); + expect(String(failure)).toContain("superseded"); + }); + + it("EP10: middleware from another loaded copy inspects, transforms and delegates", function* () { + const seen: string[] = []; + // A descriptor of the same stable name, built here rather than imported — + // which is what a separately loaded copy of core is. + const foreign: Api<{ + execute(request: ExecutionRequest): Operation; + }> = createApi("Execution", { + // deno-lint-ignore require-yield + *execute(_request: ExecutionRequest): Operation {}, + }); + + const output = yield* scoped(function* () { + yield* foreign.around({ + *execute([request], next) { + seen.push(typeof request.options.stream); + yield* next(request.withOptions({ ...request.options, ...inlineSource("# Foreign\n") })); + }, + }); + return yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + + expect(seen).toEqual(["object"]); + expect(String(output)).toContain("Foreign"); + }); + + // EP19: a nested invocation cannot settle its caller. The outer request is + // still live and unconsumed when the nested chain hands it to the nested + // terminal, so what refuses it is the exact-invocation comparison. + it("EP19: a nested invocation cannot delegate its caller's live request", function* () { + let refusal: unknown; + const expanded: string[] = []; + let outer: ExecutionRequest | undefined; + const nested = new InMemoryStream(); + const outer_ = new InMemoryStream(); + let depth = 0; + + const output = yield* scoped(function* () { + yield* useMark(expanded); + yield* Execution.around({ + *execute([request], next) { + const mine = depth++; + if (mine === 0) { + outer = request; + // Started while this invocation is still live and unconsumed. + yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + yield* next(request); + return; + } + // The nested invocation, handing its own terminal the caller's request. + const captured = outer; + expect(captured).toBeDefined(); + refusal = captured === undefined ? undefined : yield* raised(next(captured)); + yield* next(request); + }, + }); + return yield* collect( + yield* execute({ ...inlineSource("\n\n# Outer\n"), stream: outer_ }), + ); + }); + + expect(refusal).toBeInstanceOf(ExecutionProtocolError); + expect(String(refusal)).toContain("another execution issued"); + expect(refusal instanceof Error ? refusal.cause : "none").toBeUndefined(); + // The outer invocation settled on its own request afterwards: its document + // expanded and its own journal — not the nested one — carries it. That two + // live invocations each run their own document is EP20's claim, proved + // there with barriers rather than through nesting. + expect(String(output)).toContain("Outer"); + expect(String(output)).not.toContain("Nested"); + expect(JSON.stringify(outer_.snapshot())).toContain("Outer"); + expect(JSON.stringify(nested.snapshot())).not.toContain("Outer"); + expect(expanded).toEqual(["expanded"]); + }); + + // EP20: two invocations, both live and both unconsumed at the moment each + // tries to delegate the other's request. The barrier is what makes that true + // — without it one could already be consumed, and a consumed-request check + // alone would satisfy the assertions. + it("EP20: concurrent invocations cannot swap live requests", function* () { + const arrived = [withResolvers(), withResolvers()]; + const captured: Array = [undefined, undefined]; + const refusals: unknown[] = []; + let index = 0; + + const outputs = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + const mine = index++; + captured[mine] = request; + arrived[mine]!.resolve(); + // Both requests exist and neither has reached a terminal yet. + yield* arrived[0]!.operation; + yield* arrived[1]!.operation; + + const foreign = captured[mine === 0 ? 1 : 0]; + refusals.push(yield* raised(next(foreign!))); + // Its own request still settles this invocation afterwards. + yield* next(request); + }, + }); + + const run = (doc: string) => + function* (): Operation { + return String( + yield* collect(yield* execute({ ...inlineSource(doc), stream: new InMemoryStream() })), + ); + }; + const first = yield* spawn(run("# First\n")); + const second = yield* spawn(run("# Second\n")); + return [yield* first, yield* second]; + }); + + expect(refusals).toHaveLength(2); + for (const refusal of refusals) { + expect(refusal).toBeInstanceOf(ExecutionProtocolError); + expect(String(refusal)).toContain("another execution issued"); + expect(refusal instanceof Error ? refusal.cause : "none").toBeUndefined(); + } + // Each still ran its own document under its own options. + expect(outputs[0]).toContain("First"); + expect(outputs[1]).toContain("Second"); + }); + + it("EP21: the exported default refuses and consumes nothing", function* () { + const captured: ExecutionRequest[] = []; + const output = yield* scoped(function* () { + let reentered = false; + yield* Execution.around({ + *execute([request], next) { + // The standalone call below re-enters this same handler, because the + // public descriptor shares the stable name. Let it pass straight + // through so what is under test is the default it reaches. + if (reentered) { + yield* next(request); + return; + } + captured.push(request); + reentered = true; + const standalone = yield* raised(Execution.operations.execute(request)); + reentered = false; + expect(standalone).toBeInstanceOf(ExecutionProtocolError); + expect(String(standalone)).toContain("outside canonical core"); + // The request survived that, and still settles this invocation. + yield* next(request); + }, + }); + return yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + + expect(String(output)).toContain("Hello"); + expect(captured).toHaveLength(1); + }); + + it("EP22: null, primitives and hostile shapes are refused without a native error", function* () { + const hostile = new Proxy( + {}, + { + has() { + throw new Error("PLANTED-HAS-TRAP"); + }, + get() { + throw new Error("PLANTED-GET-TRAP"); + }, + }, + ); + const values: unknown[] = [null, undefined, 7, "request", true, Symbol("r"), {}, hostile]; + + for (const value of values) { + const journal = watched(); + const failure = yield* scoped(function* () { + yield* Execution.around({ + *execute([,], next) { + yield* next(value as ExecutionRequest); + }, + }); + return yield* raised(execute({ ...inlineSource(DOC), stream: journal.stream })); + }); + + expect(failure).toBeInstanceOf(ExecutionProtocolError); + expect(String(failure)).not.toContain("PLANTED"); + expect(failure instanceof Error ? failure.cause : "none").toBeUndefined(); + expect(journal.reads).toEqual(0); + expect(journal.stream.snapshot()).toEqual([]); + } + }); + + // EP23: contextual behavior an installation establishes is the invocation's. + // It has to reach document teardown, stay out of a concurrent invocation, and + // be gone from the next ordinary execution in the same host scope. + it("EP23: invocation-installed context reaches teardown and leaks nowhere", function* () { + const Marker = createContext("tier-ep.marker", undefined); + const seen: string[] = []; + + yield* scoped(function* () { + yield* Execution.around({ + *document([props], next) { + seen.push(`document:${(yield* Marker.get()) ?? "absent"}`); + try { + return yield* next(props); + } finally { + seen.push("teardown"); + } + }, + }); + + yield* collect( + yield* executeInstalled({ ...inlineSource(DOC), stream: new InMemoryStream() }, [ + { + *install(): Operation { + yield* Marker.set("installed"); + }, + }, + ]), + ); + + // A later ordinary execution in the same host scope must not inherit it. + yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + + expect(seen).toEqual(["document:installed", "teardown", "document:absent", "teardown"]); + }); + + // EP24: settlement owns the invocation scope. A caller that continues on the + // completion continues after this invocation's cleanup has finished — which a + // caller-owned resource would not give, since it would still be standing. + it("EP24: cleanup finishes before completion is observed", function* () { + const cases: Array<{ says: string; doc: string; policy: boolean }> = [ + { says: "success", doc: DOC, policy: false }, + { says: "document failure", doc: FAILING_DOC, policy: false }, + { says: "completion-policy failure", doc: DOC, policy: true }, + ]; + + for (const scenario of cases) { + const finalized: string[] = []; + const observed: string[] = []; + + yield* scoped(function* () { + if (scenario.policy) { + yield* Execution.around({ + *execute([request], next) { + request.addCompletionFailure(() => new Error("the policy failed it")); + yield* next(request); + }, + }); + } + + const execution = yield* executeInstalled( + { ...inlineSource(scenario.doc), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(() => { + finalized.push("cleanup"); + }); + }, + }, + ], + ); + yield* execution; + observed.push(`completion:${finalized.length}`); + }); + + // Exactly once, and already done when the completion was read. + expect(finalized).toEqual(["cleanup"]); + expect(observed).toEqual(["completion:1"]); + } + }); + + it("EP24b: concurrent invocations finalize independently, and leave nothing behind", function* () { + const finalized: string[] = []; + + yield* scoped(function* () { + const run = (name: string) => + function* (): Operation { + const execution = yield* executeInstalled( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(() => { + finalized.push(name); + }); + }, + }, + ], + ); + yield* execution; + }; + const first = yield* spawn(run("first")); + const second = yield* spawn(run("second")); + yield* first; + yield* second; + }); + + expect([...finalized].sort()).toEqual(["first", "second"]); + }); + + it("EP24c: a completed handle can be re-observed without refinalizing", function* () { + const finalized: string[] = []; + + const results = yield* scoped(function* () { + const execution = yield* executeInstalled( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(() => { + finalized.push("cleanup"); + }); + }, + }, + ], + ); + const first = yield* execution; + const second = yield* execution; + // Late subscription still replays the whole output. + const late = yield* collect(execution); + return [first.ok, second.ok, String(late).includes("Hello")]; + }); + + expect(results).toEqual([true, true, true]); + expect(finalized).toEqual(["cleanup"]); + }); + + // EP25: cancelling a consumer of an *already-returned* handle cancels the + // invocation. The handle is obtained in a caller scope that keeps running, and + // the task that is halted never started the execution — nothing but the handle + // connects them. + it("EP25: halting a consumer of a returned handle cancels the invocation", function* () { + const finalized: string[] = []; + const halted: string[] = []; + const reached = withResolvers(); + const settledBy: Array = []; + + yield* scoped(function* () { + yield* Execution.around({ + *document([props], next) { + try { + reached.resolve(); + yield* suspend(); + return yield* next(props); + } finally { + halted.push("document"); + } + }, + }); + + const observing = withResolvers(); + const execution = yield* executeObserved( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(() => { + finalized.push("cleanup"); + }); + }, + }, + ], + { observed: () => observing.resolve() }, + ); + yield* reached.operation; + + // The consumer says when it is about to observe the handle, so the halt + // lands on a task that is *inside* the observation rather than on one + // that never started. A delay would only make that likely. + // The acknowledgement comes from inside the observation itself: the + // handle notifies once the consumer is cancellable. No elapsed time, and + // a halt delivered earlier would leave `halted` empty. + const consumer = yield* spawn(function* () { + yield* execution; + }); + yield* observing.operation; + yield* consumer.halt(); + + // Asserted before this scope exits: halting the consumer is what closed + // the invocation, not the surrounding scope unwinding. + expect(halted).toEqual(["document"]); + expect(finalized).toEqual(["cleanup"]); + + // Another observer of the same handle settles rather than hanging... + settledBy.push((yield* execution).ok); + // ...and observing it again starts nothing and refinalizes nothing. + settledBy.push((yield* execution).ok); + }); + + expect(settledBy).toEqual([false, false]); + expect(halted).toEqual(["document"]); + expect(finalized).toEqual(["cleanup"]); + }); + + it("EP25b: a fatal document result survives cancellation by identity", function* () { + const durable = new DurablePersistenceError("yield", new Error("planted")); + const released = withResolvers(); + const enteredTeardown = withResolvers(); + const finalized: string[] = []; + + const observed = yield* scoped(function* () { + yield* Execution.around({ + // deno-lint-ignore require-yield + *document() { + throw durable; + }, + }); + + const execution = yield* executeInstalled( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(function* () { + finalized.push("cleanup"); + enteredTeardown.resolve(); + // Held open until the test says so, so cancellation lands while + // teardown is genuinely in progress. + yield* released.operation; + }); + }, + }, + ], + ); + + const observing = withResolvers(); + const consumer = yield* spawn(function* () { + observing.resolve(); + yield* execution; + }); + yield* observing.operation; + // The document has already failed, so teardown is what is running now. + yield* enteredTeardown.operation; + const halting = yield* spawn(function* () { + yield* consumer.halt(); + }); + released.resolve(); + yield* halting; + + return yield* execution; + }); + + // The document had already decided, and cancelling a consumer during + // teardown did not erase that — by identity. + expect(observed.ok).toBe(false); + expect(observed.ok ? undefined : observed.error).toBe(durable); + expect(finalized).toEqual(["cleanup"]); + }); + + // EP25c: §8.1's other cancellation promise — a fatal failure raised *while* + // cancellation tears the invocation down is ranked and returned by identity, + // rather than being swallowed by the cancellation result. + it("EP25c: a fatal failure raised by cancellation teardown wins by identity", function* () { + const durable = new DurablePersistenceError("close", new Error("planted-teardown")); + const enteredTeardown = withResolvers(); + const released = withResolvers(); + const reached = withResolvers(); + const finalized: string[] = []; + + const observed = yield* scoped(function* () { + yield* Execution.around({ + *document([props], next) { + reached.resolve(); + yield* suspend(); + return yield* next(props); + }, + }); + + const observing = withResolvers(); + const execution = yield* executeObserved( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(function* () { + finalized.push("cleanup"); + enteredTeardown.resolve(); + yield* released.operation; + throw durable; + }); + }, + }, + ], + { observed: () => observing.resolve() }, + ); + yield* reached.operation; + + const consumer = yield* spawn(function* () { + yield* execution; + }); + yield* observing.operation; + const halting = yield* spawn(function* () { + yield* consumer.halt(); + }); + // Cancellation is now inside teardown; let the finalizer raise. + yield* enteredTeardown.operation; + released.resolve(); + yield* halting; + + const first = yield* execution; + // Again: nothing restarts, nothing refinalizes. + const second = yield* execution; + return [first, second]; + }); + + for (const result of observed) { + expect(result.ok).toBe(false); + expect(result.ok ? undefined : result.error).toBe(durable); + } + expect(finalized).toEqual(["cleanup"]); + }); + + // EP26: what the terminal accepted stops being the caller's to change. Each + // field below is mutated *after* the private terminal returned and before the + // public chain finishes, and each one materially affects execution — so the + // accepted snapshot is what the assertions read, not the edited original. + it("EP26: the accepted options are a detached snapshot", function* () { + // Control: replacing the same fields *before* delegating still works. + const replaced = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + yield* next(request.withOptions({ ...request.options, ...inlineSource("# Control\n") })); + }, + }); + return yield* collect(yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() })); + }); + expect(String(replaced)).toContain("Control"); + + const accepted = new InMemoryStream(); + const smuggled = new InMemoryStream(); + const acceptedModifiers: Record = { + shout: (_params) => (_args, next) => + (function* () { + const inner = yield* next(); + return { ...inner, output: `SHOUTED:${inner.output}` }; + })(), + }; + + // A component reachable only through the accepted directory. + const fixture = yield* useComponentFixture(); + const acceptedDirs = [fixture]; + + const source = [ + "---", + "props:", + " type: object", + " properties:", + " nested:", + " type: object", + " properties:", + " value: { type: string }", + " required: [value]", + " additionalProperties: false", + " items:", + " type: array", + " items: { type: string }", + " required: [nested, items]", + " additionalProperties: false", + "---", + "", + "", + "", + "nested={props.nested.value} items={props.items}", + "", + "```bash shout exec", + "echo hi", + "```", + "", + ].join("\n"); + + const nested = { value: "accepted-value" }; + const items = ["accepted-item"]; + + const output = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + const options = request.options; + yield* next(request); + + // Every one of these is the caller's own object, edited after the + // terminal accepted a copy of it. + Reflect.set(options, "stream", smuggled); + Reflect.set(options, "path", "/nowhere/replaced.md"); + const dirs = options.componentDirs; + if (Array.isArray(dirs)) { + Reflect.set(dirs, 0, "/nowhere"); + Reflect.set(dirs, "length", 0); + } + const modifiers = options.modifiers; + if (typeof modifiers === "object" && modifiers !== null) { + Reflect.deleteProperty(modifiers, "shout"); + Reflect.set(modifiers, "shout", () => () => "REPLACED"); + } + // The caller's own nested values, edited after acceptance. + Reflect.set(nested, "value", "smuggled-value"); + Reflect.set(items, 0, "smuggled-item"); + }, + }); + + return yield* collect( + yield* execute({ + ...inlineSource(source), + stream: accepted, + componentDirs: acceptedDirs, + modifiers: acceptedModifiers, + props: { nested, items }, + }), + ); + }); + + // The accepted modifier ran, by identity — not the replacement. + expect(String(output)).toContain("SHOUTED:"); + expect(String(output)).not.toContain("REPLACED"); + // Events went to the accepted stream, and the substituted one saw nothing. + expect(accepted.snapshot().length).toBeGreaterThan(0); + expect(smuggled.snapshot()).toEqual([]); + // The component resolved through the accepted directory, even though the + // caller's array was emptied after acceptance. + expect(String(output)).toContain("ACCEPTED-COMPONENT"); + // Nested props: the accepted object and array, not the edited ones. + expect(String(output)).toContain("accepted-value"); + expect(String(output)).not.toContain("smuggled-value"); + expect(String(output)).toContain("accepted-item"); + expect(String(output)).not.toContain("smuggled-item"); + // The caller's own arrays and records really were edited — the snapshot is + // what protected the execution, not an absence of mutation. + expect(acceptedDirs).toEqual([]); + expect(nested.value).toEqual("smuggled-value"); + expect(items[0]).toEqual("smuggled-item"); + expect(Object.keys(acceptedModifiers)).toEqual(["shout"]); + }); + + // EP27: a document outcome and an invocation-teardown failure are ranked, not + // replaced. A fatal failure is reported by identity from wherever it came. + it("EP27: teardown reconciles with the document outcome by precedence", function* () { + const durable = new DurablePersistenceError("yield", new Error("planted")); + const otherDurable = new DurablePersistenceError("close", new Error("planted-second")); + const filesFatal = new FilesInvariantError("protocol"); + const otherFilesFatal = new FilesInvariantError("protocol"); + const cleanup = new Error("INSTALLATION-CLEANUP"); + + const cases: Array<{ + says: string; + fail?: Error; + teardown?: Error; + expected: (result: Result) => void; + }> = [ + { + says: "durability failure outranks an ordinary cleanup error", + fail: durable, + teardown: cleanup, + expected: (result) => { + expect(result.ok).toBe(false); + // By identity: the engine's fences match the exact object. + expect(result.ok ? undefined : result.error).toBe(durable); + }, + }, + { + says: "an ordinary document failure stays authoritative", + fail: new Error("DOCUMENT-FAILED"), + teardown: cleanup, + expected: (result) => { + expect(String(result.ok ? "" : result.error.message)).toContain("DOCUMENT-FAILED"); + expect(String(result.ok ? "" : result.error.message)).not.toContain( + "INSTALLATION-CLEANUP", + ); + }, + }, + { + says: "a cleanup error converts a successful document", + teardown: cleanup, + expected: (result) => { + expect(result.ok).toBe(false); + expect(result.ok ? undefined : result.error).toBe(cleanup); + }, + }, + { + says: "a fatal teardown outranks an ordinary document failure", + fail: new Error("DOCUMENT-FAILED"), + teardown: durable, + expected: (result) => { + expect(result.ok ? undefined : result.error).toBe(durable); + }, + }, + { + says: "the document's durability failure precedes the teardown's", + fail: durable, + teardown: otherDurable, + expected: (result) => { + // Same kind on both sides: the one that happened first wins, and it + // is the exact object rather than a rebuilt one. + expect(result.ok ? undefined : result.error).toBe(durable); + }, + }, + { + says: "the document's Files failure precedes the teardown's", + fail: filesFatal, + teardown: otherFilesFatal, + expected: (result) => { + expect(result.ok ? undefined : result.error).toBe(filesFatal); + }, + }, + { + says: "durability outranks Files wherever each came from", + fail: filesFatal, + teardown: durable, + expected: (result) => { + expect(result.ok ? undefined : result.error).toBe(durable); + }, + }, + { + says: "a document durability failure outranks a Files teardown", + fail: durable, + teardown: filesFatal, + expected: (result) => { + expect(result.ok ? undefined : result.error).toBe(durable); + }, + }, + ]; + + for (const scenario of cases) { + const finalized: string[] = []; + const observed: number[] = []; + + const result = yield* scoped(function* () { + if (scenario.fail) { + const failure = scenario.fail; + yield* Execution.around({ + // deno-lint-ignore require-yield + *document() { + throw failure; + }, + }); + } + const execution = yield* executeInstalled( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(() => { + finalized.push("cleanup"); + if (scenario.teardown) { + throw scenario.teardown; + } + }); + }, + }, + ], + ); + const settled = yield* execution; + observed.push(finalized.length); + return settled; + }); + + scenario.expected(result); + // Every finalizer ran, exactly once, before the result was observable. + expect(finalized).toEqual(["cleanup"]); + expect(observed).toEqual([1]); + } + }); + + // EP28: an observer belongs to the invocation it watches. Two executions run + // concurrently, each with its own callback and its own finalizer; cancelling + // one consumer must reach only that invocation. A single shared slot would + // let one execution receive or overwrite the other's notification. + it("EP28: observers, cancellation and cleanup never cross invocations", function* () { + const seen: string[] = []; + const finalized: string[] = []; + const halted: string[] = []; + const reachedFirst = withResolvers(); + const reachedSecond = withResolvers(); + const observingFirst = withResolvers(); + const observingSecond = withResolvers(); + + // A descriptor the caller keeps and edits after the invocation started: + // only the callback it held at the start may run. + yield* scoped(function* () { + const retained: InvocationObservers = { observed: () => seen.push("observed:A") }; + const replaced = yield* executeObserved( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [], + retained, + ); + retained.observed = () => seen.push("observed:B"); + yield* replaced; + }); + + const settledSecond = yield* scoped(function* () { + yield* Execution.around({ + *document([props], next) { + const which = seen.includes("start:first") ? "second" : "first"; + seen.push(`start:${which}`); + if (which === "first") { + reachedFirst.resolve(); + try { + yield* suspend(); + } finally { + halted.push("first"); + } + } + reachedSecond.resolve(); + return yield* next(props); + }, + }); + + const start = (name: string, observing: { resolve(): void }) => + executeObserved( + { ...inlineSource(DOC), stream: new InMemoryStream() }, + [ + { + *install(): Operation { + yield* ensure(() => { + finalized.push(name); + }); + }, + }, + ], + { + observed: () => { + seen.push(`observed:${name}`); + observing.resolve(); + }, + }, + ); + + const first = yield* start("first", observingFirst); + yield* reachedFirst.operation; + const second = yield* start("second", observingSecond); + + const firstConsumer = yield* spawn(function* () { + yield* first; + }); + const secondConsumer = yield* spawn(function* () { + return yield* second; + }); + yield* observingFirst.operation; + yield* observingSecond.operation; + + // Only the first invocation's consumer is cancelled. + yield* firstConsumer.halt(); + + expect(halted).toEqual(["first"]); + expect(finalized).toEqual(["first"]); + + // The second invocation is untouched and settles on its own. + return yield* secondConsumer; + }); + + // Each callback fired for its own invocation, once — and the invocation + // started with A ran A, not the B its caller substituted afterwards. + expect(seen.filter((entry) => entry.startsWith("observed:")).sort()).toEqual([ + "observed:A", + "observed:first", + "observed:second", + ]); + expect(settledSecond.ok).toBe(true); + expect(halted).toEqual(["first"]); + expect(finalized).toEqual(["first", "second"]); + }); + + it("EP11: admissions are copied before install() runs", function* () { + const order: string[] = []; + const late: JournalAdmission[] = []; + const installation: ExecutionInstallation = { + admissions: late, + *install(): Operation { + order.push("install"); + // Too late: the collection was copied before this ran. + // deno-lint-ignore require-yield + late.push(function* () { + order.push("late-admission"); + }); + }, + }; + + yield* scoped(function* () { + yield* collect( + yield* executeInstalled({ ...inlineSource(DOC), stream: new InMemoryStream() }, [ + installation, + ]), + ); + }); + + expect(order).toEqual(["install"]); + }); + + it("EP12: every captured admission runs, in order, on the retained history", function* () { + const order: string[] = []; + const seen: number[] = []; + const admission = (name: string): JournalAdmission => + // deno-lint-ignore require-yield + function* (retained) { + order.push(name); + seen.push(retained.length); + }; + + yield* scoped(function* () { + yield* collect( + yield* executeInstalled({ ...inlineSource(DOC), stream: new InMemoryStream() }, [ + { admissions: [admission("first")] }, + { admissions: [admission("second")] }, + ]), + ); + }); + + expect(order).toEqual(["first", "second"]); + expect(seen).toEqual([0, 0]); + }); + + it("EP13: one admission refusal stops everything after it", function* () { + for (const refusing of [0, 1]) { + const journal = watched(); + const expanded: string[] = []; + const ran: string[] = []; + const admissions: JournalAdmission[] = [0, 1].map( + (index) => + // deno-lint-ignore require-yield + function* () { + ran.push(`admission-${index}`); + if (index === refusing) { + throw new Error(`admission ${index} says no`); + } + }, + ); + + const failure = yield* scoped(function* () { + yield* useMark(expanded); + return yield* raised( + collect( + yield* executeInstalled({ ...inlineSource("\n"), stream: journal.stream }, [ + { admissions: [admissions[0]!] }, + { admissions: [admissions[1]!] }, + ]), + ), + ); + }); + + expect(String(failure)).toContain(`admission ${refusing} says no`); + // Nothing after the refusal: no expansion, no Yield, no Close. + expect(expanded).toEqual([]); + expect(journal.stream.snapshot()).toEqual([]); + expect(ran).toEqual(refusing === 0 ? ["admission-0"] : ["admission-0", "admission-1"]); + } + }); + + // EP15: additive means one direction. A policy can fail a success; it cannot + // stand in for a failure the document already earned. + // EP18: an admission inspects or refuses. It does not edit. `readonly` is a + // compile-time claim, so each attempt below is made at runtime through the + // reflective route that ignores it — and every one has to be ineffective or + // throw, with the next admission still seeing the original history. + it("EP18: an admission cannot change the history anyone else reads", function* () { + const attempts: Array<{ says: string; edit: (retained: readonly DurableEvent[]) => void }> = [ + { says: "length", edit: (retained) => void Reflect.set(retained, "length", 0) }, + { says: "indexed replacement", edit: (retained) => void Reflect.set(retained, 0, undefined) }, + { says: "deletion", edit: (retained) => void Reflect.deleteProperty(retained, 0) }, + { says: "reverse", edit: (retained) => void [...[]].reverse.call(retained) }, + { says: "splice", edit: (retained) => void [...[]].splice.call(retained, 0, 99) }, + ]; + + // One completed journal, replayed once per attempt. + const first = new InMemoryStream(); + yield* scoped(function* () { + yield* collect(yield* execute({ ...inlineSource(DOC), stream: first })); + }); + const original = first.snapshot().length; + expect(original).toBeGreaterThan(0); + + for (const attempt of attempts) { + const journal = new InMemoryStream(first.snapshot()); + const observed: number[] = []; + const expanded: string[] = []; + + const output = yield* scoped(function* () { + yield* useMark(expanded); + return yield* collect( + yield* executeInstalled({ ...inlineSource(DOC), stream: journal }, [ + { + admissions: [ + // deno-lint-ignore require-yield + function* (retained) { + observed.push(retained.length); + // Ineffective or throwing — either is a refusal to edit. + try { + attempt.edit(retained); + } catch { + // A frozen array throws in strict mode; that is the point. + } + }, + // deno-lint-ignore require-yield + function* (retained) { + observed.push(retained.length); + }, + ], + }, + ]), + ); + }); + + // The second admission saw exactly what the first was given. + expect(observed).toEqual([original, original]); + // Still a completed replay: nothing authored ran and nothing was appended. + expect(String(output)).toContain("Hello"); + expect(expanded).toEqual([]); + expect(journal.snapshot().length).toEqual(original); + } + }); + + it("EP15: a completion policy cannot replace an existing failure", function* () { + const asked: string[] = []; + const result = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + request.addCompletionFailure(() => { + asked.push("policy"); + return new Error("the policy's failure"); + }); + yield* next(request); + }, + }); + // A value root that declares `returns` and produces no fails on + // its own terms. + return yield* yield* execute({ + ...inlineSource("---\nreturns:\n type: object\n---\n\nbody\n"), + stream: new InMemoryStream(), + }); + }); + + expect(result.ok).toBe(false); + const message = result.ok ? "" : result.error.message; + expect(message).not.toContain("the policy's failure"); + // Not even consulted: the result was already a failure. + expect(asked).toEqual([]); + }); + + it("EP16: an additive failure still converts a success", function* () { + const result = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + request.addCompletionFailure(() => new Error("the policy's failure")); + yield* next(request); + }, + }); + return yield* yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() }); + }); + + expect(result.ok).toBe(false); + expect(result.ok ? "" : result.error.message).toContain("the policy's failure"); + }); + + it("EP17: the first policy to fail wins, and later ones do not replace it", function* () { + const asked: string[] = []; + const result = yield* scoped(function* () { + yield* Execution.around({ + *execute([request], next) { + request.addCompletionFailure(() => { + asked.push("first"); + return new Error("the first policy"); + }); + request.addCompletionFailure(() => { + asked.push("second"); + return new Error("the second policy"); + }); + yield* next(request); + }, + }); + return yield* yield* execute({ ...inlineSource(DOC), stream: new InMemoryStream() }); + }); + + expect(result.ok).toBe(false); + const message = result.ok ? "" : result.error.message; + expect(message).toContain("the first policy"); + expect(message).not.toContain("the second policy"); + // The second is never even consulted once the result is a failure. + expect(asked).toEqual(["first"]); + }); + + it("EP14: the obsolete ambient admission channel does not exist", function* () { + const ran: string[] = []; + // The exact channel a previous revision used, rebuilt by name and cleared — + // before the invocation and again from inside the middleware chain. There + // is nothing behind the name now, so neither clearing removes anything. + const obsolete = createContext("executablemd.core.journal-admission", undefined); + + yield* scoped(function* () { + yield* obsolete.set([]); + yield* Execution.around({ + *execute([request], next) { + yield* obsolete.set([]); + yield* next(request); + }, + }); + yield* collect( + yield* executeInstalled({ ...inlineSource(DOC), stream: new InMemoryStream() }, [ + { + admissions: [ + // deno-lint-ignore require-yield + function* () { + ran.push("still-ran"); + }, + ], + }, + ]), + ); + }); + + expect(ran).toEqual(["still-ran"]); + }); +}); diff --git a/packages/core/tests/root-provider.test.ts b/packages/core/tests/root-provider.test.ts index 72ed2bb4..3e8930fb 100644 --- a/packages/core/tests/root-provider.test.ts +++ b/packages/core/tests/root-provider.test.ts @@ -167,6 +167,33 @@ describe("Tier RP — root-provider lifecycle", () => { } }); + // RP5: first-failure precedence, where it is easiest to get wrong. The + // document fails on its own terms and cleanup fails too. Both happened; what + // the caller receives is the failure the document earned, because a completion + // policy adds and never replaces. + it("RP5: a failed document keeps its own failure even when cleanup also fails", function* () { + const boom = new Error("teardown boom"); + const { factory, state } = createProvider({ teardownError: boom, extraFinalizer: true }); + yield* installAgentComponents({ rootProvider: { factory, options: OPTIONS } }); + + // The document fails on its own terms — a failing command inside , + // where an error fails the execution rather than printing — with no prompt + // having failed. + const { result } = yield* runDoc( + ["", "", "```bash exec", "exit 3", "```", "", "", ""].join("\n"), + ); + + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.error).not.toBe(boom); + expect(result.error).not.toBeInstanceOf(AggregateError); + expect(result.error.message).not.toContain("teardown boom"); + } + // Cleanup still ran — both finalizers. + expect(state.extraRan).toBe(true); + expect(state.cleanupDone).toBe(true); + }); + it("RP3: a prompt failure plus a teardown failure aggregate — primary first, every cleanup runs", function* () { const boom = new Error("teardown boom"); const { factory, state } = createProvider({ diff --git a/packages/testing/src/components.ts b/packages/testing/src/components.ts index 619625c3..2b8c375e 100644 --- a/packages/testing/src/components.ts +++ b/packages/testing/src/components.ts @@ -117,7 +117,7 @@ export function* installHandlers( ]; yield* registerComponents(registrations); yield* Execution.around({ - *execute([executeOptions], next) { + *execute([request], next) { // Fresh boundary collection per execution: outcomes reported by // explicit elements in THIS run decide this run's Result. const boundaries: BoundaryOutcome[] = []; @@ -135,7 +135,7 @@ export function* installHandlers( // live or partial journal (no root Close) hydrates nothing; // re-expansion records each result exactly once via its durable // operation. - const replayed = yield* readCompletedRun(executeOptions.stream); + const replayed = yield* readCompletedRun(request.options.stream); if (replayed) { for (const result of replayed.results) { yield* record(result); @@ -144,8 +144,7 @@ export function* installHandlers( yield* boundary(outcome); } } - const inner = yield* next(executeOptions); - return decorateCompletion(inner, () => { + request.addCompletionFailure(() => { const failed = boundaries.filter((b) => b.failed > 0); if (failed.length > 0) { return new TestFailureError( @@ -157,35 +156,11 @@ export function* installHandlers( } return undefined; }); + yield* next(request); }, }); } -/** - * Map an execution's completion: an `Ok` becomes `Err(failure())` when the - * policy reports one, after the inner completion — and therefore its closed - * output stream — settles. An existing `Err` passes through unchanged. - */ -export function decorateCompletion( - inner: DocumentExecution, - failure: () => Error | undefined, -): DocumentExecution { - return { - output: inner.output, - *[Symbol.iterator]() { - const result = yield* inner; - if (!result.ok) { - return result; - } - const error = failure(); - if (error) { - return Err(error); - } - return result; - }, - }; -} - /** Whether a failure is, or wraps, a printed error an enclosing test intercepted. */ function carriesRaisedSegment(error: unknown, seen = new Set()): boolean { if (error instanceof RaisedSegmentError) { diff --git a/packages/testing/src/use-testing.ts b/packages/testing/src/use-testing.ts index dc600464..517931e2 100644 --- a/packages/testing/src/use-testing.ts +++ b/packages/testing/src/use-testing.ts @@ -19,7 +19,7 @@ import type { Operation } from "effection"; import { Execution } from "@executablemd/core"; import { sessionActive, Test, TestFailureError } from "./test-api.ts"; import type { TestResult } from "./test-api.ts"; -import { decorateCompletion, installTestingComponents } from "./components.ts"; +import { installTestingComponents } from "./components.ts"; import { flushStaged } from "./test-component.ts"; export interface Testing { @@ -79,15 +79,17 @@ export function* useTesting(options?: { verbose?: boolean }): Operation // BEFORE a handle exists — the pre-handle throw path. let executed = false; yield* Execution.around({ - *execute([executeOptions], next) { + *execute([request], next) { + // Refused before delegating: no execution is issued, which is what makes + // the second call fail rather than produce a document whose outcomes are + // the first call's. if (executed) { throw new Error( "a useTesting() session supports one execute() call — start a new session for another document", ); } executed = true; - const inner = yield* next(executeOptions); - return decorateCompletion(inner, () => { + request.addCompletionFailure(() => { if (collected.length === 0) { return new TestFailureError("no tests were discovered"); } @@ -105,6 +107,7 @@ export function* useTesting(options?: { verbose?: boolean }): Operation } return undefined; }); + yield* next(request); }, }); diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index e2b3165a..09b04fca 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -42,14 +42,25 @@ interface AgentProviderOptions { defaultAgent: string; permissionMode: Permissio ``` `installAgentComponents({ rootProvider: { factory, options } })` owns the root -provider's lifetime as part of each `DocumentExecution`: the factory runs inside -a scoped provider lifetime, the document renders through it, and the completion -resolves **only after the provider's finalizers have run**. Rendered output -closes independently of teardown. A provider-teardown failure folds into the -completion: an otherwise-successful run becomes an `Err`, and when the document -or its prompts already failed the teardown error joins them in an -`AggregateError` (primary errors first). Every finalizer runs even if one -throws. +provider's lifetime through `Execution.document`: the factory runs inside a +scoped provider lifetime that surrounds the document's expansion and ends while +the journal is still live, so the completion resolves **only after the +provider's finalizers have run**. Rendered output closes independently of +teardown, and every finalizer runs even if one throws. + +Teardown and prompt failures are *additive completion policies*, and completion +precedence is first-failure: + +- a document that already failed keeps its own failure — provider teardown + neither replaces it nor aggregates onto it, and the policy is not consulted; +- an otherwise-successful document becomes an `Err` when its prompts failed, + when teardown failed, or both — prompt failures first, then teardown, flat + rather than nested; and +- once a policy has failed the completion, no later policy replaces it. + +Cleanup still runs in every case. "The document failed" and "cleanup also +failed" are both true; what the caller receives is the document's own failure, +because that is the one it earned. ### The provider registry diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index e23c2f42..2c043a3a 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -6251,12 +6251,73 @@ execution, and the root eval scope (§5.5), and the output→stream bridge — so nothing leaks onto the caller's scope and the whole run inherits them contextually. -`execute` is delivered through the `Execution` context Api. The default -provider runs the document; extensions decorate the execution lifecycle -with `Execution.around({ execute })` middleware — observing options, -wrapping the returned handle, or mapping its completion `Result` — without -introducing another execution function. Core itself has no knowledge of -any particular extension. +`execute` is delivered through the `Execution` context Api, and the Api is a +**policy** surface rather than an execution one. A handler installed with +`Execution.around({ execute })` receives an opaque `ExecutionRequest`, returns +nothing, and has whatever it returns ignored. It may: + +- read the options as they stand (`request.options`); +- narrow or replace them (`request.withOptions(options)`), which supersedes the + request it was derived from; +- register an additive completion failure (`request.addCompletionFailure`); +- install contextual behavior the document will inherit; +- refuse, by throwing, before delegating; and +- delegate. + +It may not complete an execution. Canonical core invokes the stable `Execution` +middleware through a private, per-invocation same-name Api whose instance-owned +default handler is the authoritative terminal; the exported `Execution.execute` +default always refuses. Only canonical core runs a document, after the chain +unwinds, with the options that terminal recorded — so a handler cannot +answer with a synthetic execution, manufacture or replace a `DocumentExecution`, +wrap its output stream, turn a failure into a success, or reach the admissions +the invocation captured. + +The request is a one-use capability. It carries a private reference to a single +invocation, and delegating a reconstructed look-alike, a request a later +`withOptions()` superseded, a request another invocation issued, or the same +request twice is an `ExecutionProtocolError` — fresh and cause-free, raised +before the journal is read, before the document expands, and before anything is +appended. An invocation whose chain returns without ever reaching the terminal +fails the same way. + +Completion failures are additive and apply inside canonical completion, not by +wrapping the returned handle: an execution that already failed keeps its own +failure, the first policy that reports one turns a success into that failure, +and no later policy replaces it. + +### The invocation's lifetime + +Each execution is owned by one structured task holding its own scope. That scope +stays alive through the document's execution and through `Execution.document` +teardown, and **completion becomes observable only after teardown has +finished** — so a caller continuing on the completion continues after cleanup, +and a completed handle carries no live scope. + +Cancelling a consumer of a live returned handle cancels the invocation: authored +work is halted, the invocation's finalizers run exactly once, and every other +observer of that handle settles rather than waiting on a run that is over. +Observing a settled handle again reads the recorded result and starts nothing. + +A document outcome and an invocation-teardown failure are ranked, never +replaced: + +1. a durability failure outranks every other kind; +2. otherwise a Files infrastructure failure outranks an ordinary one; +3. **within the winning kind the earlier failure wins** — the document's + outcome precedes the invocation's teardown — and it is returned by exact + identity, because the engine's fences match the object rather than a rebuilt + copy; +4. without a fatal failure, an existing ordinary document failure remains + unchanged; and +5. only a successful document result is converted by an ordinary teardown + failure. + +Cancellation is held to the same boundary: a document that had already produced +an outcome keeps it, and a fatal failure raised while cancelling tears down +ranks by the same rules. + +Core itself has no knowledge of any particular extension. ### 8.2 Usage from standalone code @@ -7726,6 +7787,41 @@ Defined in [Workflow runs](./workflow-spec.md) §9.4 and §9.6–§9.7. | TX38–TX41 | Totality, on the envelope | A result that refuses to be read, a value that refuses to be read, a settlement that refuses to be read, and a successful result with no value are each malformed rather than unrelated — the fixed cause-free diagnostic, no recorded terminal result reused, no planted text anywhere, nothing expanded and nothing appended, for the original failing selector and for a different selector that would otherwise succeed | | TX42 | Ordinary failed settlement | A root import recorded as failed for non-selection reasons is left alone by this protocol | +### Tier EP — The execution protocol + +Defined in §8.1. + +| # | Test | Verify | +|---|------|--------| +| EP1 | Ordinary execution | `execute()` with nothing installed runs the document unchanged | +| EP2 | Option transformation | Options replaced through `withOptions()` are the options the document runs under | +| EP3 | Answering without delegating | A handler that returns instead of delegating is an `ExecutionProtocolError`: the journal is never read, nothing expands, nothing is appended | +| EP4 | A substitute return | Whatever a handler returns after delegating is ignored | +| EP5 | Refusal before delegation | A throwing handler propagates, and performs no read, expansion, Yield or Close | +| EP6 | Double delegation | Delegating the same request twice fails | +| EP7 | A consumed request | Delegating a request a previous execution consumed fails | +| EP8 | A reconstructed look-alike | A value rebuilt from the public shape is not a request, and nothing is read or run | +| EP9 | A superseded request | Delegating a request a later `withOptions()` replaced fails | +| EP10 | Another loaded copy | Middleware installed through an independently constructed descriptor of the Api's name inspects, transforms and delegates | +| EP11 | Capture precedes installation | Admissions are copied before `install()` runs | +| EP12 | Every admission, in order | Each captured admission runs, in capture order, on the retained history | +| EP13 | One refusal stops everything | A refusing admission prevents every later admission, `ReplayGuard`, terminal reuse, `Execution.document`, authored work and any append | +| EP14 | No ambient channel | Rebuilding the obsolete `executablemd.core.journal-admission` context and setting it to `[]`, before and during the invocation, removes no captured admission | +| EP15 | Precedence over an existing failure | A completion policy is not consulted when the document already failed, and cannot replace its failure | +| EP16 | Additive against a success | A completion policy still turns a successful document into a failure | +| EP17 | First failure wins | With two policies, the first to report a failure is the result and the second is never consulted | +| EP18 | An immutable history | An admission that tries to set `length`, replace an index, delete a member, reverse or splice the retained history is ineffective or refused; the next admission sees the original, the completed replay stays completed, and nothing is appended | +| EP19 | A nested live foreign request | A nested invocation handed its caller's still-live request is refused as another execution's; both invocations then settle on their own | +| EP20 | Concurrent live requests | Two invocations held at a barrier until both requests exist and neither is consumed each refuse the other's, and each still runs its own document | +| EP21 | The exported default | Calling `Execution.operations.execute` directly with a live request refuses and consumes nothing; the request still settles its own invocation | +| EP22 | Invalid values | `null`, `undefined`, primitives, symbols, plain objects and a proxy whose traps throw each produce a fresh cause-free `ExecutionProtocolError`, with no journal read or append | +| EP23 | Invocation context | Context an installation establishes is visible to the document and its teardown, and absent from the next ordinary execution in the same host scope | +| EP24 | Settlement-owned cleanup | Installation finalizers have run exactly once before the completion is observed, on success, document failure and completion-policy failure; concurrent invocations finalize independently; a completed handle re-observed does not refinalize and still replays its output | +| EP25 | Returned-handle cancellation | Halting a separate consumer of an already-returned handle halts the suspended authored work, finalizes exactly once, settles every other observer rather than leaving it waiting, and starts nothing when the handle is observed again; a fatal document result established before cancellation survives it by identity | +| EP26 | A detached options snapshot | Options edited after the private terminal accepted them do not change what executes: the accepted stream receives the events, the accepted modifier factory runs by identity, and the caller's own array and record are observably edited while the execution is not | +| EP27 | Teardown reconciliation | A document outcome and an invocation-teardown failure are ranked, not replaced: a durability failure wins by identity, then a Files infrastructure failure, then an existing document failure, and only a success is converted by teardown — with every finalizer run exactly once before the result is observable | +| EP28 | Invocation isolation | Concurrent observations of two returned handles stay separate: cancelling one consumer halts and finalizes only its own invocation while the other settles independently, and neither invocation's cleanup or authored work reaches the other | + ### Tier SL — Own-scope context updates | # | Test | Verify |