Skip to content

♻️ Give every generated eval module a scope-owned lifetime - #408

Merged
taras merged 3 commits into
mainfrom
agent/issue-182-temp-file-compiler
Aug 9, 2026
Merged

♻️ Give every generated eval module a scope-owned lifetime#408
taras merged 3 commits into
mainfrom
agent/issue-182-temp-file-compiler

Conversation

@taras

@taras taras commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Closes #182.

Base: 35060af (origin/main, PR #406). The plan named 0f11a5e (#405); main
advanced one commit before this branch was cut, so it is based on 35060af.
No ci-main-red issue was open at push time.

Why

packages/core/src/temp-file-compiler.ts is the portable eval-block compiler —
the one Node's tsx loader accepts, and the one every runtime can load. It
generates a real file on disk and did not own it: removal was a fire-and-forget
unlink(tmpPath).catch(() => {}) inside a finally. Nothing waited for the
removal, nothing reported its failure, and suspending cleanup inside finally
is what local/no-yield-in-finally exists to prevent. The same file also
predated Code Rules 2 and 3: node:fs/promises imports and call()-wrapped
promises throughout.

What changes

Before:

  • The generated .xmd-eval/<uuid>.ts outlived its compilation by an unbounded
    amount. A cancelled compilation could leave the file behind entirely, because
    the finally never ran the removal to completion.
  • A removal that failed for any reason — a permission error, a read-only mount —
    was swallowed by .catch(() => {}).
  • Filesystem work went through node:fs/promises; every promise went through
    call().

After:

  • Each compilation runs in a private scoped() region. Removal is registered
    with ensure() against the UUID path before the write can begin, so the
    file is gone before the compilation settles, whichever way it settles:
    before a compiled block is returned, before a failing import reaches the
    caller, and before a cancelled compilation finishes halting.
  • A removal that fails for any reason other than the file already being absent
    becomes the compilation's outcome instead of being discarded. { force: true }
    is what makes "already absent" the one tolerated condition.
  • ensureDir, writeTextFile and rm come from @effectionx/fs; the dynamic
    import goes through until(import(fileUrl)). No node:fs operation and no
    call()-wrapped promise remain.

How it works

compileTempFile → scoped() → ensureDir → choose UUID path → ensure(rm)
                → writeTextFile → until(import) → validate default export
                → scope teardown removes the file → the outcome leaves

scoped() is what makes the ordering a fact rather than an intention: it closes
its scope, running the registered destructor to completion, before it returns a
value, rethrows, or finishes halting.

Review guide

Start with: packages/core/tests/temp-file-compiler.test.ts

Then review:

  1. compileTempFile in packages/core/src/temp-file-compiler.ts — the
    scoped() boundary and where ensure() sits relative to the write
  2. The compiler section of specs/executable-mdx-spec.md and the Tier TC rows

Look carefully at:

  • The ensure() registration point. Registering it one line later — after the
    write — is a mutation the halt test is there to kill.
  • TC4's interceptor placement: it is installed inside a nested scope so the
    recorder's safety cleanup, registered outside it, can still remove files a
    failing run leaves behind.

What must stay true

  • The generated file does not outlive its compilation — enforced by the
    scoped() boundary plus an ensure() registered before the write, and
    checked by TC1, TC2 and TC3.
  • A non-ENOENT removal failure is not swallowed — enforced by rm(path, { force: true }) inside ensure() rather than a .catch(), and checked by
    TC4.
  • Existing compiler behavior is untouched: cwd-relative .xmd-eval, UUID .ts
    filenames, standard plus user import construction, file:// loading, and the
    default-export validation and its diagnostic. Checked by the Tier T2 suite
    (packages/core/tests/eval-context.test.ts), which drives this compiler.

How to verify it

@effectionx/fs is itself a contextual Api, so FsApi.around() observes the
compiler's real write and real removal as they happen. Each test asserts the
exact recorded order at the moment the compilation settles, which is what makes
"before it returned" falsifiable rather than eventually-true.

  • TC1 compiles and runs a valid block, and proves the write and the removal
    name the same .xmd-eval/<uuid>.ts and that the removal completed before
    compileTempFile() returned. Fails if the removal is launched without being
    awaited, or if the cleanup is registered on the caller's scope instead of a
    private one.
  • TC2 compiles generated code that does not parse, proves the import failure
    still propagates, and proves the file was removed before that failure reached
    the caller. Fails if teardown is skipped on the error path.
  • TC3 intercepts writeTextFile, delegates so the real file exists, signals
    the test, then suspends — handshake-driven, no timer. It halts the spawned
    compilation and proves the file is gone once halt() settles. Fails if the
    cleanup is registered after the write.
  • TC4 intercepts rm for that compilation's generated path, performs the
    real removal, then throws a unique sentinel. It proves compileTempFile()
    fails with that exact Error — by identity — rather than returning a block,
    swallowing the failure, or substituting an error of its own. Fails if the
    cleanup error is caught or discarded.

Every test registers a force-removal safety cleanup for the paths it caused, so
a failed mutation cannot leave scratch files behind. None asserts that
.xmd-eval is globally empty — other tests compile into it concurrently.

Mutation evidence

Each mutation was applied to the committed implementation by an edit that
asserts it matched exactly once, then reverted from a pristine copy taken with
git show HEAD:… and confirmed byte-identical afterwards
(deno task test packages/core/tests/temp-file-compiler.test.ts):

Mutation TC1 TC2 TC3 TC4
ensure() cleanup removed FAIL FAIL FAIL FAIL
cleanup registered after the write pass pass FAIL pass
private scoped() boundary removed FAIL FAIL pass FAIL
removal launched without being awaited (scope.run, unawaited) FAIL FAIL FAIL FAIL
cleanup failure caught and discarded (try/catch inside ensure) pass pass pass FAIL

Each mutation is killed, and the last one is killed by TC4 alone — the row that
shows the new test carries its own weight. .xmd-eval was empty after the whole
mutation run, which is the safety cleanup doing its job.

Commands

All run on 5c432f1; 94f85b7 adds only the specification wording correction above, re-verified with deno task lint, deno task check, the focused lifecycle suites and git diff --check:

deno task test packages/core/tests/temp-file-compiler.test.ts packages/core/tests/eval-context.test.ts   # 3 passed (13 steps), 0 failed
pnpm exec tsx --tsconfig tsconfig.node.json --test <same two files>                                       # 13 pass, 0 fail
bun test <same two files>                                                                                 # 13 pass, 0 fail
deno task lint                                                                                            # exit 0, 0 errors
deno task check                                                                                           # exit 0
deno task check:jsr                                                                                       # Success Dry run complete
deno task test --changed=origin/main                                                                      # 178 passed (1154 steps), 0 failed
git diff --check                                                                                          # clean

Scope

Included

  • compileTempFile()'s generated-file lifecycle, and its @effectionx/fs and
    until migration.
  • Tier TC in specs/executable-mdx-spec.md: the compiler section states the
    ownership in present tense, and four conformance rows match the four tests
    one-to-one.

Intentionally unchanged

  • EVAL_DIR stays the relative literal ".xmd-eval". Bring temp-file compiler in line with the Effection and filesystem rules #182 notes the
    cwd-dependence and does not require changing it. The spec now says precisely
    what that means: the path is resolved against the host process's current
    working directory at the moment a compilation chooses it — read then, not
    fixed at startup, since compileTempFile() is publicly callable and a host
    may have moved that directory. Running path/to/document.md does not itself
    move it to the document's directory, and the contextual API.Env.cwd does
    not control it, because this compiler never consults that Api. No test
    changes process-global cwd: doing so would make the parallel test corpus
    unsafe.
  • This is engine-owned compiler scratch state, not document-facing filesystem
    access, so it does not route through API.Files. A document never names this
    path.
  • node:path and node:crypto remain — @effectionx/fs has no equivalent.
  • packages/cli/src/node.ts still spells its provider return yield* compileTempFile(...). It type-checks unchanged and is outside this change.
  • useDataUriCompiler() writes nothing and needed nothing here.

Generated or mechanical changes

  • None. No dependency was added and the lockfile is untouched.

Risks and limitations

  • compileTempFile's declared return type narrows from a generator to
    Operation<EvalBlock> because it now returns scoped(...). Every yield*
    call site is unaffected — a Generator already satisfied Operation — and
    deno task check, check:jsr and the Node typecheck all pass. Only a caller
    stepping the generator by hand, which nothing does, would notice.
  • A removal failure is now the compilation's outcome where it was previously
    silent. That is the point of the change, but it does convert a class of host
    filesystem faults — a read-only mount, a revoked permission on .xmd-eval
    from invisible into a failed eval block.
  • The success path removes the file after the module is loaded, so a stack trace
    or source map resolved later cannot read it back. That was already true: the
    previous finally removed it too, just at an unpredictable moment.

Scope confirmation

  • Every changed file supports the purpose described above.
  • Unrelated cleanup and formatting changes are excluded.
  • Generated or mechanical changes are clearly identified.
  • The description matches the final diff and test results.

The temp-file compiler wrote `.xmd-eval/<uuid>.ts` and then dropped its
removal into a `finally` as a fire-and-forget promise, so the file outlived
the compilation by an unbounded amount and a removal that failed was
discarded unread.

Each compilation now runs in a private scope, and the removal is registered
against the UUID path before the write can begin. The file is gone before the
compilation settles — before a block is returned, before a failing import
reaches the caller, and before a cancelled compilation finishes halting — and
a removal that fails for any reason other than the file already being absent
leaves that scope.

Filesystem work goes through `@effectionx/fs`, and the dynamic import through
`until`, so no `node:fs` operation and no `call()`-wrapped promise remain.
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

PR #408: ♻️ Give every generated eval module a scope-owned lifetime

3 files, +248 / -17

Scope

✅ PR scope looks good.

Structural

✅ No structural bloat detected.

Slop

✅ Slop indicators look low.

Static Analysis

Oxlint: 2 diagnostics across 1 file (1 rule)
Density: 0.008 violations/added-line

no-unassigned-import (2): packages/core/src/temp-file-compiler.ts

Correctness

No extraneous code patterns detected.

taras added 2 commits August 9, 2026 07:30
The spec implied `.xmd-eval` follows the running document. It does not:
`compileTempFile()` resolves the relative literal against the host process's
working directory, which running `path/to/document.md` never moves and which
the contextual `API.Env.cwd` does not control.

The claim that a removal failure other than an already-absent file leaves the
private scope had no test behind it. TC4 performs the real removal and then
fails, and holds that exact error to be the compilation's outcome.
`node:path.resolve()` reads the process working directory when it is called,
and `compileTempFile()` is publicly callable, so a host may have moved that
directory since startup. Saying `.xmd-eval` follows the directory the process
was started in claimed an immutability the compiler does not have.
@taras
taras marked this pull request as ready for review August 9, 2026 11:56
@taras
taras merged commit db7ca71 into main Aug 9, 2026
11 checks passed
@taras
taras deleted the agent/issue-182-temp-file-compiler branch August 9, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bring temp-file compiler in line with the Effection and filesystem rules

1 participant