Implement retained Workflow Workspace restoration

[taras]

Story

As a workflow author, I want xmd workflow to retain one provider-backed Workspace with the run's journal, repositories, and Agent state, so repeatable procedures resume after interruption without reconstructing intermediate filesystem history or granting Agents mutation authority.

Settled contract

PR #358 records the normative behavior in architecture.md and specs/workflow-workspace-spec.md:

• xmd run uses the caller's current environment and has no restoration guarantee.
• xmd workflow creates a WorkflowRun with one implicit root Workspace.
• Each run owns one SQLite database containing logically separate filtered journal, versioned DOFS, Repository/Worktree metadata, and Agent-session stores.
• One expansion produces one effect and one Workspace transaction.
• Completed durable effects restore their results; ephemeral attachments rebuild the live Effection tree; partial replay continues against the retained frontier.
• Missing authoritative Workspace state is unrecoverable rather than silently replaced.
• Agents are read-only and propose mutations as constrained generated XMD.
• Prompt, Push, PullRequest, and Issue are reconciled external effects rather than local transactions.
• Worker Shell is included through a Deno Worker and just-bash, after the transaction spine exists.
• Public lifecycle commands remain host-neutral so a future cloud host can implement them; no remote-host selector ships initially.

This is the implementation umbrella. Child issues own executable acceptance details; this body owns their dependency order and the invariants they share.

Implementation order

1. Retention and one filesystem vertical slice

• Store WorkflowRuns and filtered journals in SQLite #291 — open and look up one WorkflowRun SQLite database; append/replay the filtered journal.
• Commit Workspace mutations and journal results atomically #365 — commit a Workspace mutation, logical root, and journal result atomically.
• Add the xmd workflow start/resume filesystem vertical slice #366 — implement foreground workflow start/resume with an implicit Workspace and declarative File effects.
• Add workflow inspection, cancellation, suspension, and deletion #367 — add status/list/history, suspension, cancellation, deletion, and single-executor ownership.
• Add versioned workflow history checkpoints and forks #368 — add versioned history checkpoints and compatible forks.

2. Repository and deterministic Git composition

• Add named Repository and Worktree composition #293 — named Repositories and named Worktrees with lexical cwd.
• Add deterministic local Git workflow effects #294 — transactional Git.Switch, Git.Add, and staged-only Git.Commit.
• Add shared external forge-effect reconciliation #297 — shared external forge-effect reconciliation.
• Add a deterministic Git.Push workflow effect #370 — explicit deterministic Git.Push.
• Add a deterministic PullRequest workflow effect #295 — deterministic PullRequest over an explicitly pushed head.
• Add a deterministic provenance-linked Issue workflow effect #296 — provenance-linked deferred Issue.

3. Agent authority and constrained execution

• Contain filesystem access at the provider boundary #227 — provider-correct filesystem containment guarantees.
• Enforce read-only workflow Agent authority #302 — mandatory read-only workflow Agents and Agent.AddDir.
• Evaluate Agent-generated XMD through a constrained allowlist #369 — preflight and execute constrained Agent-generated XMD.
• Add transactional Worker Shell to workflow Workspaces #363 — production transactional Worker Shell after Commit Workspace mutations and journal results atomically #365 and Add the xmd workflow start/resume filesystem vertical slice #366.

4. Compose and certify the product workflow

• Namespace Markdown props under the props binding #305 — namespace document props consistently.
• Synchronize the #181 living target with settled XMD contracts #292 — synchronize the living target in PR Document adversarial implementation workflow WIP #181 with the settled contracts and this dependency map.
• Prove the shipped adversarial-planning document logic #290 — prove the shipped planning-document logic.
• Compose the supervised adversarial implementation workflow #301 — compose the supervised adversarial implementation workflow.
• Resume workflow runs from explicit durable signals #300 — resume it from explicit durable signals without premature watcher semantics.
• Reject secrets before journal persistence by default #199 — finish cross-runtime journal-secret certification and documentation.
• Certify the adversarial implementation workflow end to end #299 — certify interruption, replay, authority, reconciliation, and cross-runtime behavior end to end.

Cross-cutting invariants

• WorkflowRun establishment and base pinning remain acyclic and precede Workspace attachment.
• Shared production modules use contextual APIs and contain no Deno, SQLite, DOFS, Cloudflare, GitHub, or runtime detection.
• One authoritative host-owned DOFS connection serves a workflow database and serializes Workspace-local effects.
• Existing security filtering runs before journal insertion; co-located DOFS content is not automatically journal or training data.
• Completed root replay attaches no Workspace, Agent, process, or external provider.
• Status, list, and history are read-only and cannot advance a run.
• No required state exists only in an Agent transcript, host path, provider handle, or branch name.
• External effects never claim atomicity across SQLite and a provider; they reconcile stable identity.
• Evidence PRs 🔬 Spike: host Cloudflare DOFS directly in Deno with SQLite (#349) #350, 🔬 Spike: run Cloudflare Worker Shell and Worker JavaScript natively in Deno (#351) #353, and 🔬 Spike: make Worker Shell effects transactional (#357) #362 remain unmerged experimental evidence. Production is implemented in focused PRs from the child issues.

Completion

This umbrella closes when the retained Workspace lifecycle, repository/Git composition, read-only Agent path, constrained Worker Shell, and end-to-end workflow certification are complete and PR #181 describes only shipped behavior.

Intentionally excluded

• A public remote-host selector or deployed XMD service.
• Worker JavaScript in the initial topology.
• Native subprocess execution, writable FUSE, bundled workerd, or Containers.
• Automatic ingestion of arbitrary Workspace files for training.
• Recovery from deleted or corrupt authoritative Workspace state.
• Rewinding external systems during a history fork.

Evidence

• Add contextual workflow runs with pinned commits #289 / merged PR 💥 Add expansion identity and pinned-commit workflow runs #341 supplies WorkflowRun and expansion identity.
• Spike: host Cloudflare DOFS directly in Deno with SQLite and FUSE #349 / PR 🔬 Spike: host Cloudflare DOFS directly in Deno with SQLite (#349) #350 proves Deno-local SQLite/DOFS behavior.
• Spike: run Cloudflare Worker Shell and Worker JavaScript natively in Deno #351 / PR 🔬 Spike: run Cloudflare Worker Shell and Worker JavaScript natively in Deno (#351) #353 proves contained Worker Shell and Worker JavaScript models.
• Spike: make Worker Shell effects transactional with the workflow journal #357 / PR 🔬 Spike: make Worker Shell effects transactional (#357) #362 proves the accepted transactional Worker Shell mechanism.
• 📝 Specify retained workflow Workspaces #358 is the merged architecture and observable-contract decision.
• End goal: Document adversarial implementation workflow WIP #181.

[taras]

Design-session record

This comment preserves the rationale and source audit behind the product contract in the issue body. The body remains the implementation contract; this is the reasoning that should not be rediscovered or silently reversed.

Why the original replay choices were replaced

The issue originally asked whether an effect recorded inside a replaced temporary environment should be re-executed, include the environment in its identity, or fork the journal.

That framing assumes current filesystem state can validate every historical event. It cannot. In this sequence:

<TempDir>
  <File path="abc.txt">content</File>
  <DeleteFile path="abc.txt" />
  <File path="xyz.txt">other content</File>
</TempDir>

an interruption after DeleteFile leaves abc.txt absent. That absence is the correct journal-frontier state. A replay guard on the earlier File cannot distinguish “the create never happened” from “a later recorded delete happened.” Current state can validate a frontier invariant, not every intermediate postcondition.

Executable.md is a repeatable workflow system, not a general agent harness that repairs an arbitrarily broken machine. Normal restoration should return to a state from which the recorded workflow can continue. Missing or corrupt provider state is an explicit stale/unrecoverable condition.

Replay and ephemeral work

The original mental model remains useful:

• replay rehydrates the Effection tree;
• durable effects are restored from the journal and are not repeated; and
• non-journaled ephemeral operations run again to reconstruct scope-owned runtime structure.

That matches @executablemd/durable-streams’ existing ephemeral(operation): it emits no journal entry, consumes no replay position, reruns on replay, and remains owned by the surrounding Effection scope.

For Workspace, the rerun operation establishes an attachment. It does not create a fresh empty filesystem. The durable provider state and ephemeral attachment have different lifetimes.

Existing provider precedent

The ACPX integration already separates a provider-neutral capability from a provider-specific adapter:

• core owns the Agent API, provider-factory seam, registry, and Markdown composition;
• @executablemd/acp exports createAcpxProvider() and owns provider handles;
• the host registers and selects a root provider;
• provider installation is lazy and scope-owned; and
• tests can install the provider directly with injected dependencies.

Workspace should reuse that shape where it fits. Its important difference is that the attachment is document-execution scoped while the logical Workspace state outlives that execution.

Workspace supplies implementations for the existing runtime contexts. It does not create a second filesystem or process surface.

Cloudflare Computer capability audit

Official sources inspected during the design session:

• package overview
• VFS layout
• filesystem schema
• runtime interface
• lifecycle
• Artifacts interface

The system already provides:

• a Workspace constructed inside a Durable Object from ctx.storage;
• a SQLite-backed VFS as the durable source of truth;
• a transient container filesystem synchronized with that VFS;
• filesystem-only operation or pluggable container, shell, and JavaScript execution backends;
• push -> execute -> pull synchronization for command backends that need it;
• retained execution records and reattachment for the container and JavaScript backends; and
• optional Git and Cloudflare Artifacts capabilities.

It does not currently provide a generic point-in-time Workspace snapshot/history API. VFS revisions and tombstones are synchronization machinery, not an audit history. Artifacts is a separate, opt-in Git repository service rather than automatic Workspace capture.

That does not block normal resumption: reattach to the current durable VFS for the same run. It does rule out rollback, arbitrary journal-prefix rewind, and branching from an intermediate filesystem state.

The package is preview-only and its documentation is partly forward-looking. Its lifecycle document also identifies reconnect/hibernation gaps: a dead capnweb session currently requires callers to reconstruct the Workspace, and some mismatch recovery remains future work. The adapter must expose only behavior the selected version actually ships.

Cross-system durability

The XMD journal and Cloudflare VFS do not share a transaction. A crash can occur after a Workspace mutation commits but before its XMD result is appended. The design therefore depends on repeatable mutation, stable provider execution identity, or explicit uncertain-completion failure—not historical filesystem replay.

Cloudflare’s runtime result can report a completed command with post-command synchronization still pending. The adapter cannot turn that into a successful durable XMD effect until the Workspace source of truth is synchronized.

Retention, training, and secrets

The run-scoped Workspace is retained because its resulting environment is useful evidence and may contribute to future training. Scope cleanup releases attachment resources; it does not delete durable state.

The journal was given default secret detection because durable run history is intended to be retained and reused. #218 does not invent another capture/export pipeline or security policy. #199 and #291 continue to own the persistence boundaries they create. If a future training exporter promotes raw Workspace contents into another store, that exporter becomes a new persistence boundary and must be reviewed then.

Relationship to PR #341

PR #341 is the correct foundation and remains safe to merge:

• it creates the stable run ID and deterministic expansion ID that Workspace provenance needs;
• it keeps both values independent of provider-specific state;
• it scopes live metadata to one document execution, matching the attachment lifetime;
• completed replay restores metadata without invoking providers; and
• it explicitly says its contextual values are not authority boundaries.

The last point is load-bearing. The runId + expansionId pair identifies a logical Workspace, but provider authorization must be rooted in a host-owned installation. A document-rebound context value must not grant access to another run’s provider resources.

Workflow-run establishment also precedes root import, while <Workspace> is reached during root expansion. Base pinning must therefore remain independent of Workspace acquisition. The intended order is: host pins the base and records the run, Workspace attaches using that run, then #293 reconciles the Worktree inside it.

One host-topology question remains

@cloudflare/computer is designed to construct a Workspace inside a Cloudflare Durable Object using Workers bindings. This repository currently has no Cloudflare Worker, Durable Object, or Wrangler host.

Before implementation is delegated, the host boundary must be selected: either #218 introduces a Cloudflare Workers/Durable Object host for Executable.md, or an existing XMD host reaches a separately deployed provider through an explicit transport. Importing @cloudflare/computer into the current local CLI is not by itself a viable provider topology.

This question does not affect whether #341 can merge, but it materially changes #218’s implementation scope and should be decided by Taras rather than by its implementor.