Skip to content

hub-mcp: agents can see render errors — local WASM validation, render-checked writes, fix_errors prompt - #452

Open
icarusz wants to merge 9 commits into
mainfrom
feature/hub-mcp-get-errors-v2
Open

hub-mcp: agents can see render errors — local WASM validation, render-checked writes, fix_errors prompt#452
icarusz wants to merge 9 commits into
mainfrom
feature/hub-mcp-get-errors-v2

Conversation

@icarusz

@icarusz icarusz commented Aug 4, 2026

Copy link
Copy Markdown
Member

What

Agents using the Quarto Hub MCP server can now see render errors and verify their own edits:

  • get_errors tool (read-only, available in --read-only mode): renders project documents locally using the same wasm-quarto-hub-client module the browser preview runs, hosted in the MCP's Node process, and reports structured diagnostics (kind, title, Q-* code, line/column, hints) for exactly the content it read — checkedContentSha256 names the bytes. Pass path for one document, omit it to check every .qmd (capped at 25). Engine execution errors are surfaced from the existing captures sidecar (they happen on executors elsewhere and can't be recomputed locally).
  • Writes render-check themselves: write_file / patch_file / create_file on a .qmd stage the new text over the project file map, render it, and append the outcome to the tool response — Render check: clean. or the structured error list. Error visibility is part of completing an update, not a separate call the agent must remember. A check that cannot run degrades to a pointer at get_errors and never fails the write. Non-.qmd writes are unchanged.
  • fix_errors MCP prompt (surfaced as a slash command, e.g. /mcp__quarto-hub__fix_errors in Claude Code): one-command entry into the loop — check, read, minimal patch, repeat until clean. The prompt only instructs; the LLM does the fixing with the existing tools.

Why this architecture

v1 of this work had the browser preview publish diagnostics into an automerge index-doc sidecar. Review guidance from @cscheid killed that approach:

Don't try to chase synchronization with the CRDT. Just grab the content of the file/project you care about and have an API entry point to check for the validity. You're never going to be able to know if the document you just changed ends up looking exactly how you expected it to, because it's a distributed system.

v2 (this PR) makes validity a function of content: render what you hold, report what you rendered. No schema change, no new API entry point, no cross-peer choreography, hub-client untouched. (v1 is preserved on feature/hub-mcp-get-errors if a human-facing "see collaborators' preview state" feature is ever wanted.)

How

  • src/local-render.ts — lazy-loaded WASM host (server startup stays instant; the ~38 MB init is paid on first use), VFS fill with the /project/ prefix, renders serialized via a promise chain (the VFS is instance-global), diagnostics mapped and stripped of ANSI/$schema noise.
  • scripts/build-wasm-host.mjs — esbuild-bundles the wasm-bindgen JS for plain Node: aliases the Vite-root bridge imports (/src/wasm-js-bridge/*) to ts-packages/wasm-js-bridge sources, bundles dart-sass in (the embedded bundle has no node_modules), inits from bytes. Consumed by both the tsc dev build (dist/) and the embedded bundle (dist-bundle/, via cargo xtask build-hub-mcp-bundle).
  • src/prompts.ts — prompts capability + fix_errors.
  • src/tools.tsget_errors handler, renderCheckSuffix on the write tools.

Note: the q2 binary now embeds a second copy of wasm_quarto_hub_client_bg.wasm (~38 MB; the preview SPA already ships one). Deduping at the launcher level is a known follow-up, deliberately not in this PR.

Testing

  • local-render.test.ts — real WASM, no mocks: broken YAML/markdown → structured diagnostics with line/col; sibling pass-1 attribution; binary tolerance; render serialization.
  • get-errors-handler.test.ts, write-render-check.test.ts — real registerTools dispatch against a fake connection manager, renderer mocked at the module seam; fail-first TDD.
  • get-errors-live.test.ts — the real server binary + in-process test hub over stdio: create broken project → get_errors reports Q-2-13 at 5:24 → patch_file (response carries Render check: clean) → get_errors immediately clean.
  • bundle.test.ts — pins that dist-bundle ships wasm-host.mjs + the .wasm.
  • End-to-end beyond CI: exercised against production quarto-hub.com through the real q2 mcp binary — found and fixed a real unclosed-** error in a live deck, then authored a full revealjs deck through the MCP with every write returning its render check inline.

Plan doc with the full verification record: claude-notes/plans/2026-07-28-hub-mcp-get-errors-v2.md.

🤖 Generated with Claude Code

icarusz and others added 8 commits August 3, 2026 11:24
…ion)

Pivot per review guidance: drop the CRDT diagnostics sidecar entirely;
get_errors will render the files the MCP already holds using the same
wasm-quarto-hub-client module the browser preview runs (feasibility
spike in plan doc), plus execution errors from the existing captures
sidecar. Carries over from v1: the local-prod WS proxy crash fix and
the two get_errors test files (to be reworked onto the new backing).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
renderDiagnostics(files, path) fills the WASM VFS with the project the
MCP already holds and calls render_page_in_project — the same
wasm-quarto-hub-client module the browser preview runs — returning the
structured errors/warnings plus sibling pass-1 failures and the sha256
of exactly the text it rendered. The module lives in a prebundled host
(scripts/build-wasm-host.mjs: esbuild with the /src/wasm-js-bridge/*
aliases and dart-sass bundled in, since html theme compilation needs
it and the embedded bundle has no node_modules), loaded lazily on
first use. Tests run the real WASM, no mocks — including a pin that
`title: "broken` front matter is a warning, not an error, exactly as
the preview reports it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… was checked

The tool renders the requested document (or every .qmd, capped at 25)
through the local WASM pipeline and reports structured errors/warnings
plus the sha256 of exactly the content it checked; sibling pass-1
failures surface under their own paths, and execution errors still
come from the captures sidecar (now mirrored into ProjectState via
onCapturesChange). No staleness concept: after an edit, calling the
tool again validates the new content immediately.

Live integration test drives the real server binary + real WASM over
stdio against the in-process test hub: broken doc reported at line 5,
patch_file fix, immediate clean re-check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bundle.mjs now builds wasm-host.mjs (bridges + dart-sass inlined) and
copies wasm_quarto_hub_client_bg.wasm alongside index.mjs, so the
embedded `q2 mcp` and the npx channel can run get_errors' local
validation with no node_modules. bundle.test.ts pins the artifacts.
Note: the q2 binary now embeds a second copy of this WASM (the preview
SPA has its own) — dedupe tracked as follow-up in the v2 plan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Agents get structured fields (code, problem, line/col, sub-details) —
the ariadne escape-code blob is token noise. Plan doc records the
end-to-end run: broken doc via q2 mcp against a real hub reported
Q-2-13 at 5:24 with the checked content hash; patch_file then an
immediate clean re-check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shrink the speculative failed-render fallback to a plain correctness
guard (a failed render must never read as clean) — which also fixes
its ANSI-strip regex, previously missing the ESC byte. Tidy the
CLI-main check in build-wasm-host.mjs to reuse the existing node:url
import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Clients surface it as a slash command (Claude Code:
/mcp__quarto-hub__fix-errors <project-or-share-url> [path]). The
prompt expands to loop instructions — get_errors, minimal patch_file
fixes at the reported line/column, re-check until clean, report —
and the LLM does the fixing; the server stays a set of primitives.
Protocol round-trip test pins the prompts capability declaration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
write_file/patch_file/create_file on a .qmd now stage the new text over
the project file map, render it locally, and append the outcome to the
tool response (clean / error list / check-unavailable note). Error
visibility becomes part of completing an update instead of a separate
call the agent must remember; the fix-errors prompt now leans on the
in-response check with one confirming get_errors at the end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@posit-snyk-bot

posit-snyk-bot commented Aug 4, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@cscheid

cscheid commented Aug 4, 2026

Copy link
Copy Markdown
Member

Minor question: write_file (underscore) but fix-errors (dash). Is that intentional?

@cscheid

cscheid commented Aug 4, 2026

Copy link
Copy Markdown
Member

Note: the q2 binary now embeds a second copy of wasm_quarto_hub_client_bg.wasm (~38 MB; the preview SPA already ships one). Deduping at the launcher level is a known follow-up, deliberately not in this PR.

This is probably worth a fix before merging.

@icarusz

icarusz commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Yupyup. Makes sense. Claude and I are on it.

All tool names in this server are snake_case; the prompt now matches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@icarusz icarusz changed the title hub-mcp: agents can see render errors — local WASM validation, render-checked writes, fix-errors prompt hub-mcp: agents can see render errors — local WASM validation, render-checked writes, fix_errors prompt Aug 4, 2026
@icarusz

icarusz commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

OK made the change to standardize that. Claude felt the reasoning was sound (APIs are snake case but slash commands, like all of claudes, are kebab case) but that difference is annoying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants