Skip to content

feat(core): stop tools from flooding the model, and stop losing what they said - #268

Open
oratis wants to merge 2 commits into
mainfrom
feat/tool-output-spill
Open

feat(core): stop tools from flooding the model, and stop losing what they said#268
oratis wants to merge 2 commits into
mainfrom
feat/tool-output-spill

Conversation

@oratis

@oratis oratis commented Aug 14, 2026

Copy link
Copy Markdown
Owner

First implementation PR from docs/DSH_ADOPTION_PLAN.md §1.1. Depends on nothing; #267 carries the reasoning.

The two defects

Both verified against our own source during the DeepSeek Harness comparison, and both trace to the same cause: nothing central bounds what a tool result puts into the model's context, so each tool improvises.

WebFetch improvised by not bounding at all. web-fetch.ts returned content: body; the only ceiling was 5 MiB of bytes upstream. A 5 MiB HTML page is roughly 1.5M tokens — a single call ends the session.

Bash improvised by destroying evidence. bash.ts sliced each stream at 30 KB and appended ... [stdout truncated]. The remainder went nowhere. The tail of a failing test run is the part worth reading, and it was precisely the part being discarded.

What this adds

applySpillPolicy runs on every tool result on its way to the model:

  • At or under the threshold → returned untouched. Same object, not a copy. Invisible for the overwhelming majority of calls.
  • Over → head+tail preview naming the file holding the full text, saved under the session directory beside that session's snapshots so both fall under whatever retention policy that directory eventually gets.

Retrieval needs no new tool — the preview tells the model to Read the path, and offset/limit already page through it.

Two choices worth flagging

The preview keeps both ends, weighted toward the tail (40/60). dsh only requires a bounded preview. Stack traces, assertion diffs, and exit codes live at the end; head-only truncation keeps the preamble and throws away the answer, which is what we were doing.

Bash's capture buffer had to grow for there to be anything to save. It now uses the same bounded head+tail primitive at a much larger limit, so memory stays bounded regardless of how much a runaway command prints — a gigabyte costs the same as a kilobyte. This is a capture limit, not the model-visible limit; those are now separate numbers with separate jobs.

Degradation

A host with no filesystem (the Tauri renderer, where the loop runs in the webview) gets the bound but cannot offer retrieval. The preview says the output was not saved and suggests narrowing the command, rather than printing a path that does not exist. A save that throws degrades identically instead of failing the tool call — a partly-readable result beats an error.

The store is resolved once and memoized on the promise, not the result: tools run concurrently, and a second caller arriving mid-import must wait for the same answer rather than read "not resolved yet" as "no store".

Verification

pnpm typecheck, lint, format:check, docs:check all clean. Full suite green — core 1056 passed / 28 skipped, cli 242, desktop 104, server 49, protocol 34, lsp 13, vscode 12, scripts 42.

26 new tests. The ones that carry weight:

  • an agent-loop test driving a tool that returns 200 KB, asserting what reached the provider is under 2 KB and still starts with HEAD and ends with TAIL
  • the same run with a session attached, reading the saved file back and asserting all 200,008 characters survived — this exercises the real lazy import('./spill/local.js') path, not a stub
  • surrogate-pair safety at every cut point, so an emoji at the boundary cannot become a replacement character
  • name-collision safety: two call ids that sanitize to the same filename get separate artifacts rather than one silently overwriting the other

…they said

Two verified defects, one cause: nothing central bounds what a tool result puts
into the model's context, so each tool improvises.

WebFetch improvised by not bounding at all — `content: body`, capped only at
5 MiB of bytes upstream. A 5 MiB page is roughly 1.5M tokens; one call ends the
session. Bash improvised by slicing at 30 KB and appending a marker, writing the
remainder nowhere. The tail of a failing test run is exactly the part worth
reading, and it was the part being discarded.

The spill policy now runs on every tool result on its way to the model. Results
at or under the threshold pass through untouched — identical object, no copy —
so this is invisible for almost every call. Oversized results become a head+tail
preview naming the file where the full text was saved, under the session
directory beside its snapshots.

Head AND tail, weighted toward the tail: stack traces, assertion diffs, and exit
lines live at the end. Head-only truncation throws away the answer and keeps the
preamble.

Bash's capture buffer had to grow for there to be anything to save, so it now
uses the same bounded head+tail primitive at a much larger limit. Memory stays
bounded no matter how much a runaway command prints — a gigabyte costs the same
as a kilobyte.

Hosts without a filesystem (the Tauri renderer) still get the bound; they just
cannot offer retrieval, and the preview says so rather than implying a file
exists. Saving failures degrade the same way instead of failing the call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`new SessionManager(sessionsRoot)` passes a string where the constructor wants
`{ root }`, so `opts.root` was undefined and the manager silently fell back to
the real `~/.deepcode/sessions`. The test still passed — it reads the file back
through the locator it was handed — while writing a session and a spill artifact
into the user's actual home directory on every run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant