docs: decide what to take from DeepSeek Harness, and what not to - #267
Open
oratis wants to merge 1 commit into
Open
docs: decide what to take from DeepSeek Harness, and what not to#267oratis wants to merge 1 commit into
oratis wants to merge 1 commit into
Conversation
Cloned deepseek-ai/deepseek-harness@47f9438 and read its source tree and docs rather than release notes. It is the closest thing DeepCode has to a direct comparison: another DeepSeek-powered coding agent facing the same constraints. The research doc grades every claim A/B/C the way the Floatboat report did, and records where dsh is ahead (context economy, execution shape, frontend extensibility) alongside where DeepCode already is (file contract, change ledger, sandbox network allowlist, cache-hit pricing). Two real DeepCode defects fell out of the comparison, both verified against our own source: WebFetch returns the whole response body to the model with no text cap, and Bash discards everything past 30 KB with no way to get it back. One capability fixes both. The adoption plan argues each candidate as proponent/opponent/verdict so the implementation PRs execute a decision instead of relitigating one. The largest verdict is a rejection: dsh spends 219 packages expressing what DeepCode expresses in 4, and that ratio buys a third-party plugin ecosystem we do not ship. Take the discipline, not the framework. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Research on
deepseek-ai/deepseek-harness@47f9438and the decisions that follow from it.What this is
Two documents, following the precedent set by the Floatboat round:
docs/research/deepseek-harness.md— the research, with every claim graded A/B/C. A = read the source, B = their docs say so but I did not run it, C = hearsay and inadmissible.docs/DSH_ADOPTION_PLAN.md— each candidate argued proponent / opponent / verdict, so the implementation PRs execute a decision rather than reopen one.I cloned the repo and read its package tree,
docs/architecture.md,docs/capability-seams.md, and the READMEs of every subsystem I judged. I did not rundsh(needs a key and a full build), so every claim about its runtime behaviour is marked B and none of it is copied without redesign.Why it is worth comparing at all
dsh is the closest thing we have to a direct comparison: another DeepSeek-powered coding agent facing the same constraints. Half of what Claude Code and Codex do (cloud tasks, teams, MDM) is worthless at DeepCode's positioning; dsh's trade-offs are made against our own.
The comparison is not one-directional. DeepCode is ahead on governance (File Contract, Change Ledger, sandbox network allowlist) and on pricing awareness (cache-hit tiers,
/costhit rate). dsh is ahead on context economy and on execution shape, and a generation ahead on frontend extensibility.Two defects it surfaced in our own code
Both verified against our source, not inferred:
WebFetchreturns the entire response body to the model.web-fetch.ts:149doescontent: body; the only ceiling upstream is 5 MiB of bytes. A 5 MiB HTML page is roughly 1.5M tokens — one call destroys the session.Bashtruncation is destruction.bash.ts:43-52slices at 30 KB and appends a marker. The remainder is written nowhere. The tail of a failing test run is exactly where the useful part lives.One capability fixes both, and it is the first implementation PR.
Verdicts
The rejection is the load-bearing one. dsh spends 219 packages expressing what DeepCode expresses in 4, and that ratio is not waste — it is what shipping a third-party plugin ecosystem costs. We do not ship one, so we would pay the cost and collect none of the benefit. It also self-describes as a developer preview that will break compatibility, while we have 0.3.0 out with npm, VSIX, DMG, and an update feed downstream. Take the discipline, not the framework.
Three adoptions were narrowed by their own opponent case rather than waved through:
node-ptyis a native dependency needing a build per Tauri target. 90% of the value (survivingcd,export,source venv/bin/activate; reading incremental output) needs no PTY, so this uses a marker protocol over pipes and says plainly in the tool description that full-screen TUIs are out of scope.Verification
pnpm docs:checkpasses; prettier clean. Docs-only — no code touched.