Add Advisor Strategy and Smart Plan, Cheap Execution patterns - #56
Open
devill wants to merge 1 commit into
Open
Conversation
Two complementary ways to split work between an expensive and a cheap model. Advisor Strategy runs the cheap model by default and escalates to a stronger advisor on defined triggers; Smart Plan, Cheap Execution has the strong model plan up front and a cheap model execute. Advisor Strategy is named after and cites Anthropic's advisor strategy, which measured it: https://claude.com/blog/the-advisor-strategy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two complementary patterns for splitting work between an expensive and a cheap model. They read as a pair and cross-reference each other, so they go in together.
Advisor Strategy — the cheap model runs by default and consults a stronger model as an advisor on defined triggers (two failed attempts, self-contradiction, declared uncertainty). The advisor writes no code and runs no tools, so its share of the bill stays small.
Smart Plan, Cheap Execution — the strong model plans, the plan is checkpointed to a document, a cheap model executes it step by step.
Advisor Strategy is named after Anthropic's advisor strategy, which measured it: Sonnet 4.6 with an Opus advisor gained 2.7 points on SWE-bench Multilingual over Sonnet alone and cost 11.9% less per task; Haiku 4.5 more than doubled its BrowseComp score, 19.7% to 41.2%. The write-up presents the advisor arrangement as an inversion of plan-then-execute, so Smart Plan, Cheap Execution cites it as contrast rather than as evidence.
Replaces the drafts #43 and #44, which held the same two patterns on separate branches with earlier text, and #55, which used the working name "Advisory Strategy".
Checks
npm run validate,npm run lint, 336 unit tests, andnpm run buildall pass. Both pages checked in the browser: headings, the Anthropic link, and the Related sidebar (similar-to / solves Sunk Cost) render correctly.