feat(workflow): the solver review rejects a step too simple to justify itself - #89
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The
solverReviewcritic — which gates a generated solver — now rejects a workflow that contains a step too simple to justify its own existence. The generator prompt already says "make meaty steps, no ceremony", but that is a rule in a prompt the model can ignore; the critic is the gate that actually catches it.Why this shape (not a turn count)
It is an evaluation, not a measurement. An earlier idea was to count a step's model round-trips at runtime and flag anything under ~15 turns — dropped: the threshold is arbitrary, a live step legitimately varies, and a runtime check fires after the work is done, too late to fix the decomposition. Whether a workflow is well-split is knowable from the structure, at generation time, before anything runs — so the reviewer judges it by reading the step breakdown and asking, per step: "if it does this little, is it a step at all, or ceremony?"
The rule added to the rubric
A step whose prompt carves out only a sliver — a lookup, a restatement, one trivial action a neighbour would absorb without noticing — should be folded into that neighbour; a step must justify its own fresh context. Kept explicitly distinct from step count: having few steps is not the fault (a lone meaty step is ideal); a thin step is.
Test
theSolverReviewRejectsAStepTooSimpleToJustifyItselfpins the criterion in the reviewer's prompt.