fix(launch): tier backend formats come from the preset, not launcher overrides - #96
fix(launch): tier backend formats come from the preset, not launcher overrides#96ryan-lempka wants to merge 1 commit into
Conversation
decb81e to
e6bc8f0
Compare
WalkthroughDeterministic routing now takes per-tier wire formats from the selected preset. The strong tier defaults to ChangesDeterministic routing format ownership
Estimated code review effort: 2 (Simple) | ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
ac3b173 to
6f5c10b
Compare
elyasmnvidian
left a comment
There was a problem hiding this comment.
This fixes a real problem: launch codex and launch openclaw were sending the Claude strong model over OpenAI Chat Completions, which drops cache_control and quietly turns off prompt caching. Moving the format onto the preset (strong = AUTO) fixes it, and launch claude still behaves exactly as before. I checked the resolver: anthropic/claude-opus-4.7 matches the anthropic/ prefix, so AUTO picks Anthropic Messages with no startup probe and keeps the cache markers. Small, clean change. GitHub now shows it merges cleanly into main after the rebase.
One thing to confirm before merge. It lives in a file this PR does not change, so I can't leave it on a line: switchyard/lib/profiles/plan_execute_presets.py line 76 has the same bug you're fixing here. The plan-execute planner is anthropic/claude-opus-4.6 (a Claude model) but is pinned to format=BackendFormat.OPENAI, so the plan-execute path drops cache_control on its planner exactly like codex/openclaw did. If switchyard launch ... --plan-execute ships with these defaults, its planner loses prompt caching too. Can you confirm whether that path ships? If it does, the planner tier needs the same AUTO treatment.
While tracing that: build_plan_execute_config in route_builder.py still takes a backend_format argument, but nothing in the app calls it anymore (only tests do), so that argument and its docstring now describe code that never runs. Worth deleting in a follow-up.
Two smaller notes are on the lines themselves.
…overrides Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
6f5c10b to
0e5b488
Compare
Summary
The launcher route builder rebuilt every deterministic tier with caller-supplied
backend_format/strong_backend_formatparams, discarding the preset's per-tier formats. The preset's strong tier is a Claude model, solaunch codexandlaunch openclawforced it onto OpenAI format, which stripscache_controland silently disables prompt caching.The preset now owns tier formats: strong is
AUTO(resolves to Anthropic by model prefix; probes otherwise), weak and classifier stayOPENAI. Both override params are deleted.launch claudeis unchanged (it already overrode toAUTO);switchyard serveis untouched (YAML formats come from the config schema, not the preset). Launchers never exposed a user-facing format flag, so no capability is removed; explicit per-tier formats remain available via--routing-profiles.How tested
uv run ruff check .cleanuv run mypy switchyardcleanuv run pytest tests/green (1998 passed; builder tests now assert formats are preset-owned)tests/test_launch_preset_cache_e2e.py: real builder output, real profile build, real HTTP upstream — asserts the strong tier's wire body carriescache_controlephemeral markers on the zero-flag launcher path, and the weak tier's does not.Checklist
snake_caseof the primary class.switchyard/__init__.py.__all__if intended for downstream use. (none added)--helpupdated if customer-facing surface changed. (no surface change)