R8a: make reasoning a graded level (off/low/medium/high) across all 5 providers - #167
Merged
Merged
Conversation
… providers The reasoning control was a bare bool (Thinking/Quick), applied only to Ollama/Llama.cpp models whose name happened to contain "qwen3" or "deepseek", and completely absent for every cloud provider - the composer hid the control entirely outside local mode. Researched how Ollama and each cloud provider actually expose adjustable reasoning effort before touching any code, since the mechanisms are genuinely different per provider and per model generation, not one shared knob: - OpenAI: reasoning_effort (minimal/low/medium/high/xhigh, model-dependent) - Anthropic: thinking.budget_tokens on older models; a newer `effort` param on Opus 4.7+ that REJECTS budget_tokens outright (a real 400 error) - these are mutually exclusive request shapes, not two names for the same thing, so the model string has to be checked before either is sent - Gemini: thinkingBudget (integer, 2.5-series) vs thinkingLevel (string, Gemini 3) - same story, different generations - Ollama: think is a bool for qwen3/deepseek/qwq, but a REQUIRED string level for gpt-oss - two different mechanisms on the same provider One shared off/low/medium/high vocabulary now drives all five, each mapped to whatever that provider/model actually supports - including being honest where a model can't truly grade: Ollama/Llama.cpp's bool- only models (qwen3/deepseek/qwq) get a real second axis via a plain- language reasoning-budget system-prompt hint layered on top of the bool, documented as prompt-based guidance rather than a fake graded API param. The composer's reasoning control now shows real capability detection (does the CURRENTLY RESOLVED model support reasoning at all) instead of "is this a local provider" - cloud providers get the control whenever their active model genuinely supports it. Settings > Ollama/Llama.cpp pages get the same 4-option radio group the composer uses; adding equivalent Settings pages for the three cloud providers is left for a follow-up, since the composer's own popover already reads/writes their persisted level in the meantime. Existing session.dat files migrate the old 2-value mode to the new scale (Quick -> off, Thinking -> high) rather than resetting silently; the three new cloud fields default to off (extended thinking on a paid API is an opt-in cost/latency tradeoff, never a silent default) while the local providers keep defaulting to high (local compute is free to the user). Verified live: the real, already-migrated session.dat correctly shows ollama_reasoning_level=high/llama_cpp_reasoning_level=high (from the prior Thinking value) and all three cloud fields at off, and the composer's Reasoning popover renders all four real options against the live backend.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The reasoning control was a bare bool (Thinking/Quick), applied only to Ollama/Llama.cpp models whose name happened to contain "qwen3" or "deepseek" - and completely absent for every cloud provider, since the composer hid the control entirely outside local mode.
Research
Looked at how Ollama and each cloud provider actually expose adjustable reasoning effort before touching any code - the mechanisms are genuinely different per provider and per model generation, not one shared knob:
reasoning_effort(minimal/low/medium/high/xhigh, model-dependent)thinking.budget_tokenson older models; a newereffortparam on Opus 4.7+ that rejectsbudget_tokensoutright (a real 400 error) - mutually exclusive request shapes, not two names for the same thingthinkingBudget(integer, 2.5-series) vsthinkingLevel(string, Gemini 3) - same story, different generationsthinkis a bool for qwen3/deepseek/qwq, but a required string level for gpt-oss - two different mechanisms on the same providerChange
One shared
off/low/medium/highvocabulary now drives all five, each mapped to whatever that provider/model actually supports - including being honest where a model can't truly grade: Ollama/Llama.cpp's bool-only models (qwen3/deepseek/qwq) get a real second axis via a plain-language reasoning-budget system-prompt hint layered on top of the bool, documented as prompt-based guidance rather than a fake graded API param.The composer's reasoning control now shows real capability detection (does the currently resolved model support reasoning at all) instead of "is this a local provider" - cloud providers get the control whenever their active model genuinely supports it. Settings > Ollama/Llama.cpp pages get the same 4-option radio group the composer uses; adding equivalent Settings pages for the three cloud providers is left for a follow-up, since the composer's own popover already reads/writes their persisted level in the meantime.
Existing
session.datfiles migrate the old 2-value mode to the new scale (Quick → off, Thinking → high) rather than resetting silently; the three new cloud fields default to off (extended thinking on a paid API is an opt-in cost/latency tradeoff, never a silent default) while the local providers keep defaulting to high (local compute is free to the user).Test plan
tsc --noEmitclean, lint 0 errors, 794 tests passed, build succeedssession.dat: the migration correctly producedollama_reasoning_level=high/llama_cpp_reasoning_level=high(from the priorThinkingvalue) and all three new cloud fields atoff