Skip to content

R8a: make reasoning a graded level (off/low/medium/high) across all 5 providers - #167

Merged
dovvnloading merged 1 commit into
mainfrom
r8a/graded-reasoning-levels
Jul 28, 2026
Merged

R8a: make reasoning a graded level (off/low/medium/high) across all 5 providers#167
dovvnloading merged 1 commit into
mainfrom
r8a/graded-reasoning-levels

Conversation

@dovvnloading

Copy link
Copy Markdown
Owner

Problem

The reasoning control was a bare bool (Thinking/Quick), applied only to Ollama/Llama.cpp models whose name happened to contain "qwen3" or "deepseek" - and completely absent for every cloud provider, since the composer hid the control entirely outside local mode.

Research

Looked at how Ollama and each cloud provider actually expose adjustable reasoning effort before touching any code - the mechanisms are genuinely different per provider and per model generation, not one shared knob:

  • OpenAI: reasoning_effort (minimal/low/medium/high/xhigh, model-dependent)
  • Anthropic: thinking.budget_tokens on older models; a newer effort param on Opus 4.7+ that rejects budget_tokens outright (a real 400 error) - mutually exclusive request shapes, not two names for the same thing
  • Gemini: thinkingBudget (integer, 2.5-series) vs thinkingLevel (string, Gemini 3) - same story, different generations
  • Ollama: think is a bool for qwen3/deepseek/qwq, but a required string level for gpt-oss - two different mechanisms on the same provider

Change

One shared off/low/medium/high vocabulary now drives all five, each mapped to whatever that provider/model actually supports - including being honest where a model can't truly grade: Ollama/Llama.cpp's bool-only models (qwen3/deepseek/qwq) get a real second axis via a plain-language reasoning-budget system-prompt hint layered on top of the bool, documented as prompt-based guidance rather than a fake graded API param.

The composer's reasoning control now shows real capability detection (does the currently resolved model support reasoning at all) instead of "is this a local provider" - cloud providers get the control whenever their active model genuinely supports it. Settings > Ollama/Llama.cpp pages get the same 4-option radio group the composer uses; adding equivalent Settings pages for the three cloud providers is left for a follow-up, since the composer's own popover already reads/writes their persisted level in the meantime.

Existing session.dat files migrate the old 2-value mode to the new scale (Quick → off, Thinking → high) rather than resetting silently; the three new cloud fields default to off (extended thinking on a paid API is an opt-in cost/latency tradeoff, never a silent default) while the local providers keep defaulting to high (local compute is free to the user).

Test plan

  • Backend: 1021 passed, including a new dedicated test file for every provider's mapping function (25 tests) plus updated coverage across composer/settings
  • Frontend: tsc --noEmit clean, lint 0 errors, 794 tests passed, build succeeds
  • Verified live against the real, already-persisted session.dat: the migration correctly produced ollama_reasoning_level=high/llama_cpp_reasoning_level=high (from the prior Thinking value) and all three new cloud fields at off
  • Verified live in the browser: the composer's Reasoning popover renders all four real options against the live backend, correctly shows "High" as active, and both Settings > Ollama and Settings > Llama.cpp pages show the same 4-option radio group in sync with the composer

… providers

The reasoning control was a bare bool (Thinking/Quick), applied only to
Ollama/Llama.cpp models whose name happened to contain "qwen3" or
"deepseek", and completely absent for every cloud provider - the composer
hid the control entirely outside local mode. Researched how Ollama and
each cloud provider actually expose adjustable reasoning effort before
touching any code, since the mechanisms are genuinely different per
provider and per model generation, not one shared knob:

- OpenAI: reasoning_effort (minimal/low/medium/high/xhigh, model-dependent)
- Anthropic: thinking.budget_tokens on older models; a newer `effort`
  param on Opus 4.7+ that REJECTS budget_tokens outright (a real 400
  error) - these are mutually exclusive request shapes, not two names for
  the same thing, so the model string has to be checked before either is
  sent
- Gemini: thinkingBudget (integer, 2.5-series) vs thinkingLevel (string,
  Gemini 3) - same story, different generations
- Ollama: think is a bool for qwen3/deepseek/qwq, but a REQUIRED string
  level for gpt-oss - two different mechanisms on the same provider

One shared off/low/medium/high vocabulary now drives all five, each
mapped to whatever that provider/model actually supports - including
being honest where a model can't truly grade: Ollama/Llama.cpp's bool-
only models (qwen3/deepseek/qwq) get a real second axis via a plain-
language reasoning-budget system-prompt hint layered on top of the bool,
documented as prompt-based guidance rather than a fake graded API param.

The composer's reasoning control now shows real capability detection
(does the CURRENTLY RESOLVED model support reasoning at all) instead of
"is this a local provider" - cloud providers get the control whenever
their active model genuinely supports it. Settings > Ollama/Llama.cpp
pages get the same 4-option radio group the composer uses; adding
equivalent Settings pages for the three cloud providers is left for a
follow-up, since the composer's own popover already reads/writes their
persisted level in the meantime.

Existing session.dat files migrate the old 2-value mode to the new scale
(Quick -> off, Thinking -> high) rather than resetting silently; the
three new cloud fields default to off (extended thinking on a paid API
is an opt-in cost/latency tradeoff, never a silent default) while the
local providers keep defaulting to high (local compute is free to the
user). Verified live: the real, already-migrated session.dat correctly
shows ollama_reasoning_level=high/llama_cpp_reasoning_level=high (from
the prior Thinking value) and all three cloud fields at off, and the
composer's Reasoning popover renders all four real options against the
live backend.
@dovvnloading
dovvnloading merged commit ed467c9 into main Jul 28, 2026
2 checks passed
@dovvnloading
dovvnloading deleted the r8a/graded-reasoning-levels branch July 28, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant