Structured output: support response_format: json_object for providers without strict json_schema

[daveycodez]

Summary

outputSchema always sends response_format: { type: "json_schema", strict: true }. Providers that accept response_format: { type: "json_object" } but do not support strict structured outputs reject the request, so those models are unreachable through the library — even though the OpenRouter SDK, and TanStack AI's own core types, already model json_object.

There is no opt-out: setting modelOptions.responseFormat = { type: "json_object" } alongside outputSchema does not take effect (see Why the override does not work — it is a spread-ordering issue, not a validation one).

Versions: @tanstack/ai@0.42.0, @tanstack/ai-openrouter@0.15.10, bundled @openrouter/sdk@0.13.20.

Reproduction

import { chat } from '@tanstack/ai'
import { openRouterText } from '@tanstack/ai-openrouter'
import { z } from 'zod'

// qwen3.7-flash declares response_format + tools, but NOT structured_outputs
await chat({
  adapter: openRouterText('qwen/qwen3.7-flash'),
  messages: [{ role: 'user', content: 'Name three colors as json.' }],
  outputSchema: z.object({ colors: z.array(z.string()) }),
})

Result:

Error: Provider returned error
    at runAgenticStructuredOutput (@tanstack/ai/dist/esm/activities/chat/index.js:1671:17)

The upstream detail never reaches the caller. The actual provider response is:

400 invalid_parameter_error
'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'

(That message is the provider's reply to OpenRouter's own downgrade attempt; the point is that the caller sees none of it.)

Scope

Against the live OpenRouter model list (GET /api/v1/models, 341 models at time of writing):

supported_parameters	count
response_format and structured_outputs	258
response_format but not structured_outputs	28
neither	55

Those 28 are usable today only if the caller gives up outputSchema entirely. Verified individually via the API: qwen/qwen3.7-flash, qwen/qwen3-30b-a3b-thinking-2507, inclusionai/ring-2.6-1t, aion-labs/aion-3.0, z-ai/glm-5-turbo, minimax/minimax-m2.1, ibm-granite/granite-4.0-h-micro, and others.

Why this is worth fixing: qwen/qwen3.7-flash

Cheap, capable, and currently unreachable. Our own benchmark on a card-classification task with a ~14.5k-token cached prompt, measured against google/gemini-3.1-flash-lite as baseline (these are our numbers, not a library measurement):

• quality: statistically tied — within measurement noise on a 25-case suite
• cost: $0.000062/call vs $0.002831 — roughly 45x cheaper
• latency: comparable p50, tighter p90

We had to drop to a hand-rolled direct-to-OpenRouter call to use it at all.

Root cause

outputSchema routes to a hardcoded json_schema

chat() sends any schema-bearing call into runAgenticStructuredOutput (packages/ai/src/activities/chat/index.ts:2841), which calls the adapter's structured-output method. Both OpenRouter implementations hardcode the format:

• packages/ai-openrouter/src/adapters/text.ts:212 — structuredOutput
• packages/ai-openrouter/src/adapters/text.ts:311 — structuredOutputStream

responseFormat: {
  type: 'json_schema',
  jsonSchema: { name: 'structured_output', schema: jsonSchema, strict: true },
},

This is not OpenRouter-specific — @tanstack/openai-base does the same at src/adapters/chat-completions-text.ts:184, :329, and :1145. It looks like a library-level design point rather than an adapter quirk, which is why this is filed against the core behaviour.

Why the override does not work

mapOptionsToRequest (ai-openrouter/src/adapters/text.ts:1142) spreads caller options into the request:

const request = { ...restModelOptions, model: ..., messages, ... }

responseFormat is a declared caller-settable key (src/text/text-provider-options.ts:86, in OpenRouterBaseOptions), so it lands in restModelOptions and flows through. But the structured-output methods then spread that result first and write the literal after:

const { streamOptions: _so, tools: _t, ...cleanParams } = chatRequest
...
chatRequest: { ...cleanParams, stream: true, responseFormat: { type: 'json_schema', ... } }
//             ^^^^^^^^^^^^^ caller's responseFormat lands here …      ^^^ … and is overwritten here

So the caller's value is silently discarded. Whether an explicit modelOptions.responseFormat should win seems worth deciding explicitly either way — right now it neither wins nor warns.

The json_object path already exists end to end

Nothing needs inventing on the wire:

• @openrouter/sdk@0.13.20 — FormatJsonObjectConfig = { type: "json_object" } is a member of the ResponseFormat union (esm/models/chatrequest.d.ts).
• @tanstack/ai — the core ResponseFormat interface already declares type: 'json_object' | 'json_schema' with docs for both (packages/ai/src/types.ts:749-758). As far as I can tell that interface isn't referenced by the chat path, so it's currently descriptive only.

The adapter has capability data, but not this distinction

ai-openrouter/src/model-meta.ts carries a per-model supports array derived from OpenRouter's supported_parameters. It contains responseFormat, but structuredOutputs does not appear anywhere in the file (0 occurrences; the full capability vocabulary is frequencyPenalty, logitBias, logprobs, maxCompletionTokens, parallelToolCalls, presencePenalty, reasoning, responseFormat, seed, stop, temperature, toolChoice, topLogprobs, topP).

So the table cannot currently express "supports response_format but not structured_outputs" — precisely the case that breaks. OpenRouter exposes this per-model and per-endpoint, so the generator that produces model-meta.ts could pick it up. Note the two are not equivalent: a model's top-level supported_parameters is a union across endpoints, so endpoint-level data is the accurate source.

The provider error is swallowed

chat/index.ts:1671 rethrows finalizationError, preserving cause when it is set. But cause is only populated on the fallbackStructuredOutputStream path, which captures the raw adapter error via the onAdapterError callback (chat/index.ts:2139-2151, :2310). The OpenRouter adapter implements structuredOutputStream natively, so that fallback never runs, fallbackAdapterError stays undefined, and the error is reconstructed from the RUN_ERROR event — whose wire shape carries only message and code. The provider's status, body, and message are lost.

This is arguably the more painful half of the bug: the failure is diagnosable in about a minute with the upstream body, and close to opaque without it.

Proposal

An explicit mode on the structured-output path:

structuredOutputMode?: 'json_schema' | 'json_object' | 'auto'  // default: 'json_schema'

• json_schema — today's behaviour, unchanged default.
• json_object — send response_format: { type: 'json_object' }, then validate the returned text against the same schema client-side. The validation machinery already exists: runAgenticStructuredOutput builds a validate callback from the Standard Schema and runs it inside the engine (chat/index.ts:2996-3001), independent of how the provider was asked. Only the request-side format and the "provider guarantees shape" assumption change.
• auto — pick from declared capabilities, falling back to json_object when a model supports response_format but not structured_outputs. This needs structuredOutputs added to the model-meta.ts capability vocabulary first.

Two things worth designing deliberately:

1. The literal word "json". At least some providers require it in the messages when using json_object (that is the 400 above). The library would need to append a line to the system prompt — visible and documented, since it mutates the user's prompt — or document the requirement and fail with a clear error rather than a provider 400.
2. Strictness is no longer free. Under json_object the model can return well-formed JSON of the wrong shape. Client-side validation catches it, but the failure mode moves from request-time to response-time, and a retry policy may be worth specifying.

Alternative: tool-calling as a structured-output strategy

Worth mentioning because all 28 affected models support tools. As far as I can tell the library has no tool-based structured-output path today — response_format is the only strategy, and supportsCombinedToolsAndSchema (openai-base/src/adapters/chat-completions-text.ts:1186) is about combining tools with a schema in one call, not about using a tool as the schema. It is also notably not implemented in the OpenRouter adapter, so those models take the separate-finalization round-trip.

A forced single-tool call (tool_choice: { type: 'function', function: { name: ... } }) with the schema as the tool's parameters is the standard portable fallback, and would cover models that support neither response_format nor structured_outputs — a further 55 in the table above. Bigger change; flagging as a direction, not a request.

Notes on what I verified

To keep the report honest about its own claims:

• Verified in source: every file/line reference above, the spread ordering, the absent structuredOutputs capability, the cause-dropping path, the SDK's FormatJsonObjectConfig.
• Verified against the live OpenRouter API: all supported_parameters counts and the per-model results.
• Our own measurements, not verified here: the qwen3.7-flash cost/quality/latency numbers, and the original 400 body.
• A claim I could not reproduce, corrected: I had initially understood that every Alibaba-served Qwen was affected. Not so — qwen/qwen3.6-flash and qwen/qwen3.5-flash-02-23 both declare structured_outputs: true on their Alibaba endpoints today. The verified affected list is the one above. Separately, poolside/laguna-xs-2.1 supports neither response_format nor structured_outputs, so json_object alone would not rescue it.
• A non-bug, for completeness: I had also suspected that a chat() call without outputSchema fired no middleware and returned {}. That is just async-generator semantics — chat() without outputSchema and without stream: false returns a lazy AsyncGenerator, and awaiting it yields the generator object itself (JSON.stringify → {}) without ever running the body. Iterating it, or passing stream: false for a Promise<string>, both work correctly. No issue here — noting it only so it isn't read into the report above.