Deprecate the old response_schema#47320
Conversation
|
[For maintainers] Suggested jobs to run (before merge) run-slow: gemma4_unified |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Rocketknight1
force-pushed
the
deprecate_response_schema
branch
from
5fa27f0 to
e07dae7
Compare
|
Should be ready for review! This touches the serving code a bit so cc @SunMarc @remi-or @LysandreJik. Other than that it's mostly mechanical changes and deleting references/imports, plus updating the Gemma4 conversion scripts to only add new-style 🚨 Don't merge it yet, though - slow tests for Gemma4 will break until we've merged the PRs to update the model repos too |
CI recapDashboard: View test results in Grafana |
2 participants
We prototyped response parsing using a
response_schemaattribute, but this had a few issues in practice: It was fragile, the regexes were hard to write, and it required a complete message to parse, so it didn't support streaming.We've got a new and much more robust system now using the
response_templateattribute. The only models usingresponse_schemaare Gemma4, and there are PRs open to all of them to update that. Once that update is in, we should be able to merge this and dropresponse_schemaentirely, andresponse_templateis then the official and only way to do response parsing, and we can start adding those templates everywhere!