Python: Fix OpenAIChatCompletionClient passing raw JSON-Schema dict response_format through unwrapped#7199
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes Python OpenAIChatCompletionClient handling of response_format when callers pass a raw JSON-Schema dict by normalizing/wrapping it into the Chat Completions json_schema envelope (bringing it in line with the Responses-based client), and adds unit tests for the new behavior. It also introduces a new .NET “AgentLearn” learning project plus several large research/ markdown documents and repo-level developer settings.
Changes:
- Python: normalize raw JSON-Schema
dictresponse_formatvalues inOpenAIChatCompletionClientand add tests for wrapping/title→name behavior. - .NET: add
dotnet/learn/AgentLearnsample app (workflows + HITL + tests) and adotnet/samples/run.bashhelper script. - Docs/config: add
research/documentation, newdotnet/learndocs/config, and repo-level VS Code/default-solution and gitignore updates.
Reviewed changes
Copilot reviewed 40 out of 41 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| research/README.md | Adds a README describing the new research/ folder structure. |
| research/python/agentic-capabilities.md | Adds Python agentic-capabilities research document. |
| research/dotnet/agentic-capabilities.md | Adds .NET agentic-capabilities research document. |
| research/agentic-capabilities-overview.md | Adds cross-language agentic-capabilities overview research document. |
| python/packages/openai/tests/openai/test_openai_chat_completion_client.py | Adds tests for raw-schema wrapping and passthrough behaviors. |
| python/packages/openai/agent_framework_openai/_chat_completion_client.py | Normalizes dict response_format by wrapping raw JSON Schema into json_schema. |
| dotnet/samples/run.bash | Adds a helper script to run a .NET project with .env loaded. |
| dotnet/learn/README.md | Adds README for the new dotnet/learn area. |
| dotnet/learn/Directory.Packages.props | Disables central package management within dotnet/learn. |
| dotnet/learn/Directory.Build.targets | Adds empty targets file to isolate from parent build targets. |
| dotnet/learn/Directory.Build.props | Adds empty props file to isolate from parent build props. |
| dotnet/learn/CLAUDE.md | Adds contributor guidance for the dotnet/learn area. |
| dotnet/learn/AgentLearn/tests/integration/StoryGeneratorWorkflowTests.cs | Adds integration tests for StoryGenerator HITL workflow. |
| dotnet/learn/AgentLearn/tests/integration/MockChatClientHelper.cs | Adds a helper to mock IChatClient responses for tests. |
| dotnet/learn/AgentLearn/tests/integration/JokeWriterWorkflowTests.cs | Adds integration tests for JokeWriter workflows (single/sequential/concurrent). |
| dotnet/learn/AgentLearn/tests/integration/AgentLearn.IntegrationTests.csproj | Adds integration test project definition. |
| dotnet/learn/AgentLearn/TaskKinds/StoryGenerator/StoryGenerator.http | Adds REST client requests for StoryGenerator. |
| dotnet/learn/AgentLearn/TaskKinds/StoryGenerator/StoryGenerator.cs | Adds StoryGenerator task kind (agents/workflow/handler/DI). |
| dotnet/learn/AgentLearn/TaskKinds/JokeWriter/JokeWriter.http | Adds REST client requests for JokeWriter. |
| dotnet/learn/AgentLearn/TaskKinds/JokeWriter/JokeWriter.cs | Adds JokeWriter task kind (models/executors/agents/workflows/handler/DI). |
| dotnet/learn/AgentLearn/Services/ITaskHandler.cs | Adds task-handler abstraction for routing TaskKinds. |
| dotnet/learn/AgentLearn/Services/ChatClientFactory.cs | Adds provider-based IChatClient factory + mock chat client. |
| dotnet/learn/AgentLearn/run.sh | Adds script to start Jaeger and run AgentLearn. |
| dotnet/learn/AgentLearn/Properties/launchSettings.json | Adds local launch settings for HTTP/HTTPS profiles. |
| dotnet/learn/AgentLearn/Program.cs | Adds DI setup, OTel tracing, DevUI, and OpenAI-compatible endpoints. |
| dotnet/learn/AgentLearn/Models/TaskResponse.cs | Adds DTO for /task responses. |
| dotnet/learn/AgentLearn/Models/TaskRequest.cs | Adds DTO for /task requests. |
| dotnet/learn/AgentLearn/Models/TaskKind.cs | Adds TaskKind enum for handler selection. |
| dotnet/learn/AgentLearn/Models/LlmOptions.cs | Adds LLM provider config options DTO. |
| dotnet/learn/AgentLearn/Models/AgentMode.cs | Adds workflow mode enum (Single/Sequential/Concurrent). |
| dotnet/learn/AgentLearn/Extensions/AIAgentBuilderExtensions.cs | Adds tool-invocation logging middleware for agents. |
| dotnet/learn/AgentLearn/Controllers/TaskController.cs | Adds /task endpoint controller. |
| dotnet/learn/AgentLearn/CLAUDE.md | Adds project-specific conventions for AgentLearn. |
| dotnet/learn/AgentLearn/appsettings.json | Adds baseline configuration including LLM defaults. |
| dotnet/learn/AgentLearn/appsettings.Development.json | Adds dev configuration (debug logging). |
| dotnet/learn/AgentLearn/AgentLearn.csproj | Adds AgentLearn web project definition and package references. |
| dotnet/learn/AgentLearn.sln | Adds a Visual Studio solution for AgentLearn. |
| dotnet/learn/.gitignore | Adds ignores for .env* files under dotnet/learn. |
| CLAUDE.md | Adds repo-level Claude guidance file. |
| .vscode/settings.json | Sets a repo-wide default .NET solution. |
| .gitignore | Ignores .workspaces/ personal VS Code workspace files. |
…esponse_format through unwrapped
Raw schema dicts (e.g. {"type": "object", ...}) were forwarded to the
Chat Completions API verbatim, which OpenAI rejects with a 400. The
Responses client already auto-wraps the same input. Mirror its raw-schema
detection (primitive types / schema keywords), wrap into the
{"type": "json_schema", "json_schema": {...}} envelope with
additionalProperties: false injection and title -> name promotion, and
leave already-valid response_format dicts untouched.
Fixes microsoft#7197
(cherry picked from commit dce5c3b)
dce5c3b to
2351bd5
Compare
|
Note for reviewers: the earlier review threads (research/, dotnet/learn/, .vscode/settings.json, AgentLearn.csproj) referenced unrelated files accidentally included from a stale fork base — the branch has been rebuilt as a single commit on top of main and those files are no longer part of this PR (now 2 files: |
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
this needs a integration test, because if I recall correctly there are differences in the format for the structured output between Chat Completions and Responses API's.
There was a problem hiding this comment.
Added in 47e3a08 — a response_format_raw_json_schema param in test_integration_options for both clients, passing the same bare WeatherDigest schema dict (with title, without additionalProperties) so the wrapping, title→name promotion, and additionalProperties: false injection paths are all exercised against the live APIs.
You're right that the wire formats differ — Chat Completions nests under response_format.json_schema while Responses flattens into text.format. Both new params pass live (verified locally with a real key): the identical raw input round-trips through both APIs and yields parsed structured output (response.value validated). The Responses-side param also adds live coverage for the pre-existing raw-schema handling in _convert_response_format, which previously had none — so if either implementation drifts, one of the twin params breaks.
Python Test Coverage Report •
Python Unit Test Overview
|
||||||||||||||||||||||||||||||
…rmat dicts Adds a response_format_raw_json_schema param to test_integration_options in both the Chat Completions and Responses client test suites, proving the same bare schema dict (title set, additionalProperties omitted) round-trips through both live APIs and yields parsed structured output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Motivation and Context
Raw JSON-Schema dicts (e.g.
{"type": "object", "properties": {...}}) passed asresponse_formatare forwarded to the Chat Completions API verbatim, and OpenAI rejects them with a 400 (Invalid value: 'object'). The Responses-basedOpenAIChatClientalready auto-wraps the identical input (see_convert_response_format), so the two clients disagree on the same input. This bites the declarative path in particular, where the loader passesoutputSchema.to_json_schema()output as a bare dict.Description
_normalize_response_format_dict: mirrors the Responses client's raw-schema detection (JSON-Schema primitive types / schema keywords), wraps into the{"type": "json_schema", "json_schema": {...}}envelope withadditionalProperties: falseinjection andtitle→namepromotion.response_formattype (json_schema/json_object/text) pass through unchanged — the existingtest_response_format_dict_passthroughstill passes.json_objectpassthrough.Contribution Checklist
test_openai_chat_completion_client.py: 77 passed, 20 skipped)