Python: Hosted LRA draft updates#7122
Conversation
- azure-ai-agentserver-core: 2.0.0b5 -> 2.0.0b7 (resilient task primitive, event streaming API) - azure-ai-agentserver-invocations: 1.0.0b3 -> 1.0.0b6 (bug fixes, requires core >=b7) - azure-ai-agentserver-responses: 1.0.0b7 -> 1.0.0b8 (resilient background responses, steerable conversations, developer checkpoints) New versions are unreleased on PyPI; source them via direct URL to the pinned commit in azure-sdk-for-python via [tool.uv.sources]. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix resilient_background rename: ResponsesServerOptions no longer accepts durable_background (renamed to resilient_background in b8). The old name caused a silent TypeError that disabled crash recovery in hosted environments. - Add steerable_conversations parameter to ResponsesHostServer.__init__. When True, concurrent turns on the same conversation chain are queued rather than rejected with 409 conversation_locked. The flag is threaded into ResponsesServerOptions and composed with resilient_background in the auto-enable path for hosted environments. - Fix streaming cancellation terminal: both _handle_inner_agent and _handle_inner_workflow now emit response.completed before returning when cancellation_signal fires. Per spec §10, the handler must emit a terminal event; the framework overrides completed → cancelled when context.client_cancelled=True (real client cancel), and leaves it as completed for steering pressure (client_cancelled=False). Previously returning without a terminal caused the framework to force failed, which broke the steered-turn chain. - Add comment in _handle_inner_workflow explaining the relationship between previous_response_id and conversation_chain_id for steerable turns (they resolve to the same key, so the existing restore logic is already correct). - Update tests: rename resilient_background tests, update cancellation test to assert response.completed is emitted, add three new tests for steerable_conversations threading and precedence rules. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix async generator handler signature (b8 compatibility): _handle_response was async def returning AsyncIterable. The b8 SDK passes the raw handler result directly to _intercept_checkpoints which does 'async for raw in handler_iterator' — this fails on a coroutine with TypeError (caught as B8 pre-creation error), silently producing empty responses. Changed to async generator (using yield) so calling it returns an AsyncGenerator immediately iterable without awaiting. Update 01_basic sample (durable_background -> resilient_background): main.py comments and README referenced the old durable_background field name (renamed to resilient_background in b8). Updated both to use the correct API and added steerable_conversations documentation. Add steering integration test (test_steering.py): Self-contained end-to-end test using Hypercorn on a loopback port and concurrent asyncio tasks. Demonstrates the full steering flow: turn 1 streams, turn 2 arrives while turn 1 is in_progress and gets status=queued, turn 1 emits response.completed (cancellation terminal fix), turn 2 runs and completes. Requires clean ~/.agentserver state (no stale task files) on first run; uses unique UUID per test run. Also add pre-existing sample exclusions to pyrightconfig.samples.json for monty/tools samples not installed in the dev environment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replaces the mock slow agent with a real FoundryChatClient + Agent using FOUNDRY_PROJECT_ENDPOINT and AZURE_AI_MODEL_DEPLOYMENT_NAME env vars, consistent with the 01_basic sample. Loads .env via python-dotenv. Asks the agent to count to 100 (produces a long streaming response), then steers with a joke request after 2s. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
resilient_background=True was previously auto-enabled when running in a
hosted Foundry environment. This is now opt-in: callers must pass it
explicitly via ResponsesServerOptions.
server = ResponsesHostServer(
agent,
options=ResponsesServerOptions(resilient_background=True),
)
Reasoning: auto-enabling crash recovery silently can cause unexpected
behaviour for agents with non-idempotent side effects. Requiring explicit
opt-in makes the behaviour transparent and intentional.
Updates:
- _responses.py: remove auto-enable logic and AgentConfig check
- test_responses.py: replace auto-enable tests with a single test
asserting resilient_background is never auto-enabled
- 01_basic/main.py: rewrite comment to describe opt-in pattern
- 01_basic/README.md: rewrite durability section as opt-in
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add client.py — a two-terminal steering demonstration. Run main.py in one terminal and client.py in a second to see steering in action with a real Foundry agent: - Turn 1 streams the agent counting to 50 - 2 s later, turn 2 arrives on the same conversation with a different question - Turn 2 is accepted with status=queued (not HTTP 409) - Turn 1's handler is cancelled and emits response.completed with partial output - Turn 2 runs and its answer is printed client.py accepts an optional SERVER_URL argument so it can be pointed at a deployed instance as well as localhost. Also enable steerable_conversations=True in main.py and update README with the two-terminal usage instructions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- 01_basic: restore to minimal server-only sample; remove client.py, test_steering.py, and all steering/resilience content from README. Add brief 'Next steps' links to 13 and 14. - 13_steering: new sample demonstrating steerable_conversations=True. main.py starts the server; client.py is an interactive two-terminal demo that sends two concurrent turns and shows the queued/steered flow. - 14_resilience: new sample demonstrating resilient_background=True. main.py starts the server with crash recovery enabled. README walks through testing crash recovery locally (kill the server mid-response, restart, and poll the response to see it recover). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ence The previous README asked users to Ctrl+C the server at the right moment while counting to 1000, which was unreliable and the model often refused the prompt anyway. demo.py orchestrates everything automatically: 1. Starts main.py as a subprocess 2. Sends a background request (returns in_progress immediately) 3. Kills the server after 2s (no timing pressure) 4. Restarts the server (recovery scanner fires) 5. Polls until the response completes and prints it Uses a substantive prose prompt (explain how the internet works) that reliably generates a long response without model refusals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
giles17
added
documentation
github-actions
Bot
changed the title
There was a problem hiding this comment.
Pull request overview
This PR updates the Python “Foundry hosted agents / Responses” hosting experience by adding new resiliency-focused samples and extending the agent-framework-foundry-hosting Responses adapter to better support durable workflow checkpointing and updated SDK behaviors (including MCP output shape changes). It also hardens observability tool-definition serialization (especially for Azure SDK “model” tool objects) and updates tests accordingly.
Changes:
- Add new hosted Responses samples for steerable conversations, crash recovery, and a deterministic durable-workflow recovery demo (
13_steering,14_resilience,15_workflow_resilience). - Update
ResponsesHostServerworkflow handling to synchronize workflow checkpoints with durable response snapshots and align output-item behavior with the current SDK (e.g., MCP output emitted as separate items). - Improve OTel tool-definition serialization to handle Azure SDK Mapping-based tool models and to skip (with warnings) individual unserializable tools rather than failing telemetry.
Show a summary per file
| File | Description |
|---|---|
| python/samples/04-hosting/foundry-hosted-agents/responses/15_workflow_resilience/requirements.txt | Adds minimal deps for the deterministic durable-workflow recovery demo. |
| python/samples/04-hosting/foundry-hosted-agents/responses/15_workflow_resilience/README.md | Documents the durable workflow recovery scenario and how to run it. |
| python/samples/04-hosting/foundry-hosted-agents/responses/15_workflow_resilience/main.py | Implements a deterministic 4-stage workflow configured for resilient background execution. |
| python/samples/04-hosting/foundry-hosted-agents/responses/15_workflow_resilience/Dockerfile | Container entrypoint for hosting the workflow sample. |
| python/samples/04-hosting/foundry-hosted-agents/responses/15_workflow_resilience/demo.py | Automates host interruption/replacement and validates durable progress/output. |
| python/samples/04-hosting/foundry-hosted-agents/responses/15_workflow_resilience/.env.example | Optional knobs for state dir/stage delay and telemetry behavior. |
| python/samples/04-hosting/foundry-hosted-agents/responses/14_resilience/requirements.txt | Adds deps for the crash-recovery (resilient_background) sample. |
| python/samples/04-hosting/foundry-hosted-agents/responses/14_resilience/README.md | Explains crash recovery behavior and how the demo works. |
| python/samples/04-hosting/foundry-hosted-agents/responses/14_resilience/main.py | Minimal server enabling resilient_background=True. |
| python/samples/04-hosting/foundry-hosted-agents/responses/14_resilience/Dockerfile | Container entrypoint for the resilience sample. |
| python/samples/04-hosting/foundry-hosted-agents/responses/14_resilience/demo.py | Automates restart-based recovery and polls to completion. |
| python/samples/04-hosting/foundry-hosted-agents/responses/14_resilience/.env.example | Documents required env vars for the Foundry-connected sample. |
| python/samples/04-hosting/foundry-hosted-agents/responses/13_steering/requirements.txt | Adds deps for the steerable conversations sample. |
| python/samples/04-hosting/foundry-hosted-agents/responses/13_steering/README.md | Explains queued turns/steering behavior and how to run. |
| python/samples/04-hosting/foundry-hosted-agents/responses/13_steering/main.py | Minimal server enabling steerable_conversations=True. |
| python/samples/04-hosting/foundry-hosted-agents/responses/13_steering/Dockerfile | Container entrypoint for the steering sample. |
| python/samples/04-hosting/foundry-hosted-agents/responses/13_steering/client.py | Client that demonstrates steering by sending overlapping turns. |
| python/samples/04-hosting/foundry-hosted-agents/responses/13_steering/.env.example | Documents required env vars for the Foundry-connected sample. |
| python/samples/04-hosting/foundry-hosted-agents/responses/01_basic/README.md | Adds “Next steps” links to the new steering/resilience samples. |
| python/pyrightconfig.samples.json | Excludes additional sample scripts from pyright sample config. |
| python/pyproject.toml | Updates uv config (includes a new find-links entry). |
| python/packages/foundry/tests/foundry/test_foundry_chat_client.py | Adds regression coverage for OTel tool-definition serialization with code interpreter tools. |
| python/packages/foundry_hosting/tests/test_responses.py | Updates/expands tests for checkpoint synchronization, MCP output shape, and streaming-based behavior. |
| python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py | Implements workflow checkpoint synchronization + adjusts response handling/output tracking. |
| python/packages/core/tests/core/test_observability.py | Adds tests ensuring unserializable tool definitions are skipped with warnings. |
| python/packages/core/agent_framework/observability.py | Improves tool-definition serialization (Azure SDK models + partial failure handling). |
Review details
- Files reviewed: 26/27 changed files
- Comments generated: 2
- Review effort level: Low
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 5 | Confidence: 77%
✓ Correctness
The major refactoring of
_handle_responseintroduces a double-emit ofresponse.completedon two code paths: (1) whencancellation_signalfires, the cancellation block emitsresponse.completedat line 847 then breaks, but the post-loop code at line 855 unconditionally emits it again; (2) the consent-error path in_handle_inner_agentyieldsresponse.completedat line 932 and then returns, but since this event flows through the caller'sasync forloop, the post-loop code at line 855 emits a secondresponse.completed. These result in duplicate terminal SSE events on the stream. Two clear correctness issues: (1) a developer-local Windows path hardcoded in pyproject.toml that will break dependency resolution for all other contributors, and (2) a duplicated 'Deploying the Agent to Foundry' section in the 01_basic README. The test refactoring and new test additions appear logically sound. The uv.lock file has been updated to reference three azure-ai-agentserver packages from a local Windows developer path (C:/Users/bentho/src/long-running-agents-private-preview/artifacts/wheels) instead of PyPI. This makes the lock file unresolvable on any other machine, CI, or non-Windows environment. The sample code in 15_workflow_resilience is straightforward and has no correctness issues.
✓ Security Reliability
The main reliability concern is a double
response.completedemission on the cancellation path in_handle_response: line 847 emits it inside the cancellation handler, then afterbreakexecution falls through to line 855 which emits it again. Additionally, the_drive_workflow_streamfinally block has a narrowsuppressthat won't catch workflow runtime exceptions from already-completed tasks, potentially producing confusing traceback chaining. The most significant issue is a hardcoded developer-local Windows path committed topython/pyproject.toml(find-links), which will cause uv resolution failures or warnings for all other contributors/CI and leaks a developer username. There is also a duplicate 'Deploying the Agent to Foundry' section in the 01_basic README. The uv.lock file has been modified to source three critical azure-ai-agentserver packages from absolute Windows filesystem paths on a developer's local machine (C:/Users/bentho/src/long-running-agents-private-preview/artifacts/wheels). This will break dependency resolution in CI/CD, other developers' machines, and any non-local environment. The PR title indicates this is a draft, but this is a blocking reliability issue that also leaks internal filesystem structure. The sample code (demo.py, main.py) has no security or reliability concerns.
✓ Test Coverage
The PR adds significant new functionality around workflow checkpoint synchronization and durable recovery, with good test coverage for the core happy paths (synchronizer acknowledgement, recovery with persisted checkpoints, cancellation signals). However, there are notable gaps: the shutdown/
exit_for_recoverycode path (line 829-835) has no test at all, the_WorkflowCheckpointSynchronizer.close()error-handling logic (draining waiters/pending queue) is untested, and the non-workflow agent warning whenresilient_background=True(line 656-659) has no assertion verifying the warning is emitted. The_drive_workflow_streammethod's exception propagation from a failing workflow (non-StopAsyncIteration) lacks targeted coverage. The PR adds substantial new test coverage for durable workflow checkpoint features (synchronization, recovery, steering, resilience). The newTestDurableResponseStreamSedingtests meaningfully cover the checkpoint synchronization protocol, recovery paths, and cancellation signal behavior. The refactoring of_make_agentto streaming-only is consistent with the production code becoming streaming-only. The uv.lock file references local Windows paths (C:/Users/bentho/src/..) for three azure-ai-agentserver-* packages. This makes dependency installation and the entire test suite non-runable in CI or on any other developer's machine. The new sample (15_workflow_resilience) lacks automated unit tests but its demo.py serves as a manual integration test, which is acceptable for samples.
✗ Failure Modes
The most significant issue is a hardcoded developer-specific Windows path committed in python/pyproject.toml (find-links pointing to C:/Users/bentho/src/..). This will cause dependency resolution failures on CI and for all other developers since the azure-ai-agentserver-* packages are not on PyPI and this path won't exist on other machines. There is also a duplicate 'Deploying the Agent to Foundry' section in the 01_basic README. The uv.lock and pyproject.toml reference absolute local Windows paths (C:/Users/bentho/src/long-running-agents-private-preview/artifacts/wheels) for three critical azure-ai-agentserver-* packages. These paths won't resolve on CI or any other developer's machine, causing dependency installation failures. The PR title says 'draft updates', so this may be intentional for in-progress work, but it should not be merged as-is.
✗ Design Approach
I found one design regression in the Foundry hosted-response adapters: several hosted tool-call items are now converted back into plain
function_callcontent without the transcript-only marker that the rest of the repo uses for provider-owned Responses items. That changes their semantics during replay/round-trip compared with the established hosted Responses pattern. The new crash-recovery sample advertises and enablesresilient_backgroundon a plainAgent, but the host implementation explicitly limits that feature to workflow agents. That makes the sample's core design misleading: users who follow it will not get the recovery behavior the README describes.
Flagged Issues
- Both
python/pyproject.toml:63andpython/uv.lock(lines 1317–1352) commit absolute developer-local Windows paths (C:/Users/bentho/src/long-running-agents-private-preview/artifacts/wheels) for azure-ai-agentserver-core, -invocations, and -responses. Since these packages are not on PyPI and the documented install path consumes the lockfile via frozen resolution (python/DEV_SETUP.md:232-239), dependency resolution will fail on CI and all other contributors' machines. Remove the hardcoded find-links from the committed pyproject.toml and regenerate the lockfile against a shared registry or committed wheel source. - Hosted Foundry response-item adapters now drop
informational_only=Truefrom provider-owned tool-call transcript items, diverging from the established pattern inpython/packages/openai/agent_framework_openai/_chat_client.py:2309-2337and theContent.from_function_callcontract inpython/packages/core/agent_framework/_types.py:827-828. This changes replay/round-trip semantics so that these provider-owned calls may be incorrectly invoked locally.
Automated review by alliscode's agents
|
Flagged issue Both Source: automated DevFlow PR review |
|
Flagged issue Hosted Foundry response-item adapters now drop Source: automated DevFlow PR review |
- Switch agentserver wheel sources from GitHub raw URLs to find-links at local long-running-agents-private-preview checkout; remove [tool.uv.sources] URL entries from python/pyproject.toml - Re-lock uv.lock for find-links path sources - Guard tracker.close() in cancellation path to avoid ValueError when the SDK state machine is in not_started state - Fix test assertions to access _ContextAwareCheckpointStorage._inner.storage_path instead of .storage_path (which is not exposed directly on the wrapper) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1674d491-af7d-4f9e-99aa-5a829db81591
| # checkpoints and approval requests to survive a process restart. | ||
| # Otherwise local development keeps everything in memory. | ||
| use_file_storage = self.config.is_hosted or (resilient_background and self._is_workflow_agent) | ||
| if self._is_workflow_agent and use_file_storage: |
There was a problem hiding this comment.
| if self._is_workflow_agent and use_file_storage: | |
| if use_file_storage: |
We don't need to check self._is_workflow_agent since it's been checked above
| if not context.is_recovery and workflow_checkpoint_execution.restore_checkpoint_id is not None: | ||
| # ``response.created`` strips framework-internal metadata | ||
| # from its wire payload. Explicitly checkpoint the live | ||
| # response before executing the new turn so a crash before | ||
| # its first superstep can restore the prior durable state. | ||
| yield cast(ResponseStreamEvent, response_event_stream.checkpoint()) |
There was a problem hiding this comment.
Question: not sure why this is necessary?
| ) | ||
|
|
||
| async for item in _to_outputs_for_messages( | ||
| async for event in self._drive_workflow_stream( |
There was a problem hiding this comment.
It looks like this is restoring the workflow to a checkpointed state. We don't stream out events during restoration.
| if checkpoint_execution.restore_checkpoint_id is None or checkpoint_execution.restore_storage is None: | ||
| raise RuntimeError("Current-turn recovery requires an exact workflow checkpoint reference.") | ||
|
|
||
| async for workflow_event in self._agent.workflow.run( |
There was a problem hiding this comment.
Any reason we are invoking the workflow directly?
| checkpoint_id=checkpoint_execution.restore_checkpoint_id, | ||
| checkpoint_storage=checkpoint_execution.restore_storage, | ||
| ): | ||
| for update in self._agent._convert_workflow_event_to_agent_response_updates( # pyright: ignore[reportPrivateUsage] |
There was a problem hiding this comment.
If we invoke the workflow as an agent, we don't need to parse the workflow events.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1674d491-af7d-4f9e-99aa-5a829db81591
| provider.source_id, | ||
| ) | ||
|
|
||
| resilient_background = bool(options and options.resilient_background) |
There was a problem hiding this comment.
Could we pin the first azure-ai-agentserver-responses build that actually exposes these recovery APIs? The final lock resolves the public 1.0.0b8 artifact, but that build has no ResponsesServerOptions.resilient_background/steerable_conversations, ResponseContext.shutdown/recovery members, or ResponseEventStream.checkpoint; the new samples fail at construction and ordinary/workflow requests raise AttributeError. This is also what the current package and sample CI failures are reporting.
| reference = checkpoint_task.result() | ||
| if reference is None: | ||
| break | ||
| yield cast(ResponseStreamEvent, publisher.checkpoint_event(reference)) |
There was a problem hiding this comment.
How about we close the active output builder before snapshotting this workflow checkpoint? The agentserver SDK does not add an item to response.output until response.output_item.done, but workflow checkpoints are saved after their output events have been consumed. A crash after this checkpoint can therefore resume past an update that is absent from the persisted response, permanently dropping that text or tool call.
| self._agent.workflow.name, | ||
| latest_checkpoint.checkpoint_id, | ||
| ) | ||
| elif context.is_recovery and request.previous_response_id is not None: |
There was a problem hiding this comment.
Could this fallback cover context.conversation_id too? response.created is emitted before the baseline checkpoint event, so a process exit in that window recovers with no structured checkpoint reference. For a conversation-scoped request this condition skips the existing conversation checkpoint and reruns the input on a fresh workflow, losing the conversation's durable state.
| contents.append(Content.from_text_reasoning(id=reasoning.id, text=summary.text)) | ||
| else: | ||
| contents.append(Content.from_text_reasoning(id=reasoning.id)) | ||
| contents.append(Content.from_text(summary.text)) |
There was a problem hiding this comment.
Could we keep reconstructing this as Content.from_text_reasoning(id=reasoning.id, text=summary.text)? Turning it into ordinary text removes the marker that _mcp_call_ids_paired_with_reasoning() uses to keep reasoning and its hosted MCP call together.
On stateless multi-turn replay, the MCP call is then sent without its required reasoning item and OpenAI rejects the request with HTTP 400, reintroducing #6883/#6074.
| mcp_call = stream.add_output_item_mcp_call( | ||
| server_label=content.server_name or "default", | ||
| name=content.tool_name or "", | ||
| item_id=content.call_id, |
There was a problem hiding this comment.
Shouldn't we preserve one MCP call ID across the persisted call and output? add_output_item_mcp_call() now generates a new item ID, while aoutput_item_custom_tool_call_output() persists the original Content.call_id; on the next turn the OpenAI serializer only coalesces results whose call_id equals the persisted mcp_call.id, so it drops this result as orphaned. The new round-trip test currently accepts the mismatch and never exercises the real serializer.
Motivation & Context
Description & Review Guide
Related Issue
Fixes #
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.