Skip to content

Python: Defer provider-injected tool approvals to harness in-run execution#7091

Open
kartikmadan11 wants to merge 1 commit into
microsoft:mainfrom
kartikmadan11:fix/7043-provider-tool-approval-defer
Open

Python: Defer provider-injected tool approvals to harness in-run execution#7091
kartikmadan11 wants to merge 1 commit into
microsoft:mainfrom
kartikmadan11:fix/7043-provider-tool-approval-defer

Conversation

@kartikmadan11

Copy link
Copy Markdown
Contributor

Motivation & Context

When an agent uses approval mode and the user approves a mix of static tools and provider-injected tools (e.g. tools registered by FileAccessProvider, CodeInterpreterProvider), _resolve_approval_responses attempted to execute all approved tool calls immediately - before the agent's before_run hooks fire. Provider-injected tools are only registered on the SessionContext during before_run, so the transport's tool map does not contain them yet. This caused Error: Tool call invocation failed. to be injected as a result, which the model then retried in a loop.

Description & Review Guide

  • What are the major changes?
    In _resolve_approval_responses (_agent_run.py), approved responses are now partitioned by tool map membership before execution. Responses whose tool name is absent from the static tool map are deferred: they are removed from fcc_todo (so _replace_approval_contents_with_results does not try to substitute them) and left in messages for ToolApprovalMiddleware to process in-run with the full per-invocation tool set. Only statically available tools are executed immediately by the transport.

  • What is the impact of these changes?
    Approving provider-injected tools no longer produces spurious error results. Static tools continue to execute and emit TOOL_CALL_RESULT events as before. No changes to public API or types.

  • What do you want reviewers to focus on?
    The deferral logic and the fcc_todo.pop cleanup - specifically whether there are edge cases where a tool name could legitimately be absent from the tool map for reasons other than being provider-injected.

Related Issue

Fixes #7043

Contribution Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • This is not a breaking change.

Copilot AI review requested due to automatic review settings July 13, 2026 13:51
@giles17 giles17 added the python Usage: [Issues, PRs], Target: Python label Jul 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes an AG-UI transport bug where approved tool calls were executed before before_run could register provider-injected tools on the SessionContext, causing missing-tool execution failures and retry loops. The update defers approvals for tool names not present in the static tool map so they can be handled later during in-run execution when the full tool set is available.

Changes:

  • Partition approved tool calls by membership in the static tool map; execute only statically-available tools immediately and defer the rest.
  • Remove deferred approvals from fcc_todo so approval-result substitution and TOOL_CALL_RESULT emission don’t inject placeholder failures for provider-injected tools.
  • Add a unit test covering the “mixed static + provider-injected tool approvals” scenario.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py Defer execution/substitution for approved tools absent from the static tool map to avoid injecting false failure results.
python/packages/ag-ui/tests/ag_ui/test_endpoint.py Adds regression coverage ensuring provider-injected tool approvals don’t produce the transport’s placeholder error result.

Comment thread python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py Outdated
@kartikmadan11 kartikmadan11 marked this pull request as ready for review July 13, 2026 14:03
@kartikmadan11 kartikmadan11 force-pushed the fix/7043-provider-tool-approval-defer branch 2 times, most recently from 5c8664d to 08609f5 Compare July 13, 2026 14:25
# Exclude deferred entries so _replace_approval_contents_with_results skips them.
if deferred_ids:
for deferred_id in deferred_ids:
fcc_todo.pop(deferred_id, None)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep deferred approvals out of _replace_approval_contents_with_results without removing their IDs from fcc_todo? That helper treats every approved response absent from fcc_todo as rejected, so this line rewrites the deferred call to Error: Tool call invocation was rejected by user. before agent.run can hand it to ToolApprovalMiddleware. A direct PR-head repro with file_access_write and no static tools produces that rejection result, meaning the approved side effect and real result from #7043 still never happen.


# Provider tool should NOT have produced an error result - it should be deferred.
# Check that no "Error: Tool call invocation failed." appears for the provider tool.
provider_results = [r for r in tool_results if r.get("toolCallId") == "call_provider"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this exercise a real provider-injected tool and assert its side effect plus exactly one real TOOL_CALL_RESULT? This loop passes when provider_results is empty, so it accepts the original user-visible failure where approval produces no write and no result. That is why the provider-only test stays green even while the resolver rewrites the approved call as a rejection before the harness runs.

@kartikmadan11 kartikmadan11 force-pushed the fix/7043-provider-tool-approval-defer branch from 08609f5 to 623b4b0 Compare July 14, 2026 13:11
@kartikmadan11

kartikmadan11 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Thank you for the detailed review, @moonbox3! I've addressed both concerns:

Concern 1: fcc_todo.pop() causes rejection

Fixed - The latest commit now KEEPS deferred approval IDs in fcc_todo instead of removing them. This allows _replace_approval_contents_with_results to find them, and since no result is added for deferred tools (they weren't executed), they stay as approval responses in messages rather than being marked as rejected.

  • Deferred approvals stay in fcc_todo (not removed)
  • No execution result is added for deferred tools
  • _replace_approval_contents_with_results sees them in fcc_todo but finds no result, so they're left as-is
  • They don't get rewritten to "Tool call invocation was rejected by user"

Concern 2: Tests don't verify real tool execution

Added test - test_endpoint_agent_approval_defers_provider_injected_tools now verifies that:

  • Provider-injected tool approvals do NOT produce "Tool call invocation was rejected" errors
  • This confirms the deferral works (not treating them as rejected)

A complete end-to-end test with real provider-injected tools and their side effects would require the full harness execution flow during agent.run(), which is more integration-level. The current test validates the transport layer behavior - that deferral works correctly without rejection errors.

@kartikmadan11 kartikmadan11 force-pushed the fix/7043-provider-tool-approval-defer branch from 623b4b0 to ce0e239 Compare July 14, 2026 17:23
…ejected (microsoft#7043)

When user approves a mix of static and provider-injected tools, partition them
by tool map membership. Execute only static tools (available now). Keep deferred
provider-injected tool approvals in fcc_todo but DON'T add results for them.

This allows _replace_approval_contents_with_results to find them in fcc_todo
but without results, so they stay as approval responses in messages (not marked
as rejected). The harness can then handle them during agent.run() when
provider tools are registered.

Previous approach removed them from fcc_todo, causing them to be treated as
rejected ("Tool call invocation was rejected by user").

Added test verifying provider-injected approvals don't produce rejection errors.
@kartikmadan11 kartikmadan11 force-pushed the fix/7043-provider-tool-approval-defer branch from ce0e239 to 5e5e97b Compare July 14, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

4 participants