fix(always-on): extract report from LLM text when report tool is not invoked#153
Open
Gucc111 wants to merge 1 commit into
Open
fix(always-on): extract report from LLM text when report tool is not invoked#153Gucc111 wants to merge 1 commit into
Gucc111 wants to merge 1 commit into
Conversation
…invoked (#148) When the report-phase agent produces text but does not call always_on_report, the run was incorrectly marked as failed with reason "report_tool_not_invoked". Now the runtime extracts assistant_text_delta events, runs them through parseReportMarkdown, and persists the result — so the outcome reflects the actual execution success rather than a missed tool call. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Always-On runtime to avoid incorrectly marking a run as failed when the report-phase agent produces a valid textual report but does not invoke the always_on_report tool. It adds a fallback path that reconstructs the report from streamed assistant_text_delta events, validates/completes it via parseReportMarkdown, and persists it so the run outcome reflects execution success.
Changes:
- Capture report-phase gateway events (
reportEvents) and extract concatenated assistant text when the report tool was not invoked and no report error occurred. - Parse the extracted text with
parseReportMarkdownand persist the normalized report, settingreportCtx.reportso the outcome becomes"executed". - Preserve the existing fallback-template behavior when no tool call occurs and the assistant produced no report text.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1160
to
+1167
| function extractAssistantText(events: GatewayEvent[]): string { | ||
| let text = ""; | ||
| for (const event of events) { | ||
| if (event.type === "assistant_text_delta") { | ||
| text += event.text; | ||
| } | ||
| } | ||
| return text.trim(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
always_on_report, the run was incorrectly marked asfailedwith reasonreport_tool_not_invoked. Now the runtime extractsassistant_text_deltaevents from the report session, runs them throughparseReportMarkdownfor validation/completion, and persists the result — so the outcome reflects the actual execution success rather than a missed tool call.reportCtx.reportis unset and there is noreportError.Changes
File:
src/always-on/runtime/DiscoveryFire.tsextractAssistantText()helper that concatenates allassistant_text_deltaevents into a single string.fire()andrun()methods, afterdrainTurncompletes and before outcome determination: if the report tool was not called and there is no error, attempt to extract LLM text from events. If non-empty, parse it viaparseReportMarkdownand write to disk, settingreportCtx.reportso the outcome resolves to"executed".writeFallbackReportpath is preserved (outcome ="failed").Priority: tool call > LLM text > fallback template
executed(unchanged)executed(new)failed(unchanged)Closes #148
Made with Cursor