fix(mcp): browser-use screenshot saving path and inline rendering#166
Open
Gucc111 wants to merge 1 commit into
Open
fix(mcp): browser-use screenshot saving path and inline rendering#166Gucc111 wants to merge 1 commit into
Gucc111 wants to merge 1 commit into
Conversation
… as inline images The browser-use MCP process inherited an unpredictable CWD from the parent PilotDeck process, so screenshots saved with a user-specified `filename` parameter ended up in the wrong directory. Set `cwd: outDir` on the per-session spec so both auto-generated and named screenshots land in `projectRoot/.pilotdeck/browser_screenshots/<sid>/`. When @playwright/mcp returns only a text block with a Markdown file reference (no base64 image block — happens when `filename` is provided), read the referenced image from disk and inject it as an inline image block so it renders in the chat UI through the existing toolResultImages pipeline. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cwd: outDiron the browser-use MCP spec so that screenshots (both auto-named and user-specifiedfilename) are saved toprojectRoot/.pilotdeck/browser_screenshots/<sessionId>/instead of an unpredictable inherited CWD.marshalMcpContent(): when@playwright/mcpreturns only a text block with a Markdown image link (no base64 — happens when the LLM passesfilename), read the referenced file from disk and inject it as an inline image block, so the existingtoolResultImagesrendering pipeline can display it.Root cause
@playwright/mcphas two path-resolution strategies forbrowser_take_screenshot:filename?--output-dir(absolute)text+image(base64) blocksworkspaceFile(filename)→ relative to process CWDtextblock only (Markdown link)PilotDeck spawned the browser-use MCP process without setting
cwd, soprocess.cwd()was inherited from the PilotDeck parent — an unpredictable location. Screenshots withfilenamewere written to the wrong directory (or silently failed), and even when the file existed, the response lacked base64 data for the UI to render.Test plan
browser_take_screenshot)projectRoot/.pilotdeck/browser_screenshots/<sessionId>/filename— verify base64 inline rendering still worksfilename— verify the file-reading fallback kicks inMade with Cursor