feat(pip): add computer use snapshot preview

docs/architecture/nativekit-agent-browser-pip/spec.md

@@ -56,6 +56,13 @@ contracts remain in force. The current renderer Canvas remains the compatibility
 
 GitHub issue sync was not requested and was not performed.
 
+On 2026-07-28, a
+[Computer Use latest-snapshot PiP extension](../../features/computer-use-snapshot-pip/spec.md) was
+implemented. `AgentPreviewCoordinator` now owns the process-global NativeKit lifecycle and
+arbitrates one Browser or Computer Use presentation. Browser retains its page, capture, panel
+handoff, and **Open in panel** plus **Close** behavior; Computer Use contributes only bounded latest
+snapshots with **Close**.
+
 ## Counterpoint
 
 The NativeKit path will make panel dragging fluid and allow the panel to leave the DeepChat window.
@@ -247,21 +254,24 @@ focusless render-host BaseWindow -> Agent WebContentsView at 1280 x 800
 - branches completed frames to exactly one surface;
 - stops capture and clears the surface on every existing lifecycle edge.
 
-#### `AgentBrowserNativeOverlay`
+#### `AgentPreviewCoordinator`
 
-A focused main-process adapter owns only:
+A focused process-global main-process coordinator owns only:
 
 - dynamic package loading and one-time capability detection;
 - `overlay.start()` / `stop()` lifecycle;
 - one active host and one active presentation;
+- Browser and Computer Use toolbar profile switching;
+- latest explicit source claim and shared run-scoped dismissal;
 - host move/resize/close synchronization;
 - JPEG `Buffer` to data-URL conversion;
 - prepaint-before-show ordering;
-- mapping NativeKit `activate` and configured `control` IDs to the current logical
-  `{ sessionId, runId, windowId }`;
+- mapping NativeKit activation and configured controls to the current source-specific handler;
 - timing counters around synchronous native calls.
 
-It is not a general Presenter, registry, plugin system, or cross-native abstraction.
+`YoBrowserPresenter` remains the Browser producer and `ComputerUsePreviewPresenter` remains the
+Computer Use producer. The coordinator is not a public preview framework, plugin system, or
+cross-native abstraction.
 
 #### Renderer
 

docs/features/agent-browser-pip/spec.md

@@ -16,6 +16,12 @@ Canvas described here remains the compatibility fallback. This document's page o
 background viewport, read-only preview, panel handoff, and run-scoped dismissal contracts remain
 authoritative.
 
+On 2026-07-28, a
+[Computer Use latest-snapshot PiP extension](../computer-use-snapshot-pip/spec.md) was implemented.
+It shares only process-global NativeKit presentation ownership; it does not generalize Browser
+page, capture, panel-handoff, or public feature contracts. Browser keeps **Open in panel** plus
+**Close**, continuous bounded capture, and its existing Canvas compatibility behavior.
+
 The referenced screenshot was not available in the current task context, so the interaction and
 layout are specified here while exact visual styling remains provisional.
 

docs/features/computer-use-snapshot-pip/plan.md

@@ -0,0 +1,333 @@
+# Computer Use Snapshot Picture-in-Picture Implementation Plan
+
+## Status
+
+Implemented on branch `codex/computer-use-snapshot-pip` on 2026-07-28.
+
+Phases 1-5 are implemented with focused automated coverage, and the complete renderer suite passes.
+The complete main suite retains one isolated, unrelated provider-metadata expectation failure.
+Performance measurement and packaged native and fallback platform QA remain open; the exact
+remaining work is recorded in `tasks.md`.
+
+CUA runtime, plugin, policy, skill, and asset updates are explicitly out of scope. The plan consumes
+successful Agent-requested snapshots plus one bounded private `get_window_state` result after an
+eligible successful `click`.
+
+## Delivery Strategy
+
+Establish one process-global PiP owner before adding the Computer Use presenter. NativeKit toolbar
+configuration is global, so implementing a second standalone overlay adapter first would create a
+known Browser/Computer race.
+
+Then carry existing Agent run identity through the MCP execution path and observe only resolved,
+trusted CUA calls. Build the snapshot transform and renderer fallback after those identities are
+available. Finish with arbitration, lifecycle hardening, and Browser regression verification.
+
+```text
+shared NativeKit owner
+        |
+        +--> existing Browser producer
+        |
+        +--> Computer Use observer -> presenter -> snapshot producer
+                                            |
+                                  native or Canvas surface
+```
+
+## Phase 1: Shared PiP Ownership
+
+### 1. Extract the process-global overlay owner
+
+Refactor `AgentBrowserNativeOverlay` so NativeKit lifecycle and host synchronization are owned by
+one coordinator instantiated in `src/main/app/composition.ts`.
+
+Keep the extraction narrow:
+
+- one dynamic NativeKit import and capability result;
+- one `overlay.start()` and `overlay.stop()` lifecycle;
+- one host attachment and host/display listener set;
+- one visible presentation;
+- toolbar profiles for Browser and Computer Use;
+- source-specific activation/control callbacks;
+- common visibility, removal, prepaint, and latency behavior.
+
+The Browser presenter remains authoritative for Browser page/run/capture state. Migrate its current
+native adapter calls to the coordinator without changing Browser contracts or fallback behavior.
+
+### 2. Add explicit toolbar profiles
+
+Configure:
+
+- Browser: **Open in panel**, then **Close**; activation opens the Browser panel.
+- Computer Use: **Close** only; activation is ignored.
+
+Switch profiles before presenting a source. Hide/remove the previous presentation during a profile
+change so controls can never act on the wrong target.
+
+### 3. Add claim arbitration
+
+Represent the latest retained claim per source and host/session with run, target identity, and a
+monotonic activity sequence.
+
+- Explicit Browser tool activity claims Browser.
+- An actual resolved CUA `get_window_state` invocation claims Computer Use.
+- Frame updates are refreshes and cannot claim.
+- A background or inactive host/session claim cannot preempt the foreground active session.
+- A producer may present only while its claim is the latest eligible sequence for the foreground
+  active host/session.
+- Focus/session changes reevaluate retained sequences without manufacturing new activity.
+- Dismissal applies to the current shared session/run.
+
+Add coordinator unit tests before wiring the Computer Use image path.
+
+## Phase 2: MCP Execution Identity
+
+### 4. Propagate `runId`
+
+Extend MCP call options through:
+
+1. `McpServicePort.callTool`.
+2. `McpService.callTool`.
+3. `ToolManager.callTool`.
+4. Narrow test doubles and call sites that implement those contracts.
+
+Update `ToolService` to forward `ToolCallOptions.runId` on the MCP branch alongside access and abort
+metadata. Do not add it to `MCPToolCall`, serialize it to the MCP server, or place it in tool
+arguments.
+
+Add a focused regression test proving that an Agent run ID reaches `ToolManager` for MCP calls and
+that the outgoing CUA arguments remain byte-for-byte equivalent.
+
+### 5. Introduce a narrow Computer Use observer
+
+Define an injected main-process observer port with `started`, `completed`, and `failed` callbacks.
+Call it only after `ToolManager` has:
+
+- resolved the real MCP client and original tool name;
+- loaded and validated the server config;
+- completed access and permission checks;
+- repaired and prepared tool arguments;
+- identified the plugin source through `ownerPluginId` or `sourceId`.
+
+Emit `started` immediately before actual client invocation. Emit one terminal callback without
+changing response mapping or error propagation.
+
+Calls lacking run/session identity and non-CUA calls should produce no preview callbacks.
+
+### 5a. Add a private post-click refresh
+
+After a successful exact `click` on the resolved official CUA client:
+
+- require valid current session/run plus positive integer `pid` and `window_id`;
+- synchronously ask the presenter whether the same target is PiP-eligible and not dismissed;
+- asynchronously call `get_window_state` on that already resolved client with only
+  `{ pid, window_id }`;
+- send the private result only through the preview observer;
+- never publish or return the private result to the Agent or conversation;
+- retain the original click response and latency behavior if capture fails.
+
+Do not trigger on permission responses, failed clicks, stale targets, `right_click`,
+`double_click`, inactive PiP, or arbitrary servers. Do not add timers, polling, or CUA runtime
+changes.
+
+## Phase 3: Computer Use Snapshot Presenter
+
+### 6. Add presenter state and lifecycle
+
+Create `ComputerUsePreviewPresenter` in `src/main/desktop/computerUse/` and instantiate it once from
+the application composition root.
+
+Track:
+
+- host window and active chat session;
+- run and tool call;
+- `pid` and `windowId`;
+- epoch and active coordinator claim;
+- renderer preview mode;
+- dismissed run;
+- current valid encoded frame;
+- transform in-flight and latest pending result.
+
+On a new run or target, advance the epoch, hide/remove the old presentation, release the old frame,
+and wait for the first valid new frame.
+
+### 7. Implement snapshot recognition
+
+On observer callbacks:
+
+- require the trusted Computer Use plugin identity;
+- require original tool name `get_window_state`;
+- parse finite positive integer `pid` and `window_id`;
+- claim the Computer Use source at invocation start;
+- accept only a successful result containing a supported inline image;
+- retain the prior valid same-target frame on failure;
+- discard results whose session, run, target, tool call, claim, or epoch is stale.
+
+Use shared utilities only where an existing utility already matches the raw inline MCP image
+contract. Do not route through cached message-preview artifacts.
+
+### 8. Build the bounded image pipeline
+
+In main:
+
+1. Select the first supported PNG/JPEG inline image block.
+2. Validate base64 and a 16 MiB encoded-input ceiling.
+3. Decode and validate dimensions no larger than 8192 x 8192.
+4. Resize proportionally within 480 x 300 without upscaling.
+5. Encode JPEG quality 72.
+6. Reject output larger than 512 KiB.
+7. Revalidate epoch and claim.
+8. Present to NativeKit or emit the Canvas fallback frame.
+
+Keep one transform in flight and one replaceable latest pending result. Add rate-limited,
+metadata-only warnings for validation and latency failures.
+
+## Phase 4: Typed Desktop Contracts and Renderer Fallback
+
+### 9. Add routes and events
+
+Add Computer Use preview schemas beside the existing desktop/browser contract organization:
+
+- `computerUse.setPreviewMode`;
+- `computerUse.dismissPreview`;
+- `computerUse.preview.frame`.
+
+Expose them through the existing typed preload/client boundary. Validate route sender and session
+binding in main. Emit frame events only for Canvas fallback.
+
+### 10. Add the renderer controller
+
+Mount `AgentComputerUsePiP.vue` beside `AgentBrowserPiP.vue` in `ChatTabView.vue`.
+
+Derive:
+
+- active chat session;
+- working versus terminal session state;
+- current route;
+- host focus.
+
+Send `eligible`, `suspended`, or `stopped` only when the derived mode changes. On native surfaces,
+render nothing and never subscribe to image payloads.
+
+### 11. Add the Canvas fallback
+
+For fallback only:
+
+- render one bounded Canvas card;
+- expose one existing translated **Close** button;
+- make non-control surface points draggable with pointer capture;
+- decode a frame before replacing current Canvas pixels;
+- reject stale session/run/sequence events;
+- release image resources on replacement and unmount;
+- forward no input to the target application.
+
+Do not add a generic Browser/Computer Vue component unless the completed implementation has
+meaningful stable duplication.
+
+## Phase 5: Lifecycle, Privacy, and Failure Hardening
+
+### 12. Complete foreground lifecycle
+
+Use existing window/session binding and host listeners to:
+
+- hide on blur, hide, minimize, inactive route, or suspended mode;
+- restore only when the same current claim remains eligible;
+- remove on stopped session, terminal run, target change, host close, or shutdown;
+- prevent one host/session from seeing another session's retained frame.
+
+When the target application comes to the foreground, DeepChat loses focus and the preview hides.
+
+### 13. Enforce run-scoped dismissal
+
+Validate exact session and run on close. Suppress all later frames and source changes for that run
+without canceling tools. Clear suppression only for a later run or terminal cleanup.
+
+### 14. Enforce memory-only image handling
+
+Audit logs, events, caches, and teardown:
+
+- no base64 or pixel buffer logging;
+- no disk writes or settings persistence;
+- no generic MCP image broadcast;
+- no renderer image payload on native path;
+- prompt release of stale buffers, Canvas resources, and native presentations.
+
+## Phase 6: Verification and Rollout
+
+### 15. Add focused automated coverage
+
+Add the smallest tests for:
+
+- shared coordinator profile switching, claims, dismissal, and stale refresh rejection;
+- Browser toolbar/activation/dismissal regression;
+- MCP `runId` propagation and unchanged outgoing arguments;
+- trusted source recognition and permission/non-invocation paths;
+- successful, failed, aborted, and out-of-order observer callbacks;
+- presenter run/target/epoch transitions;
+- image validation, resize, output limit, and latest-wins behavior;
+- native delivery versus Canvas-only frame publication;
+- renderer focus/session modes, close, frame retention, and cleanup.
+
+### 16. Measure performance
+
+Instrument metadata-only durations for result receipt, transform completion, and NativeKit push.
+Verify:
+
+- at most one added capture call per eligible successful `click` and none while PiP is ineligible;
+- zero idle polling;
+- one in-flight transform;
+- p95 result-to-visible latency at or below 150 ms;
+- no renderer image traffic on native surface;
+- no unbounded buffer retention.
+
+### 17. Run platform QA
+
+Verify at least:
+
+- one packaged native-overlay runtime;
+- one packaged Canvas-fallback runtime;
+- host focus/minimize/restore;
+- target application foreground transition;
+- Browser-to-Computer and Computer-to-Browser claims;
+- native toolbar profile switching;
+- run close/restart and session switching;
+- malformed and oversized image handling.
+
+Keep the feature behind an internal gate until both surface paths pass if the platform matrix cannot
+complete in the implementation change.
+
+### 18. Close documentation
+
+After implementation:
+
+- update `spec.md` status and record material deviations;
+- update the linked Browser NativeKit architecture with the final shared-owner contract;
+- mark completed tasks in `tasks.md`;
+- include the BEFORE/AFTER ASCII layout in the PR;
+- record remaining packaged platform QA explicitly.
+
+## Verification Commands
+
+Run after implementation:
+
+```bash
+pnpm run format
+pnpm run i18n
+pnpm run lint
+pnpm run typecheck
+pnpm run test:main
+pnpm run test:renderer
+pnpm exec electron-vite build
+```
+
+Prefer narrower test file invocations during development. Run the complete affected main and
+renderer suites before handoff because the change crosses MCP execution, desktop presentation, and
+renderer lifecycle boundaries.
+
+## Rollback
+
+The Computer Use observer and presenter must be independently disableable. Disabling them restores
+the previous behavior without changing CUA tool execution.
+
+If the shared coordinator causes a Browser regression, revert the coordinator integration as one
+unit and keep the run-propagation changes only if their focused tests pass and no public behavior
+changes. Do not fall back to two independent NativeKit owners.