Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 21 additions & 6 deletions src/lib/mcp/prompts.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";
import { TELEMETRY_EVENT_CATALOG } from "@/lib/mcp/telemetry";

export function registerKernelPrompts(server: McpServer) {
// MCP Prompt explaining Kernel concepts
Expand Down Expand Up @@ -128,7 +129,19 @@ kernel browsers process --help
kernel browsers playwright --help
\`\`\`

**MCP Exception:** The \`computer_action\` MCP tool with action "screenshot" is useful since it returns images directly to the agent.
**MCP Exceptions:** The \`computer_action\` MCP tool with action "screenshot" is useful since it returns images directly to the agent, and \`get_browser_telemetry\` reads structured telemetry events (see below).

---

## Telemetry Events (structured signal — works even after the session is deleted)

When telemetry was captured, it's usually the fastest way to pinpoint a failure — read it before reaching for screenshots or logs.

Start broad: call \`get_browser_telemetry\` with session_id "${session_id}" and no filters. That reads the whole session and definitively answers whether anything was archived. Narrow only when the output is too large to scan: \`categories\` to isolate a signal you've already spotted, order "desc" to inspect the end of the session, \`since\`/\`until\` to bracket the failing step. Correlate event timestamps with the failing automation step, and page with \`next_offset\` while \`has_more\` is true.

**Gotcha: telemetry is opt-in and only covers activity that happened while capture was on.** Archived events survive telemetry being disabled and the session being deleted, so the archive — not the current config — is the ground truth: \`manage_browsers\` action "get" showing a null \`telemetry\` field means capture is off now, not that nothing was recorded. The default bundle (control/connection/system/captcha) also omits the debug-critical categories. To capture new evidence on an active browser, use \`manage_browsers\` action "update" to enable \`telemetry_console\`, \`telemetry_network\`, and \`telemetry_page\`, then reproduce the issue; recreate the browser only if the session has ended.

${TELEMETRY_EVENT_CATALOG}

---

Expand Down Expand Up @@ -224,6 +237,7 @@ These are **normal** and don't indicate problems:
## Debugging Checklist

- [ ] Session exists and is active
- [ ] Telemetry events reviewed (if any were captured)
- [ ] Screenshot shows expected content (or reveals error)
- [ ] Current URL is as expected
- [ ] Supervisor logs show all services running
Expand All @@ -237,11 +251,12 @@ These are **normal** and don't indicate problems:

Based on your issue "${issue_description}", start with:

1. **Get browser info** to confirm session is active
2. **Take screenshot** to see current state
3. **Check page URL** to see if on error page
4. **Test network** if seeing connection errors
5. **Review logs** for specific error patterns`;
1. **Get browser info** to confirm session is active and check whether telemetry was enabled
2. **Read telemetry events**; if needed, enable telemetry on an active session and reproduce
3. **Take screenshot** to see current state
4. **Check page URL** to see if on error page
5. **Test network** if seeing connection errors
6. **Review logs** for specific error patterns`;

return {
messages: [
Expand Down
14 changes: 14 additions & 0 deletions src/lib/mcp/telemetry.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
export const telemetryEventCategories = [
"console",
"network",
"page",
"interaction",
"control",
"connection",
"system",
"screenshot",
"captcha",
"monitor",
] as const;

export const TELEMETRY_EVENT_CATALOG = `Event categories: console (console output and uncaught exceptions), network (request/response metadata), page (navigation and lifecycle), interaction (clicks, keys, scrolls), control (agent-driven API calls), connection (CDP/live-view attach/detach), system (VM health), screenshot (periodic monitor screenshots), captcha (captcha detection and solve outcomes), monitor (telemetry collector health; captured automatically with any CDP category). High-signal event types: console_error, network_loading_failed, network_response with non-2xx status, captcha_solve_result, system_oom_kill, service_crashed, monitor_disconnected (telemetry gap — treat following events as incomplete).`;
200 changes: 196 additions & 4 deletions src/lib/mcp/tools/browsers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,18 @@ import {
toolErrorResponse,
} from "@/lib/mcp/responses";
import { paginationParams } from "@/lib/mcp/schemas";
import {
TELEMETRY_EVENT_CATALOG,
telemetryEventCategories,
} from "@/lib/mcp/telemetry";

type BrowserCreateParams = NonNullable<
Parameters<KernelClient["browsers"]["create"]>[0]
>;
type BrowserUpdateParams = Parameters<KernelClient["browsers"]["update"]>[1];
type TelemetryEventsQuery = NonNullable<
Parameters<KernelClient["browsers"]["telemetry"]["events"]>[1]
>;

type TelemetryParams = {
telemetry_enabled?: boolean;
Expand Down Expand Up @@ -79,6 +86,46 @@ function buildTelemetry(
};
}

type TelemetryEnvelope = Awaited<
ReturnType<KernelClient["browsers"]["telemetry"]["events"]>
>["items"][number];

// Payload fields that can carry kilobytes per event (response bodies, header
// maps). Dropped so a full page of events fits in an agent context window;
// omitted_fields tells the agent what to fetch via the API/CLI if needed.
const bulkyTelemetryDataFields = ["body", "headers", "post_data"] as const;

function compactTelemetryEvent({ seq, event }: TelemetryEnvelope) {
const { ts, category, type, source, truncated } = event;
const data = "data" in event ? event.data : undefined;

let compactData: Record<string, unknown> | undefined;
let omittedFields: string[] | undefined;
if (data) {
compactData = { ...(data as Record<string, unknown>) };
for (const field of bulkyTelemetryDataFields) {
if (compactData[field] !== undefined) {
delete compactData[field];
(omittedFields ??= []).push(field);
}
}
}

return {
seq,
// Raw ts (Unix microseconds) is kept alongside the readable time so exact
// event boundaries can be fed back as since/until.
ts,
time: new Date(ts / 1000).toISOString(),
category,
type,
source,
...(compactData && { data: compactData }),
...(truncated && { truncated }),
...(omittedFields && { omitted_fields: omittedFields }),
};
}

function browserSessionNextActions(sessionId: string) {
return [
`Use computer_action with session_id "${sessionId}" to inspect or control the browser.`,
Expand Down Expand Up @@ -299,21 +346,25 @@ export function registerBrowserCapabilities(server: McpServer) {
telemetry_enabled: z
.boolean()
.describe(
"(create, update) Enable telemetry with VM defaults, or disable telemetry when false.",
"(create, update) Enable telemetry, or disable telemetry when false. Telemetry is off unless requested. The default category set is the lightweight operational bundle (control, connection, system, captcha) and does NOT include console, network, or page — enable those explicitly when you intend to debug page behavior.",
)
.optional(),
telemetry_console: z
.boolean()
.describe("(create, update) Enable or disable console telemetry.")
.describe(
"(create, update) Enable or disable console telemetry (console output and uncaught exceptions). Off by default; enable for debugging.",
)
.optional(),
telemetry_network: z
.boolean()
.describe("(create, update) Enable or disable network telemetry.")
.describe(
"(create, update) Enable or disable network telemetry (request/response metadata). Off by default; enable for debugging.",
)
.optional(),
telemetry_page: z
.boolean()
.describe(
"(create, update) Enable or disable page lifecycle telemetry.",
"(create, update) Enable or disable page lifecycle telemetry (navigation, load, layout shifts, LCP). Off by default; enable for debugging.",
)
.optional(),
telemetry_interaction: z
Expand Down Expand Up @@ -460,4 +511,145 @@ export function registerBrowserCapabilities(server: McpServer) {
}
},
);

// get_browser_telemetry -- Read archived telemetry events for a session
server.tool(
"get_browser_telemetry",
`Read archived telemetry events for a browser session. Works while the session is active and after it is deleted, including events captured before telemetry was disabled. If the response reports status "telemetry_currently_disabled", widen or remove filters before enabling telemetry and reproducing: update an active browser, or recreate one that has ended. Page through long sessions with offset/next_offset instead of raising limit. ${TELEMETRY_EVENT_CATALOG}`,
{
session_id: z.string().describe("Browser session ID."),
categories: z
.array(z.enum(telemetryEventCategories))
.min(1)
.describe(
"Restrict results to these event categories. The filter applies within each page, so a filtered page can be empty while has_more is true.",
)
.optional(),
limit: z
.number()
.int()
.min(1)
.max(100)
.describe("Max events per page (1-100). Default 100.")
.optional(),
offset: z
.number()
.int()
.min(0)
.describe(
"Pagination cursor: pass next_offset from the previous response to fetch the next page. Opaque — do not derive it from event seq values.",
)
.optional(),
since: z
.string()
.describe(
"Start of the window: an RFC-3339 timestamp or a duration like '30m' meaning that long ago. Defaults to the session's creation time. Ignored when offset is set; cannot be combined with order=desc.",
)
.optional(),
until: z
.string()
.describe(
"End of the window (exclusive): an RFC-3339 timestamp or a duration like '5m'.",
)
.optional(),
order: z
.enum(["asc", "desc"])
.describe(
"Read direction. asc (default) reads oldest first starting from since; desc reads newest first — useful for inspecting the end of a session.",
)
.optional(),
},
{
title: "Read browser telemetry events",
readOnlyHint: true,
destructiveHint: false,
idempotentHint: true,
openWorldHint: false,
},
async (params, extra) => {
if (!extra.authInfo) throw new Error("Authentication required");
if (params.since !== undefined && params.order === "desc") {
return errorResponse(
"Error in get_browser_telemetry (events): since cannot be combined with order=desc. Use until to bound a newest-first read, or order=asc with since.",
);
}
const client = createKernelClient(extra.authInfo.token);

// Best-effort lookup for the session's telemetry config and creation
// time; when it fails we still read events but skip disambiguation.
const fetchBrowser = () =>
client.browsers.retrieve(params.session_id).catch(() => null);

try {
const query: TelemetryEventsQuery = { limit: params.limit ?? 100 };
if (params.categories) query.category = params.categories;
if (params.offset !== undefined) query.offset = params.offset;
if (params.since !== undefined) query.since = params.since;
if (params.until !== undefined) query.until = params.until;
if (params.order !== undefined) query.order = params.order;

let browser: Awaited<ReturnType<typeof fetchBrowser>> = null;
if (
query.offset === undefined &&
query.since === undefined &&
query.order !== "desc"
) {
// The API's since default is only 5m; cover the whole session.
browser = await fetchBrowser();
query.since = browser?.created_at ?? "1970-01-01T00:00:00Z";
}
// When the read covers the whole session with no filters (asc from
// creation, or desc from the newest event), an empty result means the
// archive is empty — there is nothing to widen.
const fullSessionRead =
params.offset === undefined &&
params.since === undefined &&
params.until === undefined &&
params.categories === undefined;

const page = await client.browsers.telemetry.events(
params.session_id,
query,
);
const items = page.getPaginatedItems().map(compactTelemetryEvent);

let status: "ok" | "telemetry_currently_disabled" | "no_events" = "ok";
let note: string | undefined;
if (items.length === 0) {
if (page.has_more) {
note =
"This page had no matching events, but more are archived — continue paging with next_offset.";
} else {
const emptyReason = fullSessionRead
? "No telemetry events are archived for this session"
: "No archived events matched this window and filter — widen since/until or drop the categories filter";
browser ??= await fetchBrowser();
if (browser && !browser.telemetry) {
status = "telemetry_currently_disabled";
note = `${emptyReason}. Telemetry is currently disabled: update this active browser with telemetry_enabled=true plus telemetry_console, telemetry_network, and telemetry_page, then reproduce the issue.`;
} else {
status = "no_events";
note = browser
? `${emptyReason}.`
: `${emptyReason}, and the session could not be fetched. If the session has ended and telemetry was not enabled, recreate it with telemetry enabled (including console, network, and page) and reproduce the issue.`;
}
}
}

// Compact serialization: a full page of events would waste a large
// share of its size on pretty-printing indentation.
return textResponse(
JSON.stringify({
status,
items,
has_more: page.has_more,
next_offset: page.next_offset,
...(note && { note }),
}),
);
} catch (error) {
return toolErrorResponse("get_browser_telemetry", "events", error);
}
},
);
}