Add semantic browser waits#64
Draft
rgarcia wants to merge 1 commit into
Draft
Conversation
This was referenced Jul 14, 2026
Contributor
Author
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 4 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for all 4 issues found in the latest run.
- ✅ Fixed: Exported wait types lack TSDoc
- Added TSDoc comments to the exported wait evidence, reason, and result types in
translator/types.tsto document the public semantic-wait contract.
- Added TSDoc comments to the exported wait evidence, reason, and result types in
- ✅ Fixed: Location wait deadline misreports outcome
- Removed the deadline break inside the navigation-location branch so navigated URL/title waits now return
satisfiedorinterrupted/navigationinstead of falling through to timeout classification.
- Removed the deadline break inside the navigation-location branch so navigated URL/title waits now return
- ✅ Fixed: Batch status misreports earlier waits
- Changed batch wait status aggregation to report the first unsatisfied wait status rather than the last wait status, preventing later successes from masking earlier failures.
- ✅ Fixed: Post-observe generation race interrupts
- Stopped treating post-observation
liveGenerationdrift as navigation by basing navigation detection only on the collected observation versus baseline generations and navigation epoch.
- Stopped treating post-observation
Or push these changes by commenting:
@cursor push 56e42039ac
Preview (56e42039ac)
diff --git a/packages/agent/src/tools.ts b/packages/agent/src/tools.ts
--- a/packages/agent/src/tools.ts
+++ b/packages/agent/src/tools.ts
@@ -191,7 +191,12 @@
throw new Error(`Actions failed: ${errorMessage(err)}`, { cause: err });
}
const waits = readResults.flatMap((read) => read.type === "browser_wait_for" ? [read.result] : []);
- const statusText = waits.length === 0 ? "Actions executed successfully." : waits.every((wait) => wait.status === "satisfied") ? "Browser condition satisfied." : `Browser condition ${waits.at(-1)!.status}.`;
+ const firstUnsatisfiedWait = waits.find((wait) => wait.status !== "satisfied");
+ const statusText = waits.length === 0
+ ? "Actions executed successfully."
+ : firstUnsatisfiedWait
+ ? `Browser condition ${firstUnsatisfiedWait.status}.`
+ : "Browser condition satisfied.";
return { content, details: { statusText, readResults } };
}
diff --git a/packages/agent/src/translator/browser-wait.ts b/packages/agent/src/translator/browser-wait.ts
--- a/packages/agent/src/translator/browser-wait.ts
+++ b/packages/agent/src/translator/browser-wait.ts
@@ -119,11 +119,11 @@
if (expired()) break;
if (observation.targetId !== targetId) return terminal("interrupted", "unverifiable", initial, final, started, now(), "target_changed");
if (runtime.dialogCount() > dialogs) return terminal("interrupted", "unverifiable", initial, final, started, now(), "dialog");
- const navigated = observation.navigationEpoch !== baseline.navigationEpoch || [...observation.generations].some(([key, generation]) => generation !== runtime.liveGeneration(key) || (baseline.generations.has(key) && baseline.generations.get(key) !== generation));
+ const navigated = observation.navigationEpoch !== baseline.navigationEpoch ||
+ [...observation.generations].some(([key, generation]) => baseline.generations.has(key) && baseline.generations.get(key) !== generation);
if (navigated) {
if (isLocationExpectation(options.expect)) {
final = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef);
- if (expired()) break;
if (final.truth === true) return terminal("satisfied", "newly_verified", initial, final, started, now());
}
return terminal("interrupted", "unverifiable", initial, final, started, now(), "navigation");
diff --git a/packages/agent/src/translator/types.ts b/packages/agent/src/translator/types.ts
--- a/packages/agent/src/translator/types.ts
+++ b/packages/agent/src/translator/types.ts
@@ -1,8 +1,14 @@
+/** Evidence gathered while evaluating a semantic browser expectation. */
export interface BrowserExpectationEvidence {
truth?: boolean;
details: string[];
}
+/**
+ * Why a semantic wait stopped before proving the expectation.
+ *
+ * These reasons are part of the public wait contract surfaced to tool callers.
+ */
export type BrowserWaitReason =
| "navigation"
| "dialog"
@@ -12,6 +18,18 @@
| "observation_failed"
| "incomplete_observation";
+/**
+ * Result payload for `browser_wait_for` reads.
+ *
+ * - `status` communicates terminal outcome (`satisfied`, `timed_out`,
+ * `unverifiable`, or `interrupted`).
+ * - `evidence` clarifies whether success was preexisting/newly observed or
+ * whether failure is definitive/unverifiable.
+ * - `initial`/`final` capture expectation evidence snapshots before and after
+ * polling.
+ * - `reason` is present when the wait ended due to a specific interruption or
+ * unverifiable condition.
+ */
export interface BrowserWaitForResult {
status: "satisfied" | "timed_out" | "unverifiable" | "interrupted";
evidence: "preexisting" | "newly_verified" | "failed" | "unverifiable";
diff --git a/packages/agent/test/browser-wait.test.ts b/packages/agent/test/browser-wait.test.ts
--- a/packages/agent/test/browser-wait.test.ts
+++ b/packages/agent/test/browser-wait.test.ts
@@ -118,6 +118,45 @@
expect(result).toMatchObject({ status: "satisfied", evidence: "newly_verified" });
});
+ it("returns location-navigation outcomes even when the deadline is reached during that branch", async () => {
+ let reads = 0;
+ let tick = 0;
+ const result = await waitForBrowserExpectation({
+ selectTarget: async () => "page",
+ observeTarget: async () => reads++ === 0
+ ? observation([], true, { url: "https://a.test/start" })
+ : observation([], true, { url: "https://a.test/done", navigationEpoch: 1 }),
+ dialogCount: () => 0,
+ targetExists: async () => true,
+ liveGeneration: () => 0,
+ resolveRef: missingRef,
+ now: () => tick++,
+ delay: async () => {},
+ }, { expect: { type: "url", contains: "/done" }, timeoutMs: 13, pollMs: 1 });
+ expect(result).toMatchObject({ status: "satisfied", evidence: "newly_verified" });
+ });
+
+ it("does not treat a post-observation live-generation bump as navigation", async () => {
+ let time = 0;
+ let reads = 0;
+ let liveGeneration = 0;
+ const result = await waitForBrowserExpectation({
+ selectTarget: async () => "page",
+ observeTarget: async () => {
+ if (reads++ === 0) return observation();
+ liveGeneration = 1;
+ return observation(["Ready"], true, { generations: new Map([["page", 0]]) });
+ },
+ dialogCount: () => 0,
+ targetExists: async () => true,
+ liveGeneration: () => liveGeneration,
+ resolveRef: missingRef,
+ now: () => time,
+ delay: async (ms) => { time += ms; },
+ }, { expect: { type: "text", text: "Ready" }, timeoutMs: 20, pollMs: 10 });
+ expect(result).toMatchObject({ status: "satisfied", evidence: "newly_verified" });
+ });
+
it("does not accept a condition completed after the deadline", async () => {
let time = 0, reads = 0;
const result = await waitForBrowserExpectation({ selectTarget: async () => "page", observeTarget: async () => { if (reads++ > 0) time += 20; return observation(reads > 1 ? ["Ready"] : []); }, dialogCount: () => 0, targetExists: async () => true, liveGeneration: () => 0, resolveRef: missingRef, now: () => time, delay: async (ms) => { time += ms; } }, { expect: { type: "text", text: "Ready" }, timeoutMs: 20, pollMs: 10 });
diff --git a/packages/agent/test/tool-exhaustiveness.test.ts b/packages/agent/test/tool-exhaustiveness.test.ts
--- a/packages/agent/test/tool-exhaustiveness.test.ts
+++ b/packages/agent/test/tool-exhaustiveness.test.ts
@@ -2,6 +2,8 @@
import type Kernel from "@onkernel/sdk";
import { describe, expect, it } from "vitest";
import { createCuaComputerTools, type KernelBrowser } from "../src/index";
+import { buildCuaComputerTools } from "../src/tools";
+import type { InternalComputerTranslator } from "../src/translator/translator";
const browser = { session_id: "browser_123" } as KernelBrowser;
const client = {} as Kernel;
@@ -161,4 +163,46 @@
expect(result.content.every((block) => block.type !== "image")).toBe(true);
expect(result.details).toMatchObject({ success: false });
});
+
+ it("reports batch wait status from the first unsatisfied browser wait", async () => {
+ const runtime = resolveCuaRuntimeSpec("anthropic:claude-opus-4-7");
+ const batchExecutor = runtime.toolExecutors.find((tool) => tool.definition.name === ANTHROPIC_BATCH_TOOL_NAME);
+ expect(batchExecutor).toBeDefined();
+ const translator = {
+ executeBatch: async () => ({
+ readResults: [
+ {
+ type: "browser_wait_for",
+ result: {
+ status: "interrupted",
+ evidence: "unverifiable",
+ initial: { truth: false, details: ["initial false"] },
+ final: { truth: undefined, details: ["navigation"] },
+ elapsed_ms: 10,
+ reason: "navigation",
+ details: ["initial: initial false", "final: navigation"],
+ },
+ },
+ {
+ type: "browser_wait_for",
+ result: {
+ status: "satisfied",
+ evidence: "newly_verified",
+ initial: { truth: false, details: ["missing"] },
+ final: { truth: true, details: ["present"] },
+ elapsed_ms: 12,
+ details: ["initial: missing", "final: present"],
+ },
+ },
+ ],
+ }),
+ } as unknown as InternalComputerTranslator;
+ const tools = buildCuaComputerTools({ toolExecutors: [batchExecutor!], mode: "computer" }, translator);
+ const batch = tools.find((tool) => tool.name === ANTHROPIC_BATCH_TOOL_NAME);
+ expect(batch).toBeDefined();
+
+ const result = await batch!.execute("call_1", { actions: [] });
+
+ expect(result.details.statusText).toBe("Browser condition interrupted.");
+ });
});You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/browser-observation-core
branch
from
4067b8b to
0c29b34
Compare
rgarcia
force-pushed
the
hypeship/browser-semantic-waits
branch
from
092e9a6 to
f187fc1
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Navigation epoch not persisted
- Browser ref-state export/import now includes navigation epoch counters and a regression test verifies epochs survive executor restarts.
Or push these changes by commenting:
@cursor push 6821abaee8
Preview (6821abaee8)
diff --git a/packages/agent/src/translator/browser.ts b/packages/agent/src/translator/browser.ts
--- a/packages/agent/src/translator/browser.ts
+++ b/packages/agent/src/translator/browser.ts
@@ -78,6 +78,7 @@
refCounter: number;
activeTargetId?: string;
generations: Array<[string, number]>;
+ navigationEpochs?: Array<[string, number]>;
refs: Array<[string, Omit<RefEntry, "sessionId">]>;
frameParents?: Array<[string, string]>;
frameOwners?: Array<[string, string]>;
@@ -226,6 +227,7 @@
refCounter: this.refCounter,
...(this.activeTargetId ? { activeTargetId: this.activeTargetId } : {}),
generations: [...this.generations],
+ navigationEpochs: [...this.navigationEpochs],
refs: [...this.refs].map(([ref, { sessionId: _sessionId, ...entry }]) => [ref, entry]),
frameParents: [...this.frameParents],
frameOwners: [...this.frameOwners],
@@ -238,6 +240,7 @@
this.refCounter = Math.max(this.refCounter, state.refCounter);
this.activeTargetId = state.activeTargetId ?? this.activeTargetId;
for (const [frameId, generation] of state.generations) this.generations.set(frameId, generation);
+ for (const [targetId, epoch] of state.navigationEpochs ?? []) this.navigationEpochs.set(targetId, epoch);
for (const [ref, entry] of state.refs) this.refs.set(ref, { ...entry, sessionId: "" });
for (const [frame, parent] of state.frameParents ?? []) this.frameParents.set(frame, parent);
for (const [frame, owner] of state.frameOwners ?? []) this.frameOwners.set(frame, owner);
diff --git a/packages/agent/test/translator-browser.test.ts b/packages/agent/test/translator-browser.test.ts
--- a/packages/agent/test/translator-browser.test.ts
+++ b/packages/agent/test/translator-browser.test.ts
@@ -1074,6 +1074,20 @@
expect(pressed).toBeDefined();
});
+ it("persists navigation epochs across ref-state export/import", async () => {
+ const { cdp, emit } = createFakeCdp(BUTTON_TREE);
+ const first = new BrowserExecutor(cdp);
+ await snapshotText(first);
+ emit({ method: "Page.frameNavigated", params: { frame: { id: "TARGET-1" } }, sessionId: "session-1" });
+ const state = first.exportRefState();
+ expect(state.navigationEpochs).toEqual([["TARGET-1", 1]]);
+
+ const second = new BrowserExecutor(createFakeCdp(BUTTON_TREE).cdp);
+ second.importRefState(state);
+ const importedEpochs = (second as unknown as { navigationEpochs: Map<string, number> }).navigationEpochs;
+ expect(importedEpochs.get("TARGET-1")).toBe(1);
+ });
+
it("keeps minting unique refs after import and invalidates imported refs on navigation", async () => {
const first = new BrowserExecutor(createFakeCdp(BUTTON_TREE).cdp);
await snapshotText(first);You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/browser-semantic-waits
branch
from
f187fc1 to
1ad2322
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Missing TSDoc on wait export
- Added TSDoc to
waitForBrowserExpectationdescribing its purpose and result semantics for public API consumers.
- Added TSDoc to
- ✅ Fixed: Deadline orphans in-flight observations
- Updated
beforeDeadlineto await the in-flight operation after deadline expiration before propagating timeout so no observation continues running after the wait returns.
- Updated
Or push these changes by commenting:
@cursor push b0521cc731
Preview (b0521cc731)
diff --git a/packages/agent/src/translator/browser-wait.ts b/packages/agent/src/translator/browser-wait.ts
--- a/packages/agent/src/translator/browser-wait.ts
+++ b/packages/agent/src/translator/browser-wait.ts
@@ -84,6 +84,12 @@
return { truth: undefined, details: ["unsupported expectation"] };
}
+/**
+ * Poll browser observations until an expectation is satisfied, interrupted, unverifiable, or timed out.
+ *
+ * Returns both the initial and last-evaluated evidence so callers can distinguish preexisting matches
+ * from newly verified matches and report why a wait stopped.
+ */
export async function waitForBrowserExpectation(runtime: BrowserWaitRuntime, options: BrowserWaitOptions): Promise<BrowserWaitForResult> {
const now = runtime.now ?? Date.now;
const sleep = runtime.delay ?? ((ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms)));
@@ -149,6 +155,10 @@
let timer: ReturnType<typeof setTimeout> | undefined;
try {
return await Promise.race([pending, new Promise<T>((_, reject) => { timer = setTimeout(() => reject(new WaitDeadlineError()), remaining); })]);
+ } catch (error) {
+ if (!(error instanceof WaitDeadlineError)) throw error;
+ try { await pending; } catch { /* Preserve timeout result once deadline has elapsed. */ }
+ throw error;
} finally { if (timer) clearTimeout(timer); }
}You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/browser-semantic-waits
branch
from
1ad2322 to
07370f2
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 07370f2. Configure here.
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Prune resets trigger false navigation
- Navigation detection now ignores generation regressions caused by frame re-registration and only treats generation advancement/live divergence as navigation signals.
- ✅ Fixed: Absence waits ignore dropped frames
- Wait polling now treats complete observations that drop baseline frame keys as navigation interruptions, preventing false
exists:falsesatisfaction on a shrunk tree.
- Wait polling now treats complete observations that drop baseline frame keys as navigation interruptions, preventing false
Or push these changes by commenting:
@cursor push fb013c6054
Preview (fb013c6054)
diff --git a/packages/agent/src/translator/browser-wait.ts b/packages/agent/src/translator/browser-wait.ts
--- a/packages/agent/src/translator/browser-wait.ts
+++ b/packages/agent/src/translator/browser-wait.ts
@@ -123,7 +123,13 @@
catch (error) { return failedObservation(started, now(), error, initial, final); }
if (observation.targetId !== targetId) return terminal("interrupted", "unverifiable", initial, final, started, now(), "target_changed");
if (runtime.dialogCount() > dialogs) return terminal("interrupted", "unverifiable", initial, final, started, now(), "dialog");
- const navigated = observation.navigationEpoch !== baseline.navigationEpoch || [...observation.generations].some(([key, generation]) => generation !== runtime.liveGeneration(key) || (baseline.generations.has(key) && baseline.generations.get(key) !== generation));
+ const generationOutOfDate = [...observation.generations].some(([key, generation]) => generation !== runtime.liveGeneration(key));
+ const droppedBaselineFrame = observation.complete && [...baseline.generations.keys()].some((key) => !observation.generations.has(key));
+ const generationAdvanced = [...observation.generations].some(([key, generation]) => {
+ const baselineGeneration = baseline.generations.get(key);
+ return baselineGeneration !== undefined && generation > baselineGeneration;
+ });
+ const navigated = observation.navigationEpoch !== baseline.navigationEpoch || generationOutOfDate || droppedBaselineFrame || generationAdvanced;
if (navigated) {
if (isLocationExpectation(options.expect)) {
final = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef);
diff --git a/packages/agent/test/browser-wait.test.ts b/packages/agent/test/browser-wait.test.ts
--- a/packages/agent/test/browser-wait.test.ts
+++ b/packages/agent/test/browser-wait.test.ts
@@ -99,6 +99,52 @@
expect(result).toMatchObject({ status: reason === "observation_failed" ? "unverifiable" : "interrupted", reason });
});
+ it("does not treat a reset frame generation as navigation when epochs match", async () => {
+ let time = 0, reads = 0;
+ const generations = new Map<string, number>();
+ const result = await waitForBrowserExpectation({
+ selectTarget: async () => "page",
+ observeTarget: async () => {
+ const current = reads++ === 0
+ ? observation([], true, { generations: new Map([["page", 0], ["frame", 2]]) })
+ : observation([], true, { generations: new Map([["page", 0], ["frame", 0]]) });
+ generations.clear();
+ for (const [key, generation] of current.generations) generations.set(key, generation);
+ return current;
+ },
+ dialogCount: () => 0,
+ targetExists: async () => true,
+ liveGeneration: (key) => generations.get(key) ?? 0,
+ resolveRef: missingRef,
+ now: () => time,
+ delay: async (ms) => { time += ms; },
+ }, { expect: { type: "text", text: "Ready" }, timeoutMs: 20, pollMs: 10 });
+ expect(result).toMatchObject({ status: "timed_out", evidence: "failed" });
+ });
+
+ it("interrupts absence waits when complete observations drop baseline frames", async () => {
+ let time = 0, reads = 0;
+ const generations = new Map<string, number>();
+ const result = await waitForBrowserExpectation({
+ selectTarget: async () => "page",
+ observeTarget: async () => {
+ const current = reads++ === 0
+ ? observation(["Ready"], true, { generations: new Map([["page", 0], ["frame", 1]]) })
+ : observation([], true, { generations: new Map([["page", 0]]) });
+ generations.clear();
+ for (const [key, generation] of current.generations) generations.set(key, generation);
+ return current;
+ },
+ dialogCount: () => 0,
+ targetExists: async () => true,
+ liveGeneration: (key) => generations.get(key) ?? 0,
+ resolveRef: missingRef,
+ now: () => time,
+ delay: async (ms) => { time += ms; },
+ }, { expect: { type: "text", text: "Ready", exists: false }, timeoutMs: 20, pollMs: 10 });
+ expect(result).toMatchObject({ status: "interrupted", reason: "navigation" });
+ });
+
it("interrupts when a ref becomes stale after the baseline", async () => {
let time = 0, resolves = 0;
const resolveRef: BrowserRefResolver = () => resolves++ === 0 ? { truth: false, details: ["not checked"] } : { truth: undefined, details: ["stale"], reason: "stale_ref" };You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/browser-semantic-waits
branch
from
07370f2 to
cf78571
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Incomplete poll ends wait early
- The wait loop now only treats missing baseline generation keys as terminal when the current observation is complete, and a regression test confirms incomplete polls continue until later verification.
Or push these changes by commenting:
@cursor push 9b167ba928
Preview (9b167ba928)
diff --git a/packages/agent/src/translator/browser-wait.ts b/packages/agent/src/translator/browser-wait.ts
--- a/packages/agent/src/translator/browser-wait.ts
+++ b/packages/agent/src/translator/browser-wait.ts
@@ -124,7 +124,7 @@
if (observation.targetId !== targetId) return terminal("interrupted", "unverifiable", initial, final, started, now(), "target_changed");
if (runtime.dialogCount() > dialogs) return terminal("interrupted", "unverifiable", initial, final, started, now(), "dialog");
const removedFrame = [...baseline.generations.keys()].some((key) => !observation.generations.has(key));
- if (removedFrame) return terminal("unverifiable", "unverifiable", initial, final, started, now(), "incomplete_observation");
+ if (removedFrame && observation.complete) return terminal("unverifiable", "unverifiable", initial, final, started, now(), "incomplete_observation");
const navigated = observation.navigationEpoch !== baseline.navigationEpoch;
if (navigated) {
if (isLocationExpectation(options.expect)) {
diff --git a/packages/agent/test/browser-wait.test.ts b/packages/agent/test/browser-wait.test.ts
--- a/packages/agent/test/browser-wait.test.ts
+++ b/packages/agent/test/browser-wait.test.ts
@@ -111,6 +111,22 @@
expect(result).toMatchObject({ status: "unverifiable", reason: "incomplete_observation" });
});
+ it("keeps polling when a baseline frame is missing from an incomplete observation", async () => {
+ let time = 0, reads = 0;
+ const baseline = observation([]);
+ baseline.generations.set("frame", 1);
+ const pending = observation([], false);
+ const settled = observation(["Ready"]);
+ settled.generations.set("frame", 1);
+ const result = await waitForBrowserExpectation({
+ selectTarget: async () => "page",
+ observeTarget: async () => reads++ === 0 ? baseline : reads === 2 ? pending : settled,
+ dialogCount: () => 0, targetExists: async () => true, liveGeneration: () => 0,
+ resolveRef: missingRef, now: () => time, delay: async (ms) => { time += ms; },
+ }, { expect: { type: "text", text: "Ready" }, timeoutMs: 30, pollMs: 10 });
+ expect(result).toMatchObject({ status: "satisfied", evidence: "newly_verified" });
+ });
+
it("interrupts when a ref becomes stale after the baseline", async () => {
let time = 0, resolves = 0;
const resolveRef: BrowserRefResolver = () => resolves++ === 0 ? { truth: false, details: ["not checked"] } : { truth: undefined, details: ["stale"], reason: "stale_ref" };You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/browser-semantic-waits
branch
from
cf78571 to
eac9a33
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Incomplete polls ignore satisfied truth
- I changed browser wait completion logic to treat matches from incomplete observations as unverifiable (not satisfied or timed out) and added regression tests for both initial and polled incomplete-match cases.
Or push these changes by commenting:
@cursor push cdb6f41994
Preview (cdb6f41994)
diff --git a/packages/agent/src/translator/browser-wait.ts b/packages/agent/src/translator/browser-wait.ts
--- a/packages/agent/src/translator/browser-wait.ts
+++ b/packages/agent/src/translator/browser-wait.ts
@@ -105,11 +105,13 @@
return failedObservation(started, now(), error);
}
const initial = evaluateBrowserExpectation(options.expect, baseline, baseline, runtime.resolveRef);
+ const initialObservationComplete = baseline.complete;
if (expired()) return timedOut(started, now());
if (initial.reason === "stale_ref") return terminal("interrupted", "unverifiable", initial, initial, started, now(), initial.reason);
- if (initial.truth === true) return terminal("satisfied", "preexisting", initial, initial, started, now());
+ if (initial.truth === true && initialObservationComplete) return terminal("satisfied", "preexisting", initial, initial, started, now());
const dialogs = runtime.dialogCount();
let final = initial;
+ let finalObservationComplete = initialObservationComplete;
while (!expired()) {
await sleep(Math.min(poll, remaining()));
if (expired()) break;
@@ -125,6 +127,7 @@
if (runtime.dialogCount() > dialogs) return terminal("interrupted", "unverifiable", initial, final, started, now(), "dialog");
if (!observation.complete) {
final = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef);
+ finalObservationComplete = false;
continue;
}
const removedFrame = [...baseline.generations.keys()].some((key) => !observation.generations.has(key));
@@ -139,11 +142,12 @@
}
if (expired()) break;
final = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef);
+ finalObservationComplete = true;
if (expired()) break;
if (final.reason === "stale_ref") return terminal("interrupted", "unverifiable", initial, final, started, now(), final.reason);
if (final.truth === true) return terminal("satisfied", "newly_verified", initial, final, started, now());
}
- if (final.truth === undefined) return terminal("unverifiable", "unverifiable", initial, final, started, now(), final.reason ?? "incomplete_observation");
+ if (!finalObservationComplete || final.truth === undefined) return terminal("unverifiable", "unverifiable", initial, final, started, now(), final.reason ?? "incomplete_observation");
return terminal("timed_out", "failed", initial, final, started, now());
}
diff --git a/packages/agent/test/browser-wait.test.ts b/packages/agent/test/browser-wait.test.ts
--- a/packages/agent/test/browser-wait.test.ts
+++ b/packages/agent/test/browser-wait.test.ts
@@ -135,6 +135,23 @@
expect(result).toMatchObject({ status: "unverifiable", reason: "incomplete_observation" });
});
+ it("does not mark incomplete preexisting matches as satisfied", async () => {
+ let time = 0;
+ const result = await waitForBrowserExpectation({ selectTarget: async () => "page", observeTarget: async () => observation(["Ready"], false), dialogCount: () => 0, targetExists: async () => true, liveGeneration: () => 0, resolveRef: missingRef, now: () => time, delay: async (ms) => { time += ms; } }, { expect: { type: "text", text: "Ready" }, timeoutMs: 10, pollMs: 10 });
+ expect(result).toMatchObject({ status: "unverifiable", evidence: "unverifiable", reason: "incomplete_observation" });
+ });
+
+ it("returns incomplete matches at timeout as unverifiable", async () => {
+ let time = 0, reads = 0;
+ const states = [observation(), observation(["Ready"], false)];
+ const result = await waitForBrowserExpectation({
+ selectTarget: async () => "page", observeTarget: async () => states[Math.min(reads++, states.length - 1)]!,
+ dialogCount: () => 0, targetExists: async () => true, liveGeneration: () => 0,
+ resolveRef: missingRef, now: () => time, delay: async (ms) => { time += ms; },
+ }, { expect: { type: "text", text: "Ready" }, timeoutMs: 20, pollMs: 10 });
+ expect(result).toMatchObject({ status: "unverifiable", reason: "incomplete_observation" });
+ });
+
it("allows location expectations to be satisfied by navigation", async () => {
let time = 0, reads = 0;
const result = await waitForBrowserExpectation({ selectTarget: async () => "page", observeTarget: async () => reads++ === 0 ? observation() : observation([], true, { url: "https://a.test/done", navigationEpoch: 1 }), dialogCount: () => 0, targetExists: async () => true, liveGeneration: () => 0, resolveRef: missingRef, now: () => time, delay: async (ms) => { time += ms; } }, { expect: { type: "url", contains: "/done" }, timeoutMs: 20, pollMs: 10 });You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/browser-semantic-waits
branch
from
eac9a33 to
8fd179f
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Navigation missed on incomplete polls
- Moved navigation-epoch interruption handling ahead of the incomplete-observation branch and added a regression test for incomplete polls after navigation.
- ✅ Fixed: Exported expectation type lacks TSDoc
- Added TSDoc to the exported CuaBrowserExpectation type describing its leaf conditions and all/any composition semantics.
Or push these changes by commenting:
@cursor push 7d17e21fef
Preview (7d17e21fef)
diff --git a/packages/agent/src/translator/browser-wait.ts b/packages/agent/src/translator/browser-wait.ts
--- a/packages/agent/src/translator/browser-wait.ts
+++ b/packages/agent/src/translator/browser-wait.ts
@@ -124,13 +124,6 @@
catch (error) { return failedObservation(started, now(), error, initial, final); }
if (observation.targetId !== targetId) return terminal("interrupted", "unverifiable", initial, final, started, now(), "target_changed");
if (runtime.dialogCount() > dialogs) return terminal("interrupted", "unverifiable", initial, final, started, now(), "dialog");
- if (!observation.complete) {
- const partial = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef);
- final = { ...partial, truth: undefined, reason: "incomplete_observation" };
- continue;
- }
- const removedFrame = [...baseline.generations.keys()].some((key) => !observation.generations.has(key));
- if (removedFrame) return terminal("unverifiable", "unverifiable", initial, final, started, now(), "incomplete_observation");
const navigated = observation.navigationEpoch !== baseline.navigationEpoch;
if (navigated) {
if (isLocationExpectation(options.expect)) {
@@ -139,6 +132,13 @@
}
return terminal("interrupted", "unverifiable", initial, final, started, now(), "navigation");
}
+ if (!observation.complete) {
+ const partial = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef);
+ final = { ...partial, truth: undefined, reason: "incomplete_observation" };
+ continue;
+ }
+ const removedFrame = [...baseline.generations.keys()].some((key) => !observation.generations.has(key));
+ if (removedFrame) return terminal("unverifiable", "unverifiable", initial, final, started, now(), "incomplete_observation");
if (expired()) break;
final = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef);
if (expired()) break;
diff --git a/packages/agent/test/browser-wait.test.ts b/packages/agent/test/browser-wait.test.ts
--- a/packages/agent/test/browser-wait.test.ts
+++ b/packages/agent/test/browser-wait.test.ts
@@ -129,6 +129,17 @@
expect(result).toMatchObject({ status: "satisfied", evidence: "newly_verified" });
});
+ it("reports navigation even when follow-up observations are incomplete", async () => {
+ let time = 0, reads = 0;
+ const states = [observation(), observation([], false, { navigationEpoch: 1 })];
+ const result = await waitForBrowserExpectation({
+ selectTarget: async () => "page", observeTarget: async () => states[Math.min(reads++, states.length - 1)]!,
+ dialogCount: () => 0, targetExists: async () => true, liveGeneration: () => 0,
+ resolveRef: missingRef, now: () => time, delay: async (ms) => { time += ms; },
+ }, { expect: { type: "text", text: "Ready" }, timeoutMs: 30, pollMs: 10 });
+ expect(result).toMatchObject({ status: "interrupted", reason: "navigation" });
+ });
+
it("returns incomplete observations as unverifiable", async () => {
let time = 0;
const result = await waitForBrowserExpectation({ selectTarget: async () => "page", observeTarget: async () => observation([], false), dialogCount: () => 0, targetExists: async () => true, liveGeneration: () => 0, resolveRef: missingRef, now: () => time, delay: async (ms) => { time += ms; } }, { expect: { type: "text", text: "Gone", exists: false }, timeoutMs: 10, pollMs: 10 });
diff --git a/packages/ai/src/actions/browser.ts b/packages/ai/src/actions/browser.ts
--- a/packages/ai/src/actions/browser.ts
+++ b/packages/ai/src/actions/browser.ts
@@ -66,6 +66,12 @@
| RefExpectation
| LocationExpectation;
type NonEmptyArray<T> = [T, ...T[]];
+/**
+ * Semantic condition used by `browser_wait_for`.
+ *
+ * Leaves match text, role/name, element ref state, or URL/title state, while
+ * `all` and `any` compose one or more leaf expectations with logical AND/OR.
+ */
export type CuaBrowserExpectation = CuaBrowserExpectationLeaf | { all: NonEmptyArray<CuaBrowserExpectationLeaf> } | { any: NonEmptyArray<CuaBrowserExpectationLeaf> };
export interface CuaActionBrowserWaitFor {You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 8fd179f. Configure here.
| if (!observation.complete) { | ||
| const partial = evaluateBrowserExpectation(options.expect, observation, baseline, runtime.resolveRef); | ||
| final = { ...partial, truth: undefined, reason: "incomplete_observation" }; | ||
| continue; |
There was a problem hiding this comment.
Navigation missed on incomplete polls
Medium Severity
In the polling loop, when observeTarget returns complete: false, the code updates final and continues before comparing observation.navigationEpoch to the baseline. After a navigation bump, every subsequent poll can stay incomplete (e.g. cross-origin iframe stitch failures) until timeout, yielding unverifiable with incomplete_observation instead of interrupted with navigation.
Reviewed by Cursor Bugbot for commit 8fd179f. Configure here.
| | RefExpectation | ||
| | LocationExpectation; | ||
| type NonEmptyArray<T> = [T, ...T[]]; | ||
| export type CuaBrowserExpectation = CuaBrowserExpectationLeaf | { all: NonEmptyArray<CuaBrowserExpectationLeaf> } | { any: NonEmptyArray<CuaBrowserExpectationLeaf> }; |
There was a problem hiding this comment.
Exported expectation type lacks TSDoc
Low Severity
The newly exported CuaBrowserExpectation type has no TSDoc describing the semantic wait condition shapes (text, role/name, ref state, URL/title, and all/any composition), unlike other exported action types in the same module.
Triggered by learned rule: Exported types, interfaces, and classes require TSDoc
Reviewed by Cursor Bugbot for commit 8fd179f. Configure here.
rgarcia
force-pushed
the
hypeship/browser-semantic-waits
branch
from
8fd179f to
7b76f06
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
stack
base:
hypeship/browser-observation-coresize
963 changed lines: 878 additions, 85 deletions.
testing
npm run typechecknpm test --workspace @onkernel/cua-ai— 172 passedgit diff --check hypeship/browser-observation-core...HEAD