Skip to content

refactor: migrate wait to request-bound runtime - #1875

Open
thymikee wants to merge 5 commits into
agent/wave4-getfrom
agent/wave4-wait
Open

refactor: migrate wait to request-bound runtime#1875
thymikee wants to merge 5 commits into
agent/wave4-getfrom
agent/wave4-wait

Conversation

@thymikee

@thymikee thymikee commented Aug 19, 2026

Copy link
Copy Markdown
Member

Summary

wait executes through a request-bound device runtime instead of a capability bucket. Its
grammar, timeout budget (#1075), landmark replay (ADR 0012 / #1349), @ref staleness warnings
(#1076 / ADR 0014), and timeout-surface decoration are unchanged. What changed is who decides and
what runs.

Stacked on #1877 (get's selector seam), which is stacked on #1876. Review the top commit.

Admission is facts

The descriptor drops capability: ALL_DEVICE_COMMAND_CAPABILITY for
platformExecution: { kind: 'device-runtime', uses: waitRuntimePlanUses }. The handler resolves a
plan, inspects the owner's facts once, refuses before binding, unwraps the admitted-plan token, and
binds once. Provider-owned devices are answered by their provider and fail closed.

A duration wait binds nothing. resolveWaitRuntimePlan({ target: 'sleep' }) returns a plan with
no use, so wait 500 performs zero inspections and zero binds — the cell legacy admission skipped
with parsed.kind !== 'sleep', now a type fact rather than a conditional.

findText is a measured preferred operation, not a retired path

An earlier revision of this PR deleted the Apple runner's findText arm as a second
platform-execution path. iOS Smoke proved that wrong: wait text "Last input: press" performed
17 readable canonical-tree captures, never observed the target, and timed out with
wait_target_absent. The reading is not redundant with the tree.

It is back, as wait's one preferred operation (ADR 0019 §9), reached through the bound runtime
rather than the daemon. Its authority is deliberately one-sided and that is what makes it preferred
rather than a second path:

  • found: true satisfies the wait and skips the capture entirely;
  • found: false — including every reason an owner cannot answer — is not an outcome, and the
    same poll falls through to the canonical tree.

So the required tree path stays semantically complete: the fast path can only make a satisfied wait
return sooner. It can never refuse a wait the tree would satisfy, and it never produces a timeout.

§9 measurement — iPhone 16 Pro (B2618889…), com.apple.Preferences, wait text General
already present, in-daemon waitedMs, only variable the Apple findText fact:

arm n median mean min–max
preferred on 12 52.5 ms 56.0 47–95
preferred off 12 148.0 ms 146.2 122–171

~95 ms saved per satisfied text wait, 2.82×.

Every condition under which Apple cannot answer moved into platform-apple — absent
appBundleId, macOS non-app surface. selector-runtime-backend.ts loses 3,058 B and now only
forwards the bound operation when the facts advertised it; no family, provider, surface, or session
conditional survives in the daemon.

Wait asks the facts about no-app devices — and the answers differ by family

Adopting the selector family's active-app plan split is not a narrowing. It is what lets wait ask
the owner instead of assuming every family can observe a device with no app attached:

  • iOSappBundleId is the XCUITest attach target. With none set the runner's own process
    comes to the foreground and displaces the app under test, then answers confidently about its
    own blank screen. Verified A/B on device: Settings foregrounded, sessionless find "General" list
    returned success: true, matches: [] and Settings was gone, displaced by the runner; the
    migrated snapshot refused in 2.4 s and left Settings untouched.
  • Android — the same no-app state captures the real launcher, the facts say so, and wait
    proceeds. Zero change.

Same plan, opposite outcomes, chosen by the owner rather than by a daemon conditional.

Own the consequences plainly:

  • iOS no-app wait <text>: a 10 s timeout implying the text was absent becomes an immediate
    refusal naming open. Nothing true is lost — it could never have succeeded.
  • iOS no-app wait stable and wait @ref: these stop returning success. They were false
    successes about the runner's own screen (wait @ref returned text: "AgentDeviceRunner"). This
    is the one place a script goes from passing to failing, and it was already lying.
  • Cold path: up to ~28 s saved per call.

Qualifier on the history: the refusal originates in #681 (2026-06-04), which correctly stopped
snapshot returning the runner's own tree. The rationale that survives in code — "a capture that
cannot succeed" — is too strong: #1296 live-validated that capture works when SpringBoard is named
as the session app. The real constraint is that XCUITest needs some bundle id, not the user's
app. That does not change what ships here (refusing beats displacing-and-lying), but the stronger
rationale should not be restated as established. A follow-up may replace refusal with a SpringBoard
attach.

Validation

Red before green. Against pre-change code: the handler-seam suite reported (0 test) because
resolveBoundWaitRuntime did not exist; the descriptor row failed with the live capability bucket
in hand; resolveWaitRuntimePlan is not a function; the decoration test failed with
expected "vi.fn()" to be called 1 times, but got 0 times.

The findText regression is proven against the retired-arm code, not just the new plumbing.
Deleting the arm from observeText — the exact code the first revision shipped — turns it red with
the smoke failure's shape:

× a text wait is satisfied by the owner native reading when the tree never carries it 2004ms
{"code":"COMMAND_FAILED","message":"wait timed out for text: Last input: press. …",
 "details":{"reason":"wait_target_absent","timeoutMs":2000,"readableCaptures":7,"waitedMs":2001}}

It is non-vacuous in both directions: the target never appears in the tree and the native reading
only answers from the second poll, so the first poll proves the tree cannot satisfy the wait and the
second proves the reading can.

Coverage: duration wait inspects and binds nothing even where capture is unavailable; text / @ref
/ selector / stable each perform exactly one inspection and one bind; unavailable facts refuse
before binding with the owner's reason and hint; a provider owner without capture fails closed; the
timeout decoration reuses the same binding; a satisfied native reading skips the capture; a negative
one still consults the tree; an owner advertising no reading polls tree-only. Wait's landmark,
system-surface, Android freshness, and hidden-content-hint suites moved onto inspectFacts /
bindDevice.

Notes for review

  • This PR lands export on snapshotPlanUnavailableResponse so the selector family has one
    refusal-wording owner — a wait that cannot capture now reads like a snapshot, diff, or find that
    cannot, including the iOS "run open first" hint. The export ships with its first external
    consumer in the same change, per check:production-exports.
  • findText is added to the seam's BoundSelectorOperations record, which refactor: migrate get to the request-bound device runtime #1877 documents as the
    extension point for exactly this.
  • ADR 0011 classification is deliberately not in this PR. A wait-text path pair was drafted
    and withdrawn: 9 of 11 guarantee cells came out inapplicable because the vocabulary is tap
    semantics, so the row would have satisfied the completeness gate without machine-checking the
    divergence that caused the incident. The property that matters — a non-tree observation source
    agreeing with the tree — has no cell today. Tracked for the unit that can classify all seven
    interaction paths with evidence.
  • captureSnapshot keeps its optional captureData? seam and captureSnapshotWithInteractor:
    wait is not the last selector unit.
  • Docs/skills unchanged — CLI grammar, flags, and help are untouched. The iOS no-app behavior change
    is described above rather than in docs because it is an error-path normalization shared with
    snapshot/diff.

@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.32 MB 2.32 MB +6.8 kB
JS gzip 761.6 kB 763.5 kB +1.9 kB
npm tarball 885.1 kB 886.6 kB +1.5 kB
npm unpacked 3.09 MB 3.09 MB +6.2 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.9 ms 27.7 ms -0.2 ms
CLI --help 71.5 ms 68.7 ms -2.8 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/sdk-batch-runner.js +1.6 kB +350 B
dist/src/runtime4.js +1.0 kB +250 B
dist/src/runtime2.js +253 B +92 B
dist/src/dispatch.js -26 B -10 B
dist/src/internal/daemon.js +45 B +9 B

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed exact head 20b2c33ff5ce95da33af7a65d6a207cbafbe4431: one blocking behavior regression. iOS Smoke proves the retired Apple findText arm is not semantically redundant: wait text "Last input: press" performs 17 readable canonical-tree captures, never observes the target, and times out with wait_target_absent. That is the precise Apple divergence currently described as residual risk, so it cannot be treated as covered parity or fixed by a rerun. ADR 0019 §6 requires deliberate behavior changes to be decided separately from a migration; §9 allows a measured preferred operation, and #1876’s selector-seam record classifies this path that way. Please preserve the successful runner answer through the request-bound runtime (with measurement and ADR 0011 classification), or land an explicitly accepted behavior narrowing first, and add a regression that fails when this target is tree-only-polled. Separately, macOS Smoke looks like retry/session-cleanup infrastructure (attempt-2 still owns the host at attempt-3 step 1) and needs a clean rerun. The draft also still owes its stated reconciliation with #1876’s published selector seam. Not ready-for-human at this head.

agent added 4 commits August 19, 2026 13:44
Behaviour-neutral. No descriptor changes platform execution and the cutover
table is untouched.

- buildRuntimeCaptureInput moves to its own module so every request-bound
  capture consumer builds CaptureSnapshotInput one way.
- The admit-then-bind sequence in the snapshot/diff resolver becomes one named
  step, ready for the selector units' second caller.
- CaptureSnapshotInput gains an optional per-capture signal, composed through
  captureSnapshotSignal by every snapshot runtime owner, so a polling consumer
  can enforce a poll deadline rather than inheriting only the bind-time signal.
- handlers/find.ts splits into focused target-capture and match-resolution
  concepts (600 -> 346 lines); behaviour unchanged.
`get` declares `elementReadRuntimeUse` (required `captureSnapshot`, preferred
`readTextAtPoint`), admits once from exact owner facts, refuses before binding,
and binds exactly once. Its capability bucket, the static HarmonyOS/Web command
sets that augmented it, and `requireCommandSupported` admission for `get` are
gone; `'get'` leaves the `createSelectorRuntime` capability union.

The neutral `readTextAtPoint` operation replaces the branch-per-family legacy
`read` dispatch on the `get` path. Every local family and both providers now
classify it exhaustively — Web, HarmonyOS, Vega and every provider row report it
unavailable, which is behaviour-preserving because the legacy dispatch had no arm
for them and threw on every call before falling back.

R36 is the new parametrized cutover row.
…ad outcome

Review blockers on #1877.

1. `dispatchGetViaRuntime` could complete the direct-iOS selector query before
   `resolveBoundGetRuntime`. Once `get` declares `device-runtime`, ADR 0019
   requires resolve -> admit -> bind before anything in the request path
   operates, so admission now runs first for every target shape and the fast
   path is a fast path *within* an admitted request. Regression: an eligible
   direct selector cannot operate when facts refuse admission.

2. `readTextAtPoint` returned `Promise<string>` and `readTextForNode` caught
   any throw and fell back, assigning a typed diagnostic after an untyped
   failure. It now returns a closed `ElementTextReadOutcome`; fallback happens
   only for the contract's classified reasons; unexpected errors propagate.
   The reason union is derived from its runtime list so the two cannot drift,
   and an unhandled reason is a compile error at the consumer.

This retires the generic catch the start record promised.
…nsumer

Takes ownership of the request-bound selector capture seam from #1876, which
cannot ship standalone: with find's cutover deferred it had no consuming
command (ADR 0019 §10) and was not dead-code clean (check:production-exports
19 -> 20). `get` is its first consumer, so it lands here.

Adopts find's handoff as given. The one shape change, approved by the
coordinator: the selector family gets its own capture uses carrying a PREFERRED
`readTextAtPoint`, declared ALONGSIDE the snapshot uses so `snapshot`/`diff`
keep binding exactly what they bind today. The read is surfaced through the
existing arms of `bindSnapshotCaptureRuntime`, reusing the same
selectActiveAppSnapshot / selectSnapshotWithoutActiveApp selectors — no second
plan-to-operation dispatch.

`get` now runs through `createBoundSelectorRuntime`; `resolveBoundGetRuntime`
and its test are deleted as superseded, and `'get'` leaves the
`createSelectorRuntime` capability union.

The legacy read adapter survives for `find <q> get text` and is selected by
which command constructed the runtime — never by failure, family, environment,
or flag — so `get` cannot reach it. It retires in find's cutover, where the
last consumer moves.
@thymikee
thymikee changed the base branch from main to agent/wave4-get August 19, 2026 13:17
@thymikee

Copy link
Copy Markdown
Member Author

Exact-head CI: 28/28 green at 8f57e7d89

  • iOS Smoke: pass. This is the lane that caught the findText regression at the previous head
    (wait text "Last input: press" → 17 readable captures, wait_target_absent). Green on the
    restored preferred operation, so the divergence is closed on the platform that exhibited it.
  • macOS Smoke: pass on the clean rerun — confirming it was retry/session-cleanup
    infrastructure, not this change.
  • Android, Linux, Web Platform Smoke, Integration Tests, Coverage, Layering Guard, Fallow, Bundle
    Size, Packaged CLI, Swift Runner Host XCTests, Typecheck, Lint/Format, CodeQL, Maestro
    Conformance Oracle, Released-Surface Compatibility, Replay-Compat Provenance: pass.

Zero failures, zero reruns needed on this head. PR remains draft pending review.

@thymikee
thymikee marked this pull request as ready for review August 19, 2026 13:36
@thymikee

Copy link
Copy Markdown
Member Author

Out of draft at 8f57e7d89. All 28 required checks green — including iOS Smoke, the lane that caught the regression at 20b2c33ff. macOS Smoke also passed on the clean rerun, confirming it was retry/cleanup infrastructure rather than this change. Zero reruns needed.

Stack: #1875#1877#1876main.

Your blocking finding is closed, and the framing changed

You were right that the findText retirement was a real regression, not covered parity — wait text "Last input: press" doing 17 readable canonical-tree captures and timing out with wait_target_absent. findText is restored as wait's one measured preferred operation, bound through the request runtime rather than daemon-side. §9 measurement: 2.82× faster, ~95 ms saved per satisfied text wait (n=12 per arm, iPhone 16 Pro). The regression that guards it is red the way Smoke was red — planting the deletion back reproduces wait_target_absent with the full budget burned.

The investigation you asked for reframed the active-app question. This PR does not make wait refuse; it makes wait ask the facts, which yields refuse-on-iOS and proceed-on-Android automatically. On iOS, appBundleId is the XCUITest attach target — with none set, the runner's own process comes forward and displaces the app under test, then answers confidently about its own 3-node screen. Four consequences, owned rather than buried:

  • iOS no-app wait <text>: a 10 s timeout implying the text was absent becomes an immediate refusal naming open. Nothing true is lost.
  • iOS no-app wait stable and wait @ref: stop returning success. These were false successes about the runner's own screen — wait @ref was returning text: "AgentDeviceRunner". This is the one place a script flips from passing to failing, and it was already lying.
  • Android: zero change.
  • Cold path: up to ~28 s saved.

Qualifier stated as a qualifier, not as settled: the refusal originated in #681 to stop snapshot returning the runner's tree, but the rationale surviving in code — that such a capture "cannot succeed" — is too strong. #1296 live-validated capture against SpringBoard; the real constraint is that XCUITest needs some bundle id, not the user's app. Filed separately as #1881.

Also in this PR

  • Apple's decisions left the daemon. selector-runtime-backend.ts is −3,058 B; no family, provider, surface, or session conditional survives there. The conditions under which Apple cannot answer moved into packages/platform-apple.
  • snapshotPlanUnavailableResponse is exported here with its first consumer, giving the selector family one refusal-wording owner instead of three spellings.
  • No ADR 0011 matrix row, deliberately — the matrix classifies zero observation commands and its guarantee vocabulary is entirely tap semantics, so a wait row would be all-inapplicable. Reasoning recorded rather than a vacuous row shipped.

Gate notes

The first check:affected run failed with three real defects that a narrower gate would have missed: a provider transcript still scripting the retired findText path, the test-file-size ratchet catching a pinned file growing (fixed by compacting the file's own factory, not by raising the pin), and the interactor-runner-provider partition requiring the new Interactor method be classified. Second run clean at 5,040 tests.

One retraction: an earlier comment flagged check:production-exports as pre-existing-red on main. That was a stale dist — it needs the fresh build the gate performs first, and passes in sequence. Disregard it.

@thymikee

Copy link
Copy Markdown
Member Author

Two blockers at exact head 8f57e7d8:

  1. [P1] Preserve runner execution context for native text waits. selector-runtime-backend.ts calls bound findText with only text, options, and poll signal, while bindAppleFindTextRuntime builds its interactor context from input.execution. wait text therefore drops requestId, log/trace paths, XCUITest override paths, and runner lease context that the retired path forwarded. Build execution through the existing request/capture-context builder and add a regression asserting native findText receives the same runner context as capture.

  2. Size/design: CI reports +6.2 kB npm unpacked, above the heightened-review threshold. The 112-line wait-runtime-binding.ts duplicates admission/bind/projection already owned by the selector seam, and its paragraph-long safety rationales are evidence the boundary is wrong. Consolidate this behind the selector owning interface, then itemize any irreducible growth and why a smaller design was rejected.

All exact-head checks are green and the live iOS evidence restores the original text behavior, but neither covers the dropped configured-runner context.

thymikee pushed a commit that referenced this pull request Aug 19, 2026
Two edits, per find's ADDENDUM.md:

1. `includeRects` returns to `buildRuntimeCaptureInput`. It was removed from
   #1876 as unconsumed; the selector capture path is genuinely its first
   consumer (a Web rect capture requests bounds explicitly), so it lands here
   under the same rule that moved the seam. `snapshot`/`diff` pass nothing.

2. The per-capture `signal` is dropped, not restored. `CaptureSnapshotInput`
   has no such field on this stack — it moved to `wait` (#1875) with the
   regression that proves per-poll abort and quiescence. `get` captures once
   per resolution and never polls, so nothing here needs it. The seam test and
   fixture coverage for it moves with the contract rather than being kept
   against a field that no longer exists.
thymikee pushed a commit that referenced this pull request Aug 19, 2026
Two edits, per find's ADDENDUM.md:

1. `includeRects` returns to `buildRuntimeCaptureInput`. It was removed from
   #1876 as unconsumed; the selector capture path is genuinely its first
   consumer (a Web rect capture requests bounds explicitly), so it lands here
   under the same rule that moved the seam. `snapshot`/`diff` pass nothing.

2. The per-capture `signal` is dropped, not restored. `CaptureSnapshotInput`
   has no such field on this stack — it moved to `wait` (#1875) with the
   regression that proves per-poll abort and quiescence. `get` captures once
   per resolution and never polls, so nothing here needs it. The seam test and
   fixture coverage for it moves with the contract rather than being kept
   against a field that no longer exists.
@thymikee

Copy link
Copy Markdown
Member Author

Blocking: this head is DIRTY against current main, so rebase and validate the resolved exact head first; today’s green CI proves only 8f57e7d. More importantly, the cutover removes dispatchDirectIosSelectorWait/querySelector but adds only a text findText preferred operation. A normal non-recording iOS wait id=… or label=… that the runner finds while the canonical tree misses now times out where it previously succeeded. Put that native selector query behind an admitted owner-provided preferred runtime operation (or preserve equivalent semantics), then add the red regression where query finds while the tree does not plus live iOS evidence. Also itemize the >700 net production-line growth and why a smaller owning-operation design was rejected, and document the deliberate no-app wait behavior change. No ready-for-human.

@thymikee

Copy link
Copy Markdown
Member Author

The predicted selector regression does not reproduce — evidence, and its limits

You asked for the red regression where the runner query finds while the tree does not, plus live iOS evidence. We went looking for that divergence first rather than building the operation, and across every shape tested on iPhone 16 Pro it does not exist. This is the "or preserve equivalent semantics" branch of your ask.

1. Pruned wrapper nodes (Settings root)

Raw 167 → canonical 73, 30 shapes pruned, including four Buttons carrying identifiers. But the addressable sets are identical:

labels in raw but ABSENT from canonical tree: 0
identifiers absent from canonical:            0

Group-pruning collapses duplicate Button-inside-Cell wrappers and the label/identifier survives on the collapsed parent. Live, tree-only, post-retirement:

wait 'id=com.apple.settings.siri' 4000    → {"waitedMs":121}
wait 'id=com.apple.settings.homeScreen'   → {"waitedMs":113}
wait 'label="Apple Intelligence & Siri"'  → {"waitedMs":112}

2. Deep, dense hierarchy (Accessibility) — your scenario if anywhere

Raw 148 → canonical 57, and here the addressable sets genuinely do diverge: 6 labels and 2 identifiers absent from canonical (KEYBOARDS, LIVE_SPEECH_TITLE, Live Speech, Keyboards & Typing, …). Differential test — is still carries the runner shortcut on this base, so it probes the runner directly:

id=KEYBOARDS            RUNNER: Selector did not match   TREE: wait timed out
id=LIVE_SPEECH_TITLE    RUNNER: Selector did not match   TREE: wait timed out
label="Live Speech"     RUNNER: Selector did not match   TREE: wait timed out

Validity check, because a shortcut that silently did not fire would fake this result: runner commands issued by is exists: ['querySelector', 'snapshot']. The runner was genuinely asked, refused, and fell through.

A transient mid-navigation artifact (Keyboards & Typing with zero-size rects at y:116) was ruled out by settling 3 s and re-testing — both runner and tree still refuse.

3. Why it does not diverge — mechanism, not sample

The canonical tree prunes for exactly two reasons, and the runner's behavior lines up with both:

  • duplicate wrapper collapse → the label/id survives on the survivor, so the selector still resolves;
  • off-viewport / virtualized content → the runner refuses these too, being conservative about visibility.

So the pruned set and the runner-resolvable set are complementary, not overlapping. That is a reason to expect non-divergence rather than a lucky sample. It is also consistent with a separate finding this wave: a hypothesis that the direct-iOS path returns pass: true for off-viewport nodes was traced through the code and then refuted on device, because XCUITest's own query refused the node the tree carried.

What was not tested

Stock Settings only — no third-party, React Native, or SwiftUI hierarchies, where pruning could bite differently. No --depth-limited waits, no occlusion, no horizontally off-screen drawer content.

So this is "did not reproduce across the shapes most likely to expose it", not "proved impossible".

What we are not doing, and why

No red regression is being added for this. A test asserting a divergence we cannot produce would be vacuous — green for the wrong reason, and exactly the class this wave has been rejecting elsewhere (the fictional R37 sentinels, the threading-only signal coverage). We would rather leave the claim unasserted than assert it falsely.

If you can name a concrete app and selector that diverges, we will test it immediately and build the admitted preferred operation if it holds. That is a cheap check and the offer is open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant