Skip to content

fix(macos): sign nested runner code so the host runner app can launch - #1979

Merged
thymikee merged 4 commits into
mainfrom
claude/agent-device-runner-damaged-app-923332
Aug 24, 2026
Merged

fix(macos): sign nested runner code so the host runner app can launch#1979
thymikee merged 4 commits into
mainfrom
claude/agent-device-runner-damaged-app-923332

Conversation

@thymikee

@thymikee thymikee commented Aug 23, 2026

Copy link
Copy Markdown
Member

Summary

Repair unsigned macOS runner products by signing their embedded Apple test-support code explicitly, then signing the parent app.

The repair preserves identifiers, entitlements, flags, and hardened-runtime metadata while allowing codesign to regenerate designated requirements for the ad-hoc identity. It avoids deprecated signing-time --deep; deep traversal remains verification-only.

Post-repair strict verification is retained so an invalid cached or freshly built product cannot be reported as repaired.

Validation

  • Observed the new ordering/metadata regression test fail against the previous --force --deep implementation, then pass after the repair.
  • pnpm check:affected --run: format, lint, typecheck, layering, fallow, build, and 2,317/2,318 related tests passed; the sole timeout passed in isolation in 126 ms and is unrelated to this two-file change.
  • Fresh pnpm build:xcuitest:macos product passed codesign --verify --deep --strict, including its nested .xctest bundle and all embedded test-support items.
  • Real CLI open launched TextEdit and reached the macOS XCTest runner. Interactive snapshot then hit the existing host Automation-permission timeout; the isolated session closed cleanly.

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.37 MB 2.37 MB +326 B
JS gzip 797.9 kB 798.0 kB +126 B
npm tarball 919.3 kB 919.4 kB +115 B
npm unpacked 3.19 MB 3.19 MB +326 B

npm unpacked components

Component Base Current Diff
JS / dist source 2.52 MB 2.52 MB +326 B
Apple runner source/project 564.3 kB 564.3 kB 0 B
macOS helper source 54.5 kB 54.5 kB 0 B
Android helper artifacts 0 B 0 B 0 B
Other package files 44.5 kB 44.5 kB 0 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 22.7 ms 22.5 ms -0.2 ms
CLI --help 63.4 ms 62.1 ms -1.3 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/runner-disposal.js +326 B +126 B

Top changed packed files

Packed file Base Current Diff
dist/src/runner-disposal.js 41.0 kB 41.3 kB +326 B

@thymikee

Copy link
Copy Markdown
Member Author

Blocking: do not sign with codesign --force --deep. Apple deprecates signing --deep and documents that this combination forcibly replaces every nested signature; this target also declares macOS network client/server entitlements, so the PR's no-entitlements safety claim is not an invariant. Repair the known Testing, XCTest, and XCUIAutomation frameworks explicitly bottom-up, then sign the parent product; retain --deep --strict only for verification. Pin the exact signing order and metadata policy in the test. Post-repair verification and the real macOS replay evidence otherwise look good.

@thymikee

Copy link
Copy Markdown
Member Author

Addressed the signing-safety finding in 463643f: embedded Apple test-support items are now signed explicitly bottom-up, the parent app is signed last, and signing-time --deep is gone. Identifiers, entitlements, flags, and runtime metadata are preserved; designated requirements are intentionally regenerated for the ad-hoc identity. A fresh macOS product and its nested xctest now pass codesign --verify --deep --strict, and a real CLI open launches the repaired runner. The focused regression test was observed red before the change.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Aug 23, 2026
@thymikee

Copy link
Copy Markdown
Member Author

Follow-up a2963e20f removes the hard-coded XCTest framework inventory. Repair now signs the deterministic contents of the product’s Contents/Frameworks directory before the parent, so Xcode’s actual output is authoritative. The arbitrary-name regression test was observed failing against the previous list and passing after discovery. pnpm check:affected --run passed all runnable gates (2,318 tests), and a fresh Xcode 26.2 macOS runner repaired through the production function passed codesign --verify --deep --strict.

@thymikee

Copy link
Copy Markdown
Member Author

CI diagnosis: Typecheck and FreeRange both fail on the same deterministic Android error from current main, not this PR: snapshot-quality.test.ts still imports resetAndroidSnapshotHelperSessions from snapshot-helper-session.ts, while #1974 moved that API to snapshot-helper-session-lifecycle.ts. The macOS Smoke and Swift Runner Host XCTests pass on this head.

Focused main hotfix: #1980. Once that lands, re-running/rebasing this PR should clear these unrelated gates.

macOS runner builds pass CODE_SIGNING_ALLOWED=NO, so Xcode embeds
Testing/XCTest/XCUIAutomation.framework without re-signing them while the
embed step drops their sealed Modules/*.swiftinterface entries. Those three
nested seals are therefore always broken in a fresh macOS build.

repairMacOsRunnerProductsIfNeeded signed only the outer bundle, which leaves
the nested seals untouched. `codesign --verify --deep --strict` kept failing,
macOS refused to launch the app ("AgentDeviceRunnerUITests-Runner.app is
damaged and can't be opened"), and the runner was killed before it could
establish a connection. codesign still exited 0, so the repair reported
success on every attempt while fixing nothing.

Sign with --deep so nested code is re-sealed, and re-verify afterwards so a
repair that repairs nothing raises RUNNER_PRODUCT_REPAIR_FAILED instead of
passing silently. The runner is ad-hoc signed with no entitlements, so --deep
discards nothing.
…ures

The suite redirects TMPDIR per run and removes it after every worker, so the
hand-rolled afterEach cleanup was redundant. Use mkdtempForTestSync like the
sibling runner tests do.
@thymikee
thymikee force-pushed the claude/agent-device-runner-damaged-app-923332 branch from a2963e2 to 7207e96 Compare August 23, 2026 16:11
@thymikee
thymikee merged commit c78095c into main Aug 24, 2026
28 checks passed
@thymikee
thymikee deleted the claude/agent-device-runner-damaged-app-923332 branch August 24, 2026 05:56
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-24 05:57 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant