feat(desktop): route voice tasks to Codex/OpenClaw/Hermes with best-agent selection, fallback, and install help

[21prnv]

What changed and why

Track 1 (Agents): Omi can now hand PTT and typed tasks to OpenClaw, Hermes, and (new) Codex.
Naming an agent dispatches spawn_agent with exactly that provider in the same turn; when
none is named, the model picks the best installed agent (codex: coding, openclaw:
messaging/automation, hermes: research) or the default; a failed model-chosen spawn gets one
same-turn retry with another agent (with recordFallback telemetry); naming a not-installed
agent returns spoken install steps plus an offer to run the install via the default agent.
Codex is a full directed provider: new kernel CodexRuntimeAdapter (ACP via codex-acp,
activated by OMI_CODEX_ADAPTER_COMMAND), Swift routing/detection, and pill/picker UI.

Product invariants affected

• INV-AGENT-* — the kernel stays the sole routing/authority boundary: the new adapter
  registers through ensureRegisteredAdapter; explicit-provider spawns still cross boundaries
  only via assertAdapterAllowedForTopLevelLocalProviderSpawn; selection policy lives in
  kernel sharedSemanticGuidance, not Swift prompt fragments.
• INV-VOICE-1 — spawn acceptance still requires the canonical journal receipt; the new
  failure continuation never writes a turn.
• INV-CHAT-1 — no new transcript stores; pill projection unchanged.
• INV-AUTH-1 — no auth/session-death changes; env seeding only touches
  OMI_*_ADAPTER_COMMAND.

How it was verified

• Live PTT on a real build (omi-agent-routing named bundle, signed in, Codex CLI +
  codex-acp installed and ChatGPT-authenticated): "ask codex to create hello.txt on my
  desktop" → voice model called spawn_agent{provider:"codex"} → kernel spawned the real
  codex-acp subprocess → file created on the Desktop.
• Live fallback: a later turn where STT misheard "codex" as "codes" hit a rejected spawn;
  the same-turn fallback completed the task with the default agent and disclosed the switch
  ("Codex wasn't available, so I retried with the default agent"). The mishear is now handled
  by the directive parser (see Tests).
• cd desktop/macos/agent && npx tsc --noEmit && npx vitest run — 49 files, 563 tests passing.
• cd desktop/macos && ./scripts/agent-logic-harness.sh — passed (Swift lifecycle/state,
  agent runtime focused, pi-mono-extension).
• make preflight with this body — all 18 selected checks pass.
• Not verified live: OpenClaw/Hermes happy paths (CLIs not installed on the test machine);
  they ride the same pre-existing adapter machinery Codex just proved, and are covered by the
  contract suites. The not-installed → install-help path for them is contract-tested.

Tests

Core path: codex-adapter-subprocess.test.ts (real spawned ACP subprocess through the
production adapter) and external-surface-authority.test.ts — "runs an explicitly requested
Codex child to completion through a real ACP subprocess" (realtime spawn_agent →
local_user:codex boundary → run succeeded).
Main error path: external-surface-authority.test.ts — "returns setup instructions when an
uninstalled Codex is explicitly requested" (provider_setup_needed + install command), and
RealtimeSpawnFailureContinuationPolicyTests (one continuation per turn, second failure
terminates, fallback from-label consumed once).
Also: codex directive parsing incl. the observed "codes" STT mishear (PiMonoWiringTests),
detection (codex-acp discovery + missing-prompt install copy), provider enum with
installed/missing split (HubSystemInstructionTests), adapter-selection/capability-matrix and
cross-language contract fixture pins.

Root cause and durable guard (bug fixes)

Two small fixes shipped alongside the feature:

• Spawn failure dead-ended the voice turn. Violated contract: a legible failure should
  leave the user a next step. Durable guard: typed RealtimeSpawnFailureContinuationPolicy
  (bounded one-continuation-per-turn, behaviorally tested) instead of an ad-hoc boolean.
• Directive syntax was exact-match against ASR transcripts ("Tell codes to…" missed
  codex). Durable guard: mishear aliases live in the same providerToken switch, with
  regression tests for the observed transcript and for non-command sentences containing
  "codes".

Scoped cleanups (optional)

• Deprecated @zed-industries/codex-acp replaced by @agentclientprotocol/codex-acp in all
  install hints (own commit).
• AIProvider.swift added to ai-chat-settings.yaml e2e covers: (it backs the provider
  picker that flow drives; fixes a preflight coverage gap).
• Pre-existing, unrelated: journal outbox pump retries a revision conflict forever at 1 Hz —
  observed during testing, tracked separately (spun off as its own task).

[Git-on-my-level]

Thanks for the detailed implementation and validation notes. I did a static maintainer pass on this head.

This looks directionally coherent: Codex is wired through the existing adapter/control-plane abstractions rather than bypassing the kernel, explicit-provider routing remains behind spawn_agent, the setup-needed path is typed, and the one-continuation retry guard is bounded with test coverage. I did not see an obvious malicious/supply-chain pattern in the diff.

I’m not formally approving because this changes security- and product-sensitive desktop agent behavior and needs human maintainer review before merge:

• It adds Codex as a local-user credential scoped execution adapter and forwards CODEX_HOME, so maintainers should confirm the credential/config boundary and local subprocess expectations are acceptable.
• It changes realtime voice tool schemas to advertise missing providers and changes agent instructions so AI agents will dispatch explicitly named Codex/Hermes/OpenClaw tasks, relay install instructions, and offer a consented background install task. That is an intentional agent-behavior change; maintainers should verify the guidance is safe, accurate, and aligned with Omi’s UX/security posture.
• The docs update docs/doc/developer/agent-control-plane.mdx now treats Codex as part of the control-plane adapter model, which can influence future coding/review agents working in this repo.

If maintainers are comfortable with those product/security decisions, the implementation shape and added coverage look promising from this review.

---

by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.