Skip to content

feat(mobile): add voice input#4535

Open
iscekic wants to merge 15 commits into
mainfrom
feature/mobile-voice-input
Open

feat(mobile): add voice input#4535
iscekic wants to merge 15 commits into
mainfrom
feature/mobile-voice-input

Conversation

@iscekic

@iscekic iscekic commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add native speech-to-text input to Kilo Chat and cloud-agent composers
  • preserve editable drafts and settle final recognition before sending or creating sessions
  • handle native permissions, Android continuous recognition, lifecycle cleanup, accessibility, and privacy-safe errors

Test Plan

  • pnpm format
  • pnpm typecheck
  • pnpm lint
  • pnpm check:unused
  • pnpm test (112 files, 693 tests)
  • pnpx expo-doctor (19/19)
  • fresh iOS development build and permission-state E2E
  • fresh Android API 35 development build and continuous-listening E2E
  • independent mobile review

E2E Coverage

  • verified microphone permission grant, denial, permanent denial, and Settings recovery
  • verified accessible start/stop controls, listening status, disabled input/attachments, and clean return to idle
  • verified recognition never auto-sends or auto-creates a session
  • verified no audio persistence, upload, or transcript telemetry before explicit submission

@iscekic iscekic self-assigned this Jul 14, 2026
@kilo-code-bot

kilo-code-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 2
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/mobile/src/lib/voice-input/voice-input-submit.ts 29 Pending UI state disables the voice hook before settlement completes, racing stop() with abort() and allowing submission with a stale interim transcript.
apps/mobile/src/lib/voice-input/voice-input-controller.ts 178 A canceled queued start serializes ownership before checking cancellation, so canceling one owner can abort another owner's active recognition session.
Files Reviewed (19 files)
  • apps/mobile/src/app/(app)/agent-chat/new.tsx - 0 issues
  • apps/mobile/src/components/agents/chat-composer.tsx - 0 issues
  • apps/mobile/src/components/agents/new-session-repository-section.tsx - 0 issues
  • apps/mobile/src/components/agents/new-session-repository-state.test.ts - 0 issues
  • apps/mobile/src/components/agents/new-session-repository-state.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/hooks/use-conversation-message-controller.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-content.tsx - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-state.test.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-state.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-types.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-view.tsx - 0 issues
  • apps/mobile/src/lib/voice-input/use-voice-input-actions.ts - 0 issues
  • apps/mobile/src/lib/voice-input/use-voice-input.test.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-controller-helpers.test.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-controller-helpers.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-controller-pending-start.test.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-controller.ts - 1 issue
  • apps/mobile/src/lib/voice-input/voice-input-submit.test.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-submit.ts - 1 issue

Fix these issues in Kilo Cloud

Previous Review Summary (commit a4ad1f1)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit a4ad1f1)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 2
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/mobile/src/lib/voice-input/voice-input-submit.ts 29 Pending UI state disables the voice hook before settlement completes, racing stop() with abort() and allowing submission with a stale interim transcript.
apps/mobile/src/lib/voice-input/voice-input-controller.ts 178 A canceled queued start serializes ownership before checking cancellation, so canceling one owner can abort another owner's active recognition session.
Files Reviewed (18 files)
  • apps/mobile/src/app/(app)/agent-chat/new.tsx - 0 issues
  • apps/mobile/src/components/agents/chat-composer.tsx - 0 issues
  • apps/mobile/src/components/agents/new-session-repository-section.tsx - 0 issues
  • apps/mobile/src/components/agents/new-session-repository-state.test.ts - 0 issues
  • apps/mobile/src/components/agents/new-session-repository-state.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/hooks/use-conversation-message-controller.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-content.tsx - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-state.test.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-state.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-types.ts - 0 issues
  • apps/mobile/src/components/kilo-chat/message-input-view.tsx - 0 issues
  • apps/mobile/src/lib/voice-input/use-voice-input-actions.ts - 0 issues
  • apps/mobile/src/lib/voice-input/use-voice-input.test.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-controller-helpers.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-controller-pending-start.test.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-controller.ts - 1 issue
  • apps/mobile/src/lib/voice-input/voice-input-submit.test.ts - 0 issues
  • apps/mobile/src/lib/voice-input/voice-input-submit.ts - 1 issue

Fix these issues in Kilo Cloud


Reviewed by gpt-5.6-sol · Input: 86.9K · Output: 13.6K · Cached: 487.8K

Review guidance: REVIEW.md from base branch main

@iscekic

iscekic commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Addressed all five findings in 39db7b5 after merging latest main:

  • serialized and owner-cancelled pending voice starts, including submit during permission acquisition
  • locked New Session, Cloud Agent, and Kilo Chat submissions through voice settlement and the actual async submit
  • preserved cached repositories when a background refetch fails
  • added regression coverage for pending starts, duplicate submissions, and cached repository errors

Validated with mobile format, typecheck, lint, check:unused, and 700 tests.

@iscekic

iscekic commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Local iOS dev-build E2E uncovered and fixed a Hermes runtime incompatibility in 3d4615e (Promise.withResolvers was unavailable). Added a regression that runs the queue with that API removed.

After reloading the fixed bundle, verified the native voice startup path without injecting or manipulating audio: both Speech Recognition and microphone permission prompts completed, AVAudioSession reported record permission granted, and iOS launched the local speech-recognition service / AudioUnit-Speech discovery for com.kilocode.kiloapp. The simulator returned to idle because no audio was supplied, which is the intended scope of this check.

Fresh verification: format, typecheck, lint, check:unused, and 701 tests passed.

@iscekic iscekic requested a review from eshurakov July 14, 2026 21:09
@iscekic iscekic requested a review from jeanduplessis July 14, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants