feat(moq-audio): cancel the speaker's echo out of the microphone

[kixelated]

Blocked on a dependency release, deliberately not merged. sonora 0.1.0 panics when its adaptive filter shrinks, and this workspace builds panic = "abort", so an enabled aec feature aborts the process partway through a long call. The fix is merged upstream but unreleased (sonora#24). Bump to 0.1.1 and merge; the code here needs no change. Details under Dependency.

Summary

moq-audio can now both capture and play, which means it can also hear itself: on any laptop without a headset the microphone picks up the speaker and sends the whole call back. That is the second of the two blockers #2282 names, and the half of #2478 that #2529 left open.

playback::Engine::canceller taps the post-mix signal and capture::Config::aec subtracts it from the microphone, so buffers leaving capture are already clean and encode::publish_capture is untouched. Two things you get from an open output device now: sink() to play, canceller() to stop hearing it.

The engine, not iroh-live's API, is what carried over. Where the shape differs:

• A canceller is a handle, not a graph node. No AudioSink/AudioSource traits or firewheel nodes; Config in, an owned Canceller out, Drop removes the tap.
• The tap is a fixed-resample channel, not a raw ring. It resamples the device's rate to a fixed 48 kHz, so switching to a 44.1 kHz output doesn't invalidate an adaptive filter that spent seconds converging, and it costs nothing on the 48 kHz devices that are the norm. Theirs pinned the whole engine to one internal rate to get the same property.
• 20 ms of reference queue, not ~100 ms. The reference has to reach the canceller before the echo reaches the microphone, and everything queued is delay the output hardware has to make up for.
• The tap is drained to empty while cancellation is off, and on every microphone open. Nothing reads it while the microphone is closed, and capture is demand-gated, so a resumed call would otherwise start on a reference describing whatever was playing minutes ago.
• The adaptive filter survives demand gating. Reopening the same format keeps the processor and only drops the samples in flight, since it is still the same room. Rebuilding it on every subscriber transition would cost a re-convergence each time.
• Neither audio callback allocates or frees. Planar scratch is channel-major in one buffer, split with split_at_mut per channel count rather than collecting a Vec<&[f32]> per 10 ms frame. On the output side the tap owns a heap ring, so a replaced or cleared one leaves by the same retire channel a removed sink uses; that channel now carries a Retired enum instead of a bare sink entry.

Root cause of a deadlock found while writing this

The tap registration is the mirror of a sink's, and the driver rebuilds it inside its own lock. Holding a strong handle on the canceller there means the last one can drop there, and its Drop deregisters, which re-enters that same lock. State sits behind its own Arc so the driver only ever holds a Weak on the canceller itself and never upgrades it.

Dependency

One new crate, optional: sonora, a pure-Rust port of WebRTC's audio processing (AEC3, noise suppression, AGC2). BSD-3-Clause like the upstream it was ported from, already on deny.toml's allow list, and it adds no C++ toolchain, which was the whole reason #2478 asked whether a pure-Rust AEC existed. No duplicate versions enter the tree; no git dependency, per that issue's prerequisite.

The maturity check that issue asked for turned up a real caveat. 0.1.0 is the only release and it panics when the adaptive filter shrinks (sonora#14) — reported in the wild after a 700 s call on a MacBook Air, i.e. something a long call on real hardware does reach. The one-line fix is upstream and merged (#15) but unreleased, and a release request is open and unanswered (#24). This workspace builds panic = "abort", so there is nothing to catch in-process.

That is why aec is a non-default feature and why nothing else in the repo turns it on: enabling it today is opting into a known upstream panic on long calls. The code is unaffected by the fix, so a cargo update to 0.1.1 is the whole remedy. Worth deciding before merge whether to hold this until 0.1.1 exists, ping upstream, or land it opt-in as-is. The caveat and the tracking links are recorded next to the dependency in Cargo.toml.

Public API changes

All additive, all behind the new non-default aec feature. Nothing renamed, removed, or signature-changed, so this targets main.

• moq_audio::aec (new module)
  • aec::Config — #[non_exhaustive], Default, with noise_suppression: bool and auto_gain: bool (both on, matching what a browser gives a call).
  • aec::Canceller — Clone + Debug, with set_enabled(bool) / enabled() -> bool. Everything else on it is pub(crate).
• moq_audio::playback::Engine::canceller(aec::Config) -> aec::Canceller.
• moq_audio::capture::Config::aec: Option<aec::Canceller> — a new field on a #[non_exhaustive] struct, so callers already build it via default().

playback::{Shared, BUS_CHANNELS} became pub(crate) re-exports for the aec module; neither is public surface.

Cross-Package Sync

moq-audio has no row in the table and no doc/ page. moq-ffi and wrapper exposure is explicitly out of scope in #2478; nothing in the FFI surface changed. rs/CLAUDE.md's crate map is updated.

Test plan

cargo nextest run -p moq-audio --all-features — 74 passed. Clippy -D warnings across --all-features, --no-default-features, --features capture, and --features playback, plus the workspace --all-features / --no-default-features gates; cargo doc -D warnings, fmt, sort, shear, and deny check all clean.

New tests, all hardware-free:

• It actually cancels. A synthetic room: noise into the reference, the same noise attenuated and delayed into the microphone, nobody talking. The last half second comes back 51 dB below what went in (asserted at 20 dB). This is the test that fails if the reference path is wired up wrong rather than merely compiling.
• Frame accumulation: whole 10 ms callbacks pay no latency; a callback that straddles a boundary is short by the leftover exactly once and never loses samples across 20 rounds of 15 ms callbacks.
• Disabled is a bit-exact passthrough, which is also the assertion that sonora is not being called on that path.
• Turning it off drops the partial frame in flight, rather than replaying it seconds later.
• A missing reference, a missing microphone, and a format sonora would reject are each handled without dropping audio.
• A second canceller takes the engine's one tap, and the first does not take it back on its way out.
• On the mixer side: the tap receives the summed and clipped mix rather than any one sink, and a replaced or cleared tap is handed back to the driver rather than dropped in the callback.

One #[ignore]d device round trip (taps_a_real_output_device) confirms a real output device reaches the tap; run it with cargo test -p moq-audio --features aec -- --ignored. Whether the microphone then stops hearing the speaker is an acoustic question about the room, so it is a manual check, and CI compiles this feature without running it.

Closes #2478.

(Written by Claude Opus 5)

[coderabbitai]

Warning

Review limit reached

@kixelated, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 2 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f4bf369b-540a-4614-9247-1395f8a9edbd

📥 Commits

Reviewing files that changed from the base of the PR and between 1c2584f and 8c44ed5.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (8)

• rs/CLAUDE.md
• rs/moq-audio/Cargo.toml
• rs/moq-audio/src/aec.rs
• rs/moq-audio/src/capture.rs
• rs/moq-audio/src/lib.rs
• rs/moq-audio/src/playback.rs
• rs/moq-audio/src/playback/driver.rs
• rs/moq-audio/src/playback/mixer.rs

✨ Finishing Touches

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch claude/audio-aec

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}