speechmatics: expose end_of_turn_config and vad_config options - #6945
speechmatics: expose end_of_turn_config and vad_config options#6945rrfunde wants to merge 3 commits into
Conversation
Adds `end_of_turn_config` (EndOfTurnConfig) and `vad_config` (VoiceActivityConfig) constructor kwargs to the Speechmatics STT plugin. Both are added to STTOptions and to the advanced-params override whitelist in _prepare_config(), so a provided value replaces the preset's object and an omitted kwarg leaves preset behaviour unchanged. Previously these were unreachable from the integration, which made it impossible to tune adaptive endpointing latency (penalty multipliers, min_end_of_turn_delay, client-side VAD silence_duration) on phone calls where short pauses split turns. Fixes livekit#6924
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 10803167e9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "prefer_current_speaker", | ||
| "punctuation_overrides", | ||
| "speaker_sensitivity", | ||
| "vad_config", |
There was a problem hiding this comment.
When a caller leaves the default EXTERNAL mode (or supplies a real external vad) and passes a VoiceActivityConfig with enabled=True, this override re-enables the Speechmatics client-side VAD while the constructor has already loaded or retained the external VAD. SpeechStream._process_vad() then still calls client.finalize() for each external END_OF_SPEECH, so both endpointing paths can finalize the same turn and produce premature or duplicate boundaries. Reject this combination or disable/reconcile the external VAD when applying an enabled client VAD configuration.
Useful? React with 👍 / 👎.
Combining an enabled client-side VAD with an external VAD (or manual finalize()) would endpoint the same audio twice, producing premature or duplicate turn boundaries. Raise a ValueError for that combination and document the restriction.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d4812737cf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| resolved_vad_config = vad_config if is_given(vad_config) else None | ||
| if ( | ||
| resolved_vad_config is not None | ||
| and resolved_vad_config.enabled | ||
| and turn_detection_mode == TurnDetectionMode.EXTERNAL |
There was a problem hiding this comment.
When vad is omitted and an enabled vad_config is supplied with the default EXTERNAL mode, the constructor imports and calls SileroVAD.load() at lines 295–304 before reaching this validation. Since the Speechmatics package does not declare Silero as a dependency, affected callers can receive the unrelated Silero ImportError instead of the documented configuration error; when Silero is installed, its model is loaded needlessly before construction fails. Move this check ahead of the EXTERNAL-mode auto-load while retaining it after the supplied-VAD mode resolution.
Useful? React with 👍 / 👎.
|
Good catch — fixed in d481273: the constructor now raises |
An enabled vad_config with EXTERNAL mode now raises before Silero is imported/loaded, so callers get the configuration error instead of an unrelated ImportError (or a needlessly loaded model).
|
Fixed in cf4bf51: the vad_config conflict is now validated before the Silero auto-load, so callers get the configuration error instead of an unrelated Silero ImportError (or a needlessly loaded model). |
2 participants
Summary
end_of_turn_config: NotGivenOr[EndOfTurnConfig]andvad_config: NotGivenOr[VoiceActivityConfig]constructor kwargs to the Speechmatics STT plugin (both types are already public exports ofspeechmatics-voice).STTOptionsand added to theadvanced_paramsoverride whitelist in_prepare_config(), so a provided value replaces the preset's object and an omitted kwarg changes nothing — no behavior change for existing users.This makes adaptive endpointing tunable again: with the preset defaults, effective pause tolerance is ~0.25–0.45s regardless of
end_of_utterance_silence_trigger(the ×0.2 VAD-stop penalty andmin_end_of_turn_delayfloor were unreachable), which splits natural mid-sentence pauses on phone calls into multiple turns. Previously the only workaround was subclassingSTTand mutating private internals.Fixes #6924
Testing
tests/test_plugin_speechmatics_stt.py: presets are used when kwargs omitted, provided objects replace the preset's, options stored.uv run pytest tests/test_plugin_speechmatics_stt.py— 6 passedruff format/checkandmypyclean