Skip to content

feat(xai): add TTS websocket pooling#2025

Merged
toubatbrian merged 8 commits into
mainfrom
leeway-skywards-haggle
Jul 14, 2026
Merged

feat(xai): add TTS websocket pooling#2025
toubatbrian merged 8 commits into
mainfrom
leeway-skywards-haggle

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Add xAI TTS support to the JS plugin and configure its pooled websocket lifetime below xAI's undocumented server-side deadline.

The source change only adjusted the xAI TTS websocket pool deadline, but the target repo did not yet have an xAI TTS implementation. This PR ports the missing TTS infrastructure so the deadline behavior exists in agents-js rather than dropping it.

Source diff coverage
  • livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/tts.py: adapted to plugins/xai/src/tts.ts and plugins/xai/src/index.ts. The target had no xAI TTS counterpart, so this ports the missing xAI websocket TTS implementation into the xAI plugin, exports it publicly, and applies the source change as maxSessionDuration: 1_800_000 ms, the JS equivalent of Python's 1800 seconds. The source comment about xAI's undocumented roughly 2100s websocket deadline is preserved next to the pool setting.

Testing

  • pnpm test plugins/xai (8 passed, 1 skipped)
  • pnpm --filter @livekit/agents-plugin-xai build
  • pnpm --filter @livekit/agents-plugin-xai lint
  • pnpm build
  • Added focused websocket tests covering fresh connection reuse, rotation after 1,800,000 ms, multi-segment streaming, and provider capabilities.

The package lint completes with two pre-existing no-explicit-any warnings in plugins/xai/src/_utils.ts.

Runtime validation

  • Live xAI synthesis with the capitalized Python-parity voice Ara returned 24 kHz mono PCM successfully.
  • Two sequential chunked syntheses plus two flushed streaming segments reused one live xAI websocket.
  • Cue voice-mode session sid_07063c0f2886 exercised the changed provider through a complete LiveKit room turn. The greeting emitted xAI TTS usage for 43 characters / 3.13 seconds of audio, and a voice prompt traversed STT → LLM → xAI TTS before committing Pooling works. with tts_node_ttfb about 405 ms.

Ported from livekit/agents#6393

Original PR description

closes #6390

Cue voice E2E

  • Strict voice-mode session: sid_10ecd2a5f336 at head 432b0746.
  • A live xAI websocket served four TTS streams across the greeting and three spoken turns; persisted counters reached connectionCount=1, streamCount=4.
  • QCLI persisted framework events plus full-session and per-command WAV recordings under ~/.cue-cli/sessions/sid_10ecd2a5f336/.
  • The 1,800,000 ms deadline rotation was verified by three deterministic runtime tests rather than holding the live QCLI session open for 30 minutes.

@rosetta-livekit-bot rosetta-livekit-bot Bot requested a review from a team as a code owner July 13, 2026 03:34
@changeset-bot

changeset-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 432b074

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 36 packages
Name Type
@livekit/agents-plugin-xai Patch
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugins-test Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

toubatbrian and others added 5 commits July 14, 2026 11:13
Verify xAI TTS reuses fresh pooled connections and rotates them before the provider deadline.

Co-authored-by: Cursor <cursoragent@cursor.com>
Ensure flushed utterances complete independently while sharing the pooled websocket.

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep multi-segment coverage type-safe under the package declaration build.

Co-authored-by: Cursor <cursoragent@cursor.com>
Lock down the public streaming and audio metadata exposed by the new provider.

Co-authored-by: Cursor <cursoragent@cursor.com>
toubatbrian and others added 2 commits July 14, 2026 11:46
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@toubatbrian toubatbrian merged commit 71b8510 into main Jul 14, 2026
8 checks passed
@toubatbrian toubatbrian deleted the leeway-skywards-haggle branch July 14, 2026 22:39
@github-actions github-actions Bot mentioned this pull request Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant