ElevenLabs

OverviewCopy Anchor Link

ElevenLabs turns text into speech with its Flash, Turbo, Multilingual and v3 models. pipedog ports all three of Pipecat's ElevenLabs text-to-speech services:

• ElevenLabsTTSService: streams over ElevenLabs' multi-context WebSocket API (multi-stream-input), with word-level timestamps and per-turn audio contexts.
• ElevenLabsHttpTTSService: posts each request to the stream/with-timestamps HTTP endpoint and streams the audio and alignment back.
• ElevenLabsDialogueTTSService: streams over ElevenLabs' multi-context Text to Dialogue WebSocket, which serves the Eleven v3 and v4 dialogue models.

InstallationCopy Anchor Link

bun add @pipedog/elevenlabs

import { ElevenLabsDialogueTTSService } from "@pipedog/elevenlabs/dialogue/tts";
import { ElevenLabsHttpTTSService, ElevenLabsTTSService } from "@pipedog/elevenlabs/tts";

PrerequisitesCopy Anchor Link

• An ElevenLabs account and an API key from the ElevenLabs dashboard.
• A voice id from the ElevenLabs voice library. There is no default voice.
• Pass the key as apiKey. The examples read it from ELEVENLABS_API_KEY.

ElevenLabsTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new ElevenLabsTTSService.Settings({...}). Any setting you leave out keeps the default below. null means the field is not sent, so ElevenLabs uses its own default.

UsageCopy Anchor Link

import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { TextAggregationMode } from "@pipedog/core/services/tts_service";
import { Language } from "@pipedog/core/transcriptions/language";
import { ElevenLabsHttpTTSService, ElevenLabsTTSService } from "@pipedog/elevenlabs/tts";

const tts = new ElevenLabsTTSService({
  apiKey: process.env.ELEVENLABS_API_KEY ?? "",
  settings: new ElevenLabsTTSService.Settings({
    voice: "your-voice-id",
    model: "eleven_flash_v2_5",
    language: Language.EN,
    stability: 0.5,
    similarityBoost: 0.75,
  }),
});

Token streaming:

// Stream LLM tokens instead of whole sentences. autoMode then defaults to false.
const streaming = new ElevenLabsTTSService({
  apiKey: process.env.ELEVENLABS_API_KEY ?? "",
  textAggregationMode: TextAggregationMode.TOKEN,
  settings: new ElevenLabsTTSService.Settings({ voice: "your-voice-id" }),
});

Change settings mid-call with a TTSUpdateSettingsFrame:

// Queue it on the worker, or push it from a processor upstream of the service.
// A voice-character change (stability, speed, ...) starts a new audio context;
// a voice, model or language change reconnects.
const update = new TTSUpdateSettingsFrame({
  delta: new ElevenLabsTTSService.Settings({ speed: 1.1 }),
});

Event handlersCopy Anchor Link

Inherited from TTSService:

NotesCopy Anchor Link

• The service does not push the LLM's text frames itself. It builds TTSTextFrames from ElevenLabs' word alignment, so the context only records words the user actually heard.
• Text is aggregated into sentences by default. autoMode left null is true in sentence mode and false in token mode, where ElevenLabs' chunk scheduler needs to gather enough text.
• Changing voice, model or language reconnects, since they are part of the WebSocket URL. Changing stability, similarityBoost, style, useSpeakerBoost or speed closes the current audio context, so the next one opens with the new voice settings.
• language is checked against the model's language list. A model that takes no language code, or doesn't cover the language, gets none, and this is logged.
• The output format is chosen from the sample rate: 8000, 16000, 22050, 24000, 32000, 44100 or 48000 Hz. Any other rate falls back to pcm_24000 with a warning.
• pronunciationTransformIpa() renders IPA as SSML <phoneme> tags. Only eleven_flash_v2 and eleven_turbo_v2 read them, and this service also needs enableSsmlParsing: true. Otherwise the transform is skipped.
• Other settings changes (such as applyTextNormalization) are not applied mid-call and are logged as unhandled, unless a URL change reconnects at the same time.

ElevenLabsHttpTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

UsageCopy Anchor Link

const httpTts = new ElevenLabsHttpTTSService({
  apiKey: process.env.ELEVENLABS_API_KEY ?? "",
  settings: new ElevenLabsHttpTTSService.Settings({
    voice: "your-voice-id",
    optimizeStreamingLatency: 2,
    speed: 1.1,
  }),
});

NotesCopy Anchor Link

• Word timestamps come from the with-timestamps endpoint, so this service also emits TTSTextFrames aligned with playback.
• Earlier text in the turn is sent as previous_text, for continuity between requests.
• enableLogging: false turns on ElevenLabs' zero retention mode (enterprise only).
• IPA pronunciations (pronunciationTransformIpa()) work with eleven_flash_v2 and eleven_turbo_v2 only.

ElevenLabsDialogueTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new ElevenLabsDialogueTTSService.Settings({...}).

A model or language change reconnects the WebSocket. A voice or stability change closes the current audio context, and the next one registers the new values.

UsageCopy Anchor Link

import { ElevenLabsDialogueTTSService } from "@pipedog/elevenlabs/dialogue/tts";

// Eleven v3 and v4 dialogue models. Needs a voice id.
const dialogue = new ElevenLabsDialogueTTSService({
  apiKey: process.env.ELEVENLABS_API_KEY ?? "",
  settings: new ElevenLabsDialogueTTSService.Settings({
    voice: "your-voice-id",
    model: "eleven_v4_turbo",
    stability: 0.5,
  }),
});

NotesCopy Anchor Link

• Text is always aggregated by sentence, and a trailing space is added to each input, because the server joins inputs verbatim. A textAggregationMode other than SENTENCE is ignored with a warning.
• Eleven v3 and v4 perform inline audio tags such as [laughs]. They come back as spoken characters in the alignment, so they reach the LLM context unless a text transform removes them.
• A keepalive context is registered when the socket opens, because the server drops a connection that is idle for 20 seconds and rejects a keepalive that names no registered context. Set a voice so it can be registered.
• pronunciationTransformIpa() renders pronunciations as IPA between slashes, which Eleven v3 reads.
• seed makes generation reproducible.

Runtime notesCopy Anchor Link

• ElevenLabsTTSService uses the global WebSocket and sends the key as an xi-api-key header on the handshake. That works on Bun, Node and Deno, and on Cloudflare Workers, where the socket is opened through fetch. Upstream sets a 16 MB message size limit; the standard WebSocket has none.
• ElevenLabsDialogueTTSService uses the same WebSocket and xi-api-key header as ElevenLabsTTSService, so it runs wherever that does.
• ElevenLabsHttpTTSService uses fetch. Upstream's required aiohttp_session is replaced by an optional fetch option (the global fetch by default).
• Not ported: the deprecated PronunciationDictionaryLocator, the InputParams classes and the voiceId, model, params, pronunciationDictionaryLocators and aggregateSentences constructor options. Use settings and textAggregationMode.

Generated from pipecat.runtimes and pipecat.compat in packages/elevenlabs/package.json: ✅ means the compat smoke test passes on that runtime.