AssemblyAI

OverviewCopy Anchor Link

AssemblyAITTSService streams text to AssemblyAI's Streaming TTS WebSocket API over one session and plays the audio as it arrives, with per-word timings for word-level TTSTextFrames. It is a port of Pipecat's AssemblyAITTSService. AssemblyAI's speech-to-text services, in the same package, are on the AssemblyAI speech-to-text page.

InstallationCopy Anchor Link

bun add @pipedog/assemblyai

PrerequisitesCopy Anchor Link

• An AssemblyAI account and API key, passed as apiKey (the examples read ASSEMBLYAI_API_KEY). A temporary token opens a single session, so it can't carry the service's reconnects and session renewals.

AssemblyAITTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

The default URL is wss://tts.us.assemblyai.com/v1/ws. Use wss://tts.eu.assemblyai.com/v1/ws to process text and audio in the EU. A cloned voice works only on the host of the zone it was created in.

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new AssemblyAITTSService.Settings({...}) (AssemblyAITTSSettings). Settings you leave out keep their defaults. The session is configured when it connects, so changing any of them reconnects.

UsageCopy Anchor Link

import { AssemblyAITTSService } from "@pipedog/assemblyai/tts";
import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { Language } from "@pipedog/core/transcriptions/language";

// Speaks in Audrey, the default voice.
const tts = new AssemblyAITTSService({ apiKey: process.env.ASSEMBLYAI_API_KEY ?? "" });

Pick a voice and a region:

// A library voice by its slug, processed in the EU.
const eu = new AssemblyAITTSService({
  apiKey: process.env.ASSEMBLYAI_API_KEY ?? "",
  url: "wss://tts.eu.assemblyai.com/v1/ws",
  settings: new AssemblyAITTSService.Settings({ voice: "aai-jane", language: Language.EN }),
});

Change settings mid-call with a TTSUpdateSettingsFrame:

// The session is configured at connect time, so a new voice reconnects.
const update = new TTSUpdateSettingsFrame({
  delta: new AssemblyAITTSService.Settings({ voice: "987db62e-a2c2-569a-9c4d-82460cdea1bd" }),
});

NotesCopy Anchor Link

• Each sentence is sent as its own request (a Generate, then a Flush). With TextAggregationMode.TOKEN, the turn's text streams as it is generated and is flushed when the turn ends. Text longer than the session's limit is split across Generate messages.
• Interruptions send Cancel, which stops synthesis in flight and keeps the session open.
• The service keeps an idle session open with a KeepAlive every 30 seconds, replaces a session between requests before its one-hour limit, and waits as long as an error asks before reconnecting.
• sampleRate left out uses the pipeline's output rate.

RuntimesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/assemblyai/package.json: ✅ means the compat smoke test passes on that runtime.

The service uses the global WebSocket (through core's websocketConnect, which sends the Authorization header on the handshake on Cloudflare Workers too). No AssemblyAI SDK is needed.