Deepgram

OverviewCopy Anchor Link

pipedog ports three of Pipecat's Deepgram text-to-speech services:

• DeepgramTTSService: Deepgram Aura over a WebSocket (/v1/speak). Streams audio as text arrives and clears Deepgram's buffer on interruption.
• DeepgramHttpTTSService: Deepgram Aura over HTTP. One POST per sentence, with the audio streamed back in the response body.
• DeepgramFluxTTSService: Deepgram Flux over a WebSocket (/v2/speak). Streams LLM tokens as they arrive and keeps acoustic state across turns on one connection.

InstallationCopy Anchor Link

bun add @pipedog/deepgram

import { DeepgramHttpTTSService, DeepgramTTSService } from "@pipedog/deepgram/tts";
import { DeepgramFluxTTSService } from "@pipedog/deepgram/flux/tts";

PrerequisitesCopy Anchor Link

• A Deepgram account and an API key from the Deepgram console. Flux needs a key with Flux TTS access.
• Pass the key as apiKey. The examples read it from DEEPGRAM_API_KEY.

DeepgramTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new DeepgramTTSService.Settings({...}). Any setting you leave out keeps the default below. DeepgramHttpTTSService uses the same settings class.

UsageCopy Anchor Link

import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { TextAggregationMode } from "@pipedog/core/services/tts_service";
import { DeepgramFluxTTSService } from "@pipedog/deepgram/flux/tts";
import { DeepgramHttpTTSService, DeepgramTTSService } from "@pipedog/deepgram/tts";

const tts = new DeepgramTTSService({
  apiKey: process.env.DEEPGRAM_API_KEY ?? "",
  settings: new DeepgramTTSService.Settings({
    voice: "aura-2-helena-en",
    speed: 1.1,
  }),
});

Change settings mid-call with a TTSUpdateSettingsFrame. Any change closes the WebSocket and reconnects with the new settings:

// Queue it on the worker, or push it from a processor upstream of the service.
// Any change reconnects the WebSocket with the new settings.
const update = new TTSUpdateSettingsFrame({
  delta: new DeepgramTTSService.Settings({ voice: "aura-2-thalia-en" }),
});

Event handlersCopy Anchor Link

NotesCopy Anchor Link

• encoding must be "linear16", "mulaw" or "alaw"; anything else throws in the constructor. The received bytes go into TTSAudioRawFrames as they are, so only "linear16" plays correctly in a pipedog pipeline.
• Text is sent with Speak messages. The service sends Flush when the LLM response ends, and emits TTSStoppedFrame when Deepgram answers with Flushed.
• On interruption it sends Clear, so Deepgram drops its text buffer and stops sending audio.
• sampleRate defaults to the pipeline's output rate.
• Aura-2 reads inline pronunciations. formatPronunciation(word, ipa) renders one as Deepgram's inline object, and returns null when the IPA is empty or too long for the word (over 128 characters, or over 10 times the word's length with a floor of 15).

DeepgramHttpTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Same settings and defaults as DeepgramTTSService.

UsageCopy Anchor Link

const httpTts = new DeepgramHttpTTSService({
  apiKey: process.env.DEEPGRAM_API_KEY ?? "",
  settings: new DeepgramHttpTTSService.Settings({ voice: "aura-2-helena-en" }),
});

NotesCopy Anchor Link

• Each request is a POST to {baseUrl}/v1/speak with container=none. A status other than 200 becomes an ErrorFrame with the response body.
• The response body is read as a stream and cut into sample-aligned TTSAudioRawFrames, so audio starts before the whole response has arrived.
• Upstream requires an aiohttp_session. Here fetch is optional and defaults to the global fetch.

DeepgramFluxTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new DeepgramFluxTTSService.Settings({...}).

UsageCopy Anchor Link

const fluxTts = new DeepgramFluxTTSService({
  apiKey: process.env.DEEPGRAM_API_KEY ?? "",
  settings: new DeepgramFluxTTSService.Settings({
    voice: "flux-alexis-en",
    speed: 1.05,
    expressivity: 1,
  }),
});

fluxTts.addEventHandler("on_connection_error", async (_service, error: string) => {
  console.error("Flux connection error:", error);
});

Flux streams LLM tokens by default (TextAggregationMode.TOKEN). To aggregate text into sentences first:

// Aggregate text into sentences instead of streaming LLM tokens.
const fluxSentences = new DeepgramFluxTTSService({
  apiKey: process.env.DEEPGRAM_API_KEY ?? "",
  textAggregationMode: TextAggregationMode.SENTENCE,
});

A speed change is applied to the open connection with a Flux Configure message, so the cross-turn acoustic state is kept. Any other change (voice, expressivity, language) reconnects:

// A speed change is sent on the open connection with a Configure message.
const fluxSpeed = new TTSUpdateSettingsFrame({
  delta: new DeepgramFluxTTSService.Settings({ speed: 0.9 }),
});

Event handlersCopy Anchor Link

NotesCopy Anchor Link

• Audio is always requested as linear16. Supported sample rates are 8000, 16000, 24000, 32000, 44100 and 48000 Hz. Another rate only logs a warning on connect.
• On interruption the service sends Interrupt, which ends the active turn without closing the connection. Flush ends a turn when the LLM response is done.
• Clearing speed (setting it to null) sends Configure with speed: 1.0, Flux's default.
• Disconnecting closes the socket without a Close message: in Flux, Close asks the server to finish the active turn's audio first.
• formatPronunciation is left at the base implementation, so words are spoken as written. Flux rejects inline pronunciations on a connection whose speed is not 1.0.
• tag adds one tag query parameter per entry, for Deepgram usage reports.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/deepgram/package.json: ✅ means the compat smoke test passes on that runtime.

DeepgramTTSService and DeepgramFluxTTSService use the global WebSocket and send the key as an Authorization: Token <key> header on the handshake. That works on Bun, Node, Deno and Cloudflare Workers (where the socket is opened through fetch). Pass webSocketFactory to open the socket some other way, through a proxy for example. A standard WebSocket does not expose the handshake's response headers, so the dg-* headers upstream logs on connect are not available.

DeepgramHttpTTSService only needs fetch, available on all four runtimes.

Query values use JS number formatting, so speed: 1.0 is sent as speed=1. The deprecated voice constructor option is not ported; use settings.voice.

Not ported yet: DeepgramSageMakerTTSService and DeepgramFluxSageMakerTTSService (AWS SageMaker endpoints).