Google

OverviewCopy Anchor Link

pipedog ports Pipecat's three Google text-to-speech services:

• GoogleTTSService: Google Cloud Text-to-Speech streaming synthesis, for low latency. Supports Chirp 3 HD and Journey voices, and Chirp 3 custom voices through a voice cloning key.
• GoogleHttpTTSService: Cloud Text-to-Speech unary synthesis. Works with every Google voice (Standard, WaveNet, Neural2, ...) and builds SSML for pitch, rate, volume, emphasis, gender and style on voices that accept it.
• GeminiTTSService: Gemini TTS models, with style prompts, expressive tags and multi-speaker output. Runs against Cloud Text-to-Speech (service account) or the Gemini API (API key).

InstallationCopy Anchor Link

bun add @pipedog/google

The Cloud Text-to-Speech classes (and the Cloud backend of GeminiTTSService) also need the Node SDK, an optional peer dependency:

bun add @google-cloud/text-to-speech

import { GeminiTTSService, GoogleHttpTTSService, GoogleTTSService } from "@pipedog/google/tts";

PrerequisitesCopy Anchor Link

• Cloud Text-to-Speech: a Google Cloud project with the Text-to-Speech API enabled, and a service account key (file or JSON string) or Application Default Credentials (ADC). The examples read GOOGLE_APPLICATION_CREDENTIALS (a key file path) or GOOGLE_CREDENTIALS_JSON (the key itself).
• Gemini API backend of GeminiTTSService: a Gemini API key from Google AI Studio, read from GOOGLE_API_KEY.

GoogleTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new GoogleTTSService.Settings({...}).

UsageCopy Anchor Link

import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { Language } from "@pipedog/core/transcriptions/language";
import { GoogleHttpTTSService, GoogleTTSService } from "@pipedog/google/tts";

const tts = new GoogleTTSService({
  credentialsPath: process.env.GOOGLE_APPLICATION_CREDENTIALS ?? "",
  settings: new GoogleTTSService.Settings({
    voice: "en-US-Chirp3-HD-Charon",
    language: Language.EN_US,
    speakingRate: 1.1,
  }),
});

Change settings mid-call with a TTSUpdateSettingsFrame:

const update = new TTSUpdateSettingsFrame({
  delta: new GoogleTTSService.Settings({ voice: "en-US-Chirp3-HD-Aoede" }),
});

NotesCopy Anchor Link

• With voiceCloningKey, the request uses that Chirp 3 custom voice instead of voice.
• Audio is requested as PCM at the pipeline's output sample rate.

GoogleHttpTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

UsageCopy Anchor Link

// Any Google voice. Voices other than Chirp and Journey get SSML prosody.
const httpTts = new GoogleHttpTTSService({
  credentials: process.env.GOOGLE_CREDENTIALS_JSON ?? "",
  settings: new GoogleHttpTTSService.Settings({
    voice: "en-US-Neural2-F",
    language: Language.EN_US,
    pitch: "+2st",
    rate: "110%",
    volume: "+2dB",
  }),
});

NotesCopy Anchor Link

• Chirp and Journey voices (matched by name) don't accept SSML, so the service sends plain text and the SSML settings have no effect, including on the default voice. Use speakingRate for them. Other voices get SSML, with the text XML-escaped.

GeminiTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

UsageCopy Anchor Link

With the Gemini API backend:

import { Language } from "@pipedog/core/transcriptions/language";
import { GeminiTTSService } from "@pipedog/google/tts";

// An apiKey selects the Gemini API (GenAI) backend.
const tts = new GeminiTTSService({
  apiKey: process.env.GOOGLE_API_KEY ?? "",
  settings: new GeminiTTSService.Settings({
    model: "gemini-3.1-flash-tts-preview",
    voice: "Kore",
    language: Language.EN_US,
  }),
});

With the Cloud Text-to-Speech backend and a style prompt:

// Service account credentials (and no apiKey) select Cloud Text-to-Speech.
const styled = new GeminiTTSService({
  credentialsPath: process.env.GOOGLE_APPLICATION_CREDENTIALS ?? "",
  settings: new GeminiTTSService.Settings({
    voice: "Charon",
    prompt: "Say this in a calm, friendly tone.",
  }),
});

Multi-speaker (Cloud backend only):

// Cloud backend only. Mark up the text with the speaker aliases.
const dialogue = new GeminiTTSService({
  credentialsPath: process.env.GOOGLE_APPLICATION_CREDENTIALS ?? "",
  settings: new GeminiTTSService.Settings({
    multiSpeaker: true,
    speakerConfigs: [
      { speaker_alias: "Host", speaker_id: "Kore" },
      { speaker_alias: "Guest", speaker_id: "Charon" },
    ],
  }),
});

NotesCopy Anchor Link

• Backend selection: useGenai wins when set. Otherwise passing apiKey selects the Gemini API backend and leaving it out selects Cloud Text-to-Speech. With the Gemini API backend and no apiKey, the key falls back to GOOGLE_API_KEY. The env var alone never switches the backend.
• Gemini API backend: no multi-speaker output (a warning is logged and one speaker is used). prompt works only with Gemini 3.8 TTS models, sent as a speechMetadata.style annotation; other models ignore it with a warning.
• Model names differ by backend. Cloud Text-to-Speech takes gemini-3.1-flash-tts-preview, gemini-2.5-flash-tts, gemini-2.5-flash-lite-preview-tts and gemini-2.5-pro-tts. The Gemini API takes gemini-3.8-flash-tts, gemini-3.8-flash-lite-tts, gemini-3.1-flash-tts-preview, gemini-2.5-flash-preview-tts and gemini-2.5-pro-preview-tts.
• Gemini 3.8 TTS models treat the text as a verbatim transcript. Put momentary vocal events inline as angle-bracket tags (<laugh>, <sigh>, <short pause>) and sustained style in prompt.
• Gemini TTS outputs 24 kHz audio (GeminiTTSService.GOOGLE_SAMPLE_RATE). Any other sample rate logs a warning.
• A voice that is neither in AVAILABLE_VOICES nor a voice_... ID logs a warning and is used anyway.

Retrying blocked sentencesCopy Anchor Link

Gemini TTS sometimes refuses a harmless sentence as a possible violation of Google's usage guidelines, at times after part of its audio has played, and the same request usually goes through on the next try. By default (as upstream) the sentence is lost and an ErrorFrame goes upstream. With retryOnBlocked (pipedog only), the service asks for the sentence again:

const tts = new GeminiTTSService({
  credentialsPath: process.env.GOOGLE_APPLICATION_CREDENTIALS,
  settings: new GeminiTTSSettings({ voice: "Kore", prompt: "Warm and calm." }),
  retryOnBlocked: {
    retries: 1, // the default; `retryOnBlocked: true` is this alone
    // Only when part of the sentence already played: it starts over with this text.
    restartText: (text) => `Sorry, ${text}`,
    // Still blocked after the retries: one last attempt with these settings.
    fallback: new GeminiTTSSettings({ prompt: null }),
  },
});

• Only the usage-guidelines block is retried (isGeminiTTSBlockedError()): Cloud Text-to-Speech's INVALID_ARGUMENT that says so, the Gemini API's 400 that says so, or a Gemini API response blocked with no audio. Other errors fail the sentence as before.
• Audio that already played can't be taken back, so a sentence blocked midway starts over from the beginning, as restartText(text) if given, instead of continuing from the middle.
• fallback is either a GeminiTTSSettings delta used for that last attempt only (without the style prompt, or on the Cloud backend a Chirp 3 HD voice: new GeminiTTSSettings({ model: null, prompt: null, voice: "en-US-Chirp3-HD-Kore" })), or a function (text, contextId) => AsyncIterable<Frame>, such as another TTS service's runTts at the same sample rate.
• If every attempt is blocked, the sentence fails with one ErrorFrame, as without the option. Restarted and fallback audio is never stored in the audio cache.

Sharing a client and streaming optionsCopy Anchor Link

These options are pipedog additions:

• sharedClient (the three Cloud TTS services, and the Cloud backend of GeminiTTSService): use one client per process for the same endpoint (location), credentials and clientOptions, instead of one per service. Building a client costs a credential lookup, a TLS handshake and a token fetch on the first sentence; a server running one pipeline per call pays that once. Call prewarm() at start-up to pay it before the first call: it loads the SDK, opens the gRPC channel and fetches a token, and returns false (with a warning) instead of throwing if that fails, in which case the first sentence connects as usual.

  Copy Text

  const ttsOptions = { location: "eu", sharedClient: true };
  // At server start-up: a throwaway service warms the shared client.
  await new GeminiTTSService(ttsOptions).prewarm();
  
  // Per call: every service uses that client.
  const tts = new GeminiTTSService({ ...ttsOptions, settings: new GeminiTTSSettings({ voice: "Kore" }) });

  It is off by default (each service builds its own client, as upstream) because a shared client puts every concurrent stream of the process on one gRPC channel. Services never close a client, so one call ending doesn't affect the others. getSharedGoogleCloudTTSClient(options) returns the same shared client. The Gemini API backend talks HTTP with an API key, so it has nothing to share or prewarm.

• client (GoogleTTSService and the Cloud backend of GeminiTTSService): a GoogleCloudTTSClient to use instead of building one, for full control over sharing:

  Copy Text

  const client = createGoogleCloudTTSClient({ apiEndpoint: "eu-texttospeech.googleapis.com" });
  const tts = new GeminiTTSService({ client, settings: new GeminiTTSSettings({ voice: "Kore" }) });

  With client, the credentials, location, clientOptions and sharedClient are not used.

• bufferOnlyStartup (GoogleTTSService, GeminiTTSService): upstream re-cuts the audio into chunkSize pieces (0.5 s) for the whole utterance, which holds back up to one piece throughout. With bufferOnlyStartup: true, only the start is buffered; after the first chunkSize bytes, audio goes on as it arrives (whole samples).

Both streaming services treat a gRPC CANCELLED error, which is what an interrupted stream reports, as a quiet end of synthesis instead of an error.

Pre-synthesized linesCopy Anchor Link

All three services (and both backends of GeminiTTSService) work with the TTS audio cache: give them an audioCache and call warmAudioCache() for fixed lines such as a greeting or a filler, and those sentences play from memory with no request to Google.

const audioCache = new TTSAudioCache(); // one per process
const tts = new GeminiTTSService({ location: "eu", audioCache });

// While an outbound call rings, before the pipeline starts:
await tts.warmAudioCache("Hi! This is the clinic calling about your appointment.", {
  sampleRate: 24000, // the rate the service will run at
});

The cache key covers the text, the settings (model, voice, language, prompt, speakingRate and the rest), the sample rate, and what changes the audio outside the settings: the backend of GeminiTTSService and the voiceCloningKey of GoogleTTSService (hashed). A settings change therefore misses the cache instead of playing the old voice. Warming works before the pipeline starts, since the client is created with the service; pass sampleRate unless the service has one.

Event handlersCopy Anchor Link

All three services inherit the TTSService events, for example on_tts_request (contextId, text), called for each text sent for synthesis.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/google/package.json: ✅ means the compat smoke test passes on that runtime.

Cloud Text-to-Speech streaming is gRPC only, so GoogleTTSService, GoogleHttpTTSService and the Cloud backend of GeminiTTSService use the Node SDK @google-cloud/text-to-speech. It is imported on the first request, so importing the module is safe everywhere, but these classes need Node or Bun. The SDK sits behind the GoogleCloudTTSClient interface; pass client (or override _createClient()) to supply another client. clientOptions (pipedog only) is merged into the SDK's TextToSpeechClient options.

The Gemini API backend of GeminiTTSService uses @google/genai over fetch and runs on Bun, Node, Deno and Cloudflare Workers.

The deprecated upstream InputParams classes, the params, voiceId and (Gemini) model constructor options, and the GoogleStreamTTSSettings alias are not ported.