# Cartesia

> Text-to-speech with CartesiaTTSService (WebSocket, word timestamps) and CartesiaHttpTTSService (HTTP), configured with GenerationConfig.

Source: https://polvallverdu.github.io/pipedog/docs/services/tts/cartesia/



## Overview [#overview]

Cartesia's Sonic models turn text into speech. pipedog ports both of Pipecat's Cartesia
text-to-speech services:

* **`CartesiaTTSService`**: streams over Cartesia's WebSocket API. Audio arrives as it is
  generated, with word-level timestamps that keep the spoken text in step with playback.
* **`CartesiaHttpTTSService`**: posts each request to Cartesia's `/tts/bytes` HTTP endpoint and
  gets the audio back in one piece, without word timestamps.
* **`GenerationConfig`**: volume, speed and emotion for sonic-3 series models, shared by both.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/tts/cartesia">
    Pipecat's documentation for the same services.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/cartesia/src">
    `packages/cartesia/src/tts.ts`.
  </Card>

  <Card title="Cartesia docs" href="https://docs.cartesia.ai/api-reference/tts/tts">
    Voices, models, SSML tags and pronunciation dictionaries.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/cartesia
```

```ts
import {
  CartesiaEmotion,
  CartesiaHttpTTSService,
  CartesiaTTSService,
  GenerationConfig,
} from "@pipedog/cartesia/tts";
```

## Prerequisites [#prerequisites]

* A Cartesia account and an API key from the [Cartesia playground](https://play.cartesia.ai).
* A voice id from the Cartesia voice library. There is no default voice.
* Pass the key as `apiKey`. The examples read it from `CARTESIA_API_KEY`.

## CartesiaTTSService [#cartesiattsservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `apiKey` | `string` | yes | Cartesia API key for authentication. |
| `url` | `string \| undefined` | no | WebSocket URL for Cartesia TTS API. |
| `encoding` | `string \| undefined` | no | Audio encoding format. |
| `container` | `string \| undefined` | no | Audio container format. |
| `maxBufferDelayMs` | `number \| null \| undefined` | no | Server-side buffering window before generation starts. `0` disables server buffering (custom buffering); any value in (0, 5000] enables managed buffering. If null, derived from `textAggregationMode`: `0` for `SENTENCE` (avoids stacking client and server buffering), unset for `TOKEN` (uses Cartesia's 3000ms default). |
| `extraHeaders` | `Record<string, string> \| null \| undefined` | no | Optional additional HTTP headers to send with the WebSocket handshake. |
| `settings` | `CartesiaTTSSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `textAggregationMode` | `TextAggregationMode \| null \| undefined` | no | How to aggregate incoming text before synthesis. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `reconnectBackoffMinWait` | `number \| undefined` | no | Minimum time, in seconds, to wait between reconnection attempts. |
| `reconnectBackoffMaxWait` | `number \| undefined` | no | Maximum time, in seconds, to wait between reconnection attempts. |
| `reconnectOnError` | `boolean \| undefined` | no | Whether to automatically reconnect on connection errors. |
| `wsCloseTimeout` | `number \| undefined` | no | Maximum time, in seconds, to wait for the peer to acknowledge the websocket closing handshake before dropping the connection. Applied to connections opened through `_websocketConnect()`. Increase it for peers that need longer to complete a graceful close. |
| `wsSendTimeout` | `number \| null \| undefined` | no | Maximum time, in seconds, a single send may wait on a peer that has stopped reading before the connection is dropped and the send throws `ConnectionClosedError`. Applied to connections opened through `_websocketConnect()`. Pass `null` to wait indefinitely. Defaults to `WS_SEND_TIMEOUT`. |
| `pushTextFrames` | `boolean \| undefined` | no | Whether to push TextFrames and LLMFullResponseEndFrames. |
| `pushStopFrames` | `boolean \| undefined` | no | Whether to automatically push TTSStoppedFrames. |
| `pushStartFrame` | `boolean \| undefined` | no | Whether to automatically create audio contexts and push TTSStartedFrames. When true, the base class handles `createAudioContext` and yields `TTSStartedFrame` before each synthesis call, so `runTts` implementations do not need to. |
| `stopFrameTimeoutS` | `number \| undefined` | no | Idle time before pushing TTSStoppedFrame when pushStopFrames is true. |
| `pushSilenceAfterStop` | `boolean \| undefined` | no | Whether to push silence audio after TTSStoppedFrame. |
| `silenceTimeS` | `number \| undefined` | no | Duration of silence to push when pushSilenceAfterStop is true. |
| `pauseFrameProcessing` | `boolean \| undefined` | no | Whether to hold incoming frames after a turn until its audio contexts end. Needed by services whose audio doesn't carry its request's context ID. |
| `maxConsecutiveZeroAudioContexts` | `number \| undefined` | no | How many consecutive TTS contexts may complete without producing any audio before the service is reported unable to do its job. Catches a provider that accepts requests and stays silent — an unknown voice ID, say — which no error ever surfaces. Every silent context reports an error the service can carry on from, whatever the limit is; on reaching the limit the service reports a permanent error instead, stops being given work, and the pipeline worker applies its `ProcessorUnusablePolicy`. Set to 0 to report silent contexts without ever writing the service off. |
| `appendTrailingSpace` | `boolean \| undefined` | no | Whether to append a trailing space to text before sending to TTS. This helps prevent some TTS services from vocalizing trailing punctuation (e.g., "dot"). Only applied in sentence aggregation mode; when streaming tokens, the incoming text's own whitespace is preserved. |
| `sampleRate` | `number \| null \| undefined` | no | Output sample rate for generated audio. |
| `skipTextTypes` | `string[] \| null \| undefined` | no | List of text types that should not be spoken. |
| `textTransforms` | `[string, TextTransform][] \| null \| undefined` | no | A list of TextTransforms to transform text just before sending it to TTS. Each TextTransform takes the aggregated text and its type, and returns the transformed text. To register, provide a list of tuples of `[textType \| "*", transformFunction]`. |
| `transportDestination` | `string \| null \| undefined` | no | Destination for generated audio frames. |
| `reuseContextIdWithinTurn` | `boolean \| undefined` | no | Whether the service should reuse context IDs within the same turn. |
| `audioCache` | `TTSAudioCacheStore \| null \| undefined` | no | Where to keep synthesized audio for reuse (e.g. a shared `TTSAudioCache`). Text whose audio is in the cache is served from it, with no request to the provider. Lines get there through `warmAudioCache()`, and also from live synthesis when `cacheSynthesizedAudio` allows it. Only services that yield their audio from `runTts` (HTTP and streaming-RPC services, see `supportsAudioCache`) use it, and only in sentence aggregation mode. Defaults to null (no cache). pipedog only. |
| `cacheSynthesizedAudio` | `boolean \| ((text: string) => boolean) \| undefined` | no | Whether audio synthesized live is stored in `audioCache`, or a function that decides per text. Defaults to false: only warmed lines are stored, since a generated reply can repeat what the user said. pipedog only. |
| `audioCacheNamespace` | `string \| undefined` | no | Extra part of the cache key, for constructor options that change the audio but are not in `settings`. pipedog only. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new CartesiaTTSService.Settings({...})`. Both
services use the same settings class.

| Setting               | Type                       | Default       | Description                                                                                                                           |
| --------------------- | -------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `model`               | `string`                   | `"sonic-3.6"` | Cartesia TTS model.                                                                                                                   |
| `voice`               | `string \| null`           | `null`        | Voice id. Set one: there is no default.                                                                                               |
| `language`            | `Language`                 | `Language.EN` | Synthesis language.                                                                                                                   |
| `accent`              | `string \| null`           | `null`        | Accent a multilingual voice speaks the language with, from the `accents` of Cartesia's Get Voice response. `null` lets Cartesia pick. |
| `generationConfig`    | `GenerationConfig \| null` | `null`        | Volume, speed and emotion. See below.                                                                                                 |
| `pronunciationDictId` | `string \| null`           | `null`        | Cartesia pronunciation dictionary id.                                                                                                 |

### GenerationConfig [#generationconfig]

| Option | Type | Required | Description |
|---|---|---|---|
| `volume` | `number \| null \| undefined` | no | Volume multiplier for generated speech. Valid range: [0.5, 2.0]. Default is 1.0. |
| `speed` | `number \| null \| undefined` | no | Speed multiplier for generated speech. Valid range: [0.6, 1.5]. Default is 1.0. |
| `emotion` | `string \| null \| undefined` | no | Single emotion string to guide the emotional tone. Examples include neutral, angry, excited, content, sad, scared. Over 60 emotions are supported. For best results, use with recommended voices: Leo, Jace, Kyle, Gavin, Maya, Tessa, Dana, and Marian. |

Fields left `null` are not sent. `CartesiaEmotion` lists the emotion names Cartesia supports.

### Usage [#usage]

```ts title="tts.ts"
import {
  CartesiaEmotion,
  CartesiaHttpTTSService,
  CartesiaTTSService,
  GenerationConfig,
} from "@pipedog/cartesia/tts";
import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { TextAggregationMode } from "@pipedog/core/services/tts_service";
import { Language } from "@pipedog/core/transcriptions/language";

const tts = new CartesiaTTSService({
  apiKey: process.env.CARTESIA_API_KEY ?? "",
  settings: new CartesiaTTSService.Settings({
    voice: "your-voice-id",
    model: "sonic-3.6",
    language: Language.EN,
  }),
});
```

Token streaming and a generation config:

```ts
const expressive = new CartesiaTTSService({
  apiKey: process.env.CARTESIA_API_KEY ?? "",
  // Stream LLM tokens straight to Cartesia instead of waiting for whole sentences.
  textAggregationMode: TextAggregationMode.TOKEN,
  settings: new CartesiaTTSService.Settings({
    voice: "your-voice-id",
    generationConfig: new GenerationConfig({ speed: 1.1, emotion: CartesiaEmotion.EXCITED }),
  }),
});
```

SSML tag helpers and IPA pronunciations:

```ts
// Helpers that build Cartesia SSML tags, for use in text transforms.
const spelled = CartesiaTTSService.SPELL("A1B2");
const tagged = `${CartesiaTTSService.EMOTION_TAG(CartesiaEmotion.CALM)} One moment.${CartesiaTTSService.PAUSE_TAG(0.5)}`;

// Say words as IPA describes. Register it last in textTransforms.
const pronounce = CartesiaTTSService.pronunciationTransformIpa({ Metformin: "mɛtˈfɔɹmɪn" });
const withPronunciations = new CartesiaTTSService({
  apiKey: process.env.CARTESIA_API_KEY ?? "",
  settings: new CartesiaTTSService.Settings({ voice: "your-voice-id" }),
  textTransforms: [["*", pronounce]],
});
```

Change settings mid-call with a `TTSUpdateSettingsFrame`:

```ts
// Queue it on the worker, or push it from a processor upstream of the service.
const update = new TTSUpdateSettingsFrame({
  delta: new CartesiaTTSService.Settings({ voice: "another-voice-id", language: Language.ES }),
});
```

### Event handlers [#event-handlers]

Inherited from `TTSService`:

| Event                 | Description                                     |
| --------------------- | ----------------------------------------------- |
| `on_connected`        | Connected to Cartesia.                          |
| `on_disconnected`     | Disconnected from Cartesia.                     |
| `on_connection_error` | The connection failed.                          |
| `on_tts_request`      | A text chunk is about to be sent for synthesis. |

### Notes [#notes]

* The service does not push the LLM's text frames itself. It builds `TTSTextFrame`s from
  Cartesia's word timestamps, so the context only records words the user actually heard.
* Text is aggregated into sentences by default. `TextAggregationMode.TOKEN` streams tokens
  directly for lower latency.
* `maxBufferDelayMs` sets Cartesia's server-side buffer. Left `null`, it is `0` in sentence mode
  (no double buffering) and unset in token mode (Cartesia's 3000 ms default).
* A `voice`, `model` or `language` change takes effect from the next sentence, which opens a new
  Cartesia context.
* Static helpers build Cartesia SSML tags for text transforms: `SPELL`, `EMOTION_TAG`,
  `PAUSE_TAG`, `VOLUME_TAG` and `SPEED_TAG`. Text inside `<spell>` tags is never split across
  aggregates.
* `pronunciationTransformIpa()` turns IPA into Cartesia inline phonemes. Register it last in
  `textTransforms` so later transforms don't rewrite its markup.

## CartesiaHttpTTSService [#cartesiahttpttsservice]

### Configuration [#configuration-1]

| Option | Type | Required | Description |
|---|---|---|---|
| `apiKey` | `string` | yes | Cartesia API key for authentication. |
| `baseUrl` | `string \| undefined` | no | Base URL for Cartesia HTTP API. |
| `fetch` | `{ (input: RequestInfo \| URL, init?: RequestInit): Promise<Response>; (input: string \| URL \| Request, init?: RequestInit): Promise<Response>; } \| null \| undefined` | no | `fetch` used for HTTP requests. If not provided, the global `fetch` is used, set up in `setup()` and released at teardown. DIVERGENCE: replaces upstream's `aiohttp_session`. |
| `encoding` | `string \| undefined` | no | Audio encoding format. |
| `container` | `string \| undefined` | no | Audio container format. |
| `extraHeaders` | `Record<string, string> \| null \| undefined` | no | Optional additional HTTP headers to send with each synthesis request. |
| `settings` | `CartesiaTTSSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `textAggregationMode` | `TextAggregationMode \| null \| undefined` | no | How to aggregate incoming text before synthesis. `TextAggregationMode.SENTENCE` (default) buffers until sentence boundaries, `TextAggregationMode.TOKEN` streams tokens directly for lower latency. |
| `pushTextFrames` | `boolean \| undefined` | no | Whether to push TextFrames and LLMFullResponseEndFrames. |
| `pushStopFrames` | `boolean \| undefined` | no | Whether to automatically push TTSStoppedFrames. |
| `pushStartFrame` | `boolean \| undefined` | no | Whether to automatically create audio contexts and push TTSStartedFrames. When true, the base class handles `createAudioContext` and yields `TTSStartedFrame` before each synthesis call, so `runTts` implementations do not need to. |
| `stopFrameTimeoutS` | `number \| undefined` | no | Idle time before pushing TTSStoppedFrame when pushStopFrames is true. |
| `pushSilenceAfterStop` | `boolean \| undefined` | no | Whether to push silence audio after TTSStoppedFrame. |
| `silenceTimeS` | `number \| undefined` | no | Duration of silence to push when pushSilenceAfterStop is true. |
| `pauseFrameProcessing` | `boolean \| undefined` | no | Whether to hold incoming frames after a turn until its audio contexts end. Needed by services whose audio doesn't carry its request's context ID. |
| `maxConsecutiveZeroAudioContexts` | `number \| undefined` | no | How many consecutive TTS contexts may complete without producing any audio before the service is reported unable to do its job. Catches a provider that accepts requests and stays silent — an unknown voice ID, say — which no error ever surfaces. Every silent context reports an error the service can carry on from, whatever the limit is; on reaching the limit the service reports a permanent error instead, stops being given work, and the pipeline worker applies its `ProcessorUnusablePolicy`. Set to 0 to report silent contexts without ever writing the service off. |
| `appendTrailingSpace` | `boolean \| undefined` | no | Whether to append a trailing space to text before sending to TTS. This helps prevent some TTS services from vocalizing trailing punctuation (e.g., "dot"). Only applied in sentence aggregation mode; when streaming tokens, the incoming text's own whitespace is preserved. |
| `sampleRate` | `number \| null \| undefined` | no | Output sample rate for generated audio. |
| `skipTextTypes` | `string[] \| null \| undefined` | no | List of text types that should not be spoken. |
| `textTransforms` | `[string, TextTransform][] \| null \| undefined` | no | A list of TextTransforms to transform text just before sending it to TTS. Each TextTransform takes the aggregated text and its type, and returns the transformed text. To register, provide a list of tuples of `[textType \| "*", transformFunction]`. |
| `transportDestination` | `string \| null \| undefined` | no | Destination for generated audio frames. |
| `reuseContextIdWithinTurn` | `boolean \| undefined` | no | Whether the service should reuse context IDs within the same turn. |
| `audioCache` | `TTSAudioCacheStore \| null \| undefined` | no | Where to keep synthesized audio for reuse (e.g. a shared `TTSAudioCache`). Text whose audio is in the cache is served from it, with no request to the provider. Lines get there through `warmAudioCache()`, and also from live synthesis when `cacheSynthesizedAudio` allows it. Only services that yield their audio from `runTts` (HTTP and streaming-RPC services, see `supportsAudioCache`) use it, and only in sentence aggregation mode. Defaults to null (no cache). pipedog only. |
| `cacheSynthesizedAudio` | `boolean \| ((text: string) => boolean) \| undefined` | no | Whether audio synthesized live is stored in `audioCache`, or a function that decides per text. Defaults to false: only warmed lines are stored, since a generated reply can repeat what the user said. pipedog only. |
| `audioCacheNamespace` | `string \| undefined` | no | Extra part of the cache key, for constructor options that change the audio but are not in `settings`. pipedog only. |

### Settings [#settings-1]

Same as [`CartesiaTTSService`](#settings), with the same defaults.

### Usage [#usage-1]

```ts title="http.ts"
const httpTts = new CartesiaHttpTTSService({
  apiKey: process.env.CARTESIA_API_KEY ?? "",
  settings: new CartesiaHttpTTSService.Settings({ voice: "your-voice-id" }),
});
```

### Notes [#notes-1]

* Each request returns its audio in one piece. Use it when streaming does not matter.
* `baseUrl` defaults to `"https://api.cartesia.ai"`.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/cartesia/package.json`: ✅ means the compat smoke test passes on that runtime.

* `CartesiaTTSService` uses the global `WebSocket` and sends the key as an `X-API-Key` header on
  the handshake. The Bun, Node and Deno `WebSocket` support handshake headers, and on Cloudflare
  Workers the socket is opened through `fetch`.
* `CartesiaHttpTTSService` uses the global `fetch` unless you pass `fetch`. That option replaces
  upstream's `aiohttp_session`.
* The Cartesia API version header is pinned to `2026-03-01`. Upstream's `cartesia_version`
  option is not ported.
* Not ported: the deprecated `voiceId`, `model`, `params` (`InputParams`) and
  `aggregateSentences` constructor options. Use `settings` and `textAggregationMode`.