# Speechify

> Text-to-speech with Speechify: SpeechifyHttpTTSService streams audio and word timestamps over Server-Sent Events.

Source: https://polvallverdu.github.io/pipedog/docs/services/tts/speechify/



## Overview [#overview]

`SpeechifyHttpTTSService` posts each sentence to Speechify's `/v1/audio/stream/with-timestamps` endpoint and reads the Server-Sent Events it answers with: PCM audio chunks and word-level speech marks. The marks become word timestamps, so the bot's `TTSTextFrame`s follow the audio.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/tts/speechify">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/speechify/src">
    `packages/speechify/src/tts.ts`.
  </Card>

  <Card title="Speechify docs" href="https://docs.speechify.ai/build/api-reference/v1/audio/stream/with-timestamps.md">
    The stream-with-timestamps API reference.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/speechify
```

```ts
import { SpeechifyHttpTTSService } from "@pipedog/speechify/tts";
```

## Prerequisites [#prerequisites]

* A Speechify API account and an API key.
* Pass the key as `apiKey`. The examples read it from `SPEECHIFY_API_KEY`.

## Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `apiKey` | `string` | yes | Speechify API key for authentication. |
| `baseUrl` | `string \| undefined` | no | Base URL for the Speechify API. Defaults to "https://api.speechify.ai". |
| `fetch` | `{ (input: RequestInfo \| URL, init?: RequestInit): Promise<Response>; (input: string \| URL \| Request, init?: RequestInit): Promise<Response>; } \| null \| undefined` | no | `fetch` used for HTTP requests. Defaults to the global `fetch`. DIVERGENCE: upstream takes a required shared `aiohttp_session`. |
| `settings` | `SpeechifyTTSSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `textAggregationMode` | `TextAggregationMode \| null \| undefined` | no | How to aggregate incoming text before synthesis. `TextAggregationMode.SENTENCE` (default) buffers until sentence boundaries, `TextAggregationMode.TOKEN` streams tokens directly for lower latency. |
| `pushTextFrames` | `boolean \| undefined` | no | Whether to push TextFrames and LLMFullResponseEndFrames. |
| `pushStopFrames` | `boolean \| undefined` | no | Whether to automatically push TTSStoppedFrames. |
| `pushStartFrame` | `boolean \| undefined` | no | Whether to automatically create audio contexts and push TTSStartedFrames. When true, the base class handles `createAudioContext` and yields `TTSStartedFrame` before each synthesis call, so `runTts` implementations do not need to. |
| `stopFrameTimeoutS` | `number \| undefined` | no | Idle time before pushing TTSStoppedFrame when pushStopFrames is true. |
| `pushSilenceAfterStop` | `boolean \| undefined` | no | Whether to push silence audio after TTSStoppedFrame. |
| `silenceTimeS` | `number \| undefined` | no | Duration of silence to push when pushSilenceAfterStop is true. |
| `pauseFrameProcessing` | `boolean \| undefined` | no | Whether to hold incoming frames after a turn until its audio contexts end. Needed by services whose audio doesn't carry its request's context ID. |
| `maxConsecutiveZeroAudioContexts` | `number \| undefined` | no | How many consecutive TTS contexts may complete without producing any audio before the service is reported unable to do its job. Catches a provider that accepts requests and stays silent — an unknown voice ID, say — which no error ever surfaces. Every silent context reports an error the service can carry on from, whatever the limit is; on reaching the limit the service reports a permanent error instead, stops being given work, and the pipeline worker applies its `ProcessorUnusablePolicy`. Set to 0 to report silent contexts without ever writing the service off. |
| `appendTrailingSpace` | `boolean \| undefined` | no | Whether to append a trailing space to text before sending to TTS. This helps prevent some TTS services from vocalizing trailing punctuation (e.g., "dot"). Only applied in sentence aggregation mode; when streaming tokens, the incoming text's own whitespace is preserved. |
| `sampleRate` | `number \| null \| undefined` | no | Output sample rate for generated audio. |
| `skipTextTypes` | `string[] \| null \| undefined` | no | List of text types that should not be spoken. |
| `textTransforms` | `[string, TextTransform][] \| null \| undefined` | no | A list of TextTransforms to transform text just before sending it to TTS. Each TextTransform takes the aggregated text and its type, and returns the transformed text. To register, provide a list of tuples of `[textType \| "*", transformFunction]`. |
| `transportDestination` | `string \| null \| undefined` | no | Destination for generated audio frames. |
| `reuseContextIdWithinTurn` | `boolean \| undefined` | no | Whether the service should reuse context IDs within the same turn. |
| `audioCache` | `TTSAudioCacheStore \| null \| undefined` | no | Where to keep synthesized audio for reuse (e.g. a shared `TTSAudioCache`). Text whose audio is in the cache is served from it, with no request to the provider. Lines get there through `warmAudioCache()`, and also from live synthesis when `cacheSynthesizedAudio` allows it. Only services that yield their audio from `runTts` (HTTP and streaming-RPC services, see `supportsAudioCache`) use it, and only in sentence aggregation mode. Defaults to null (no cache). pipedog only. |
| `cacheSynthesizedAudio` | `boolean \| ((text: string) => boolean) \| undefined` | no | Whether audio synthesized live is stored in `audioCache`, or a function that decides per text. Defaults to false: only warmed lines are stored, since a generated reply can repeat what the user said. pipedog only. |
| `audioCacheNamespace` | `string \| undefined` | no | Extra part of the cache key, for constructor options that change the audio but are not in `settings`. pipedog only. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new SpeechifyHttpTTSService.Settings({...})`
(`SpeechifyTTSSettings`). Any setting you leave out keeps the default below.

| Setting                 | Type               | Default       | Description                                                                                                                                                                     |
| ----------------------- | ------------------ | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model`                 | `string`           | `"simba-3.5"` | Speech model, such as `simba-3.5` or `simba-3.0` (multilingual).                                                                                                                |
| `voice`                 | `string`           | `"geffen_32"` | Voice id.                                                                                                                                                                       |
| `language`              | `Language \| null` | `null`        | Language, sent as a tag such as `en-US`.                                                                                                                                        |
| `loudnessNormalization` | `boolean \| null`  | `null`        | Normalize the audio's loudness. Adds latency.                                                                                                                                   |
| `textNormalization`     | `boolean \| null`  | `null`        | Spell out numbers, dates and similar tokens. Adds latency.                                                                                                                      |
| `safetyIdentifier`      | `string \| null`   | `null`        | A stable, opaque id for the end user (1 to 64 characters), so Speechify can limit one user instead of the workspace. Never an email address.                                    |
| `pronunciations`        | `Dict[] \| null`   | `null`        | How specific words are spoken (up to 200), by IPA phones or a spoken alias. Applied on `simba-3.5` in the languages Speechify lists for pronunciations, which excludes English. |

## Usage [#usage]

```ts title="tts.ts"
import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { SpeechifyHttpTTSService } from "@pipedog/speechify/tts";

const tts = new SpeechifyHttpTTSService({
  apiKey: process.env.SPEECHIFY_API_KEY ?? "",
  settings: new SpeechifyHttpTTSService.Settings({
    model: "simba-3.5",
    voice: "geffen_32",
    textNormalization: true,
  }),
});
```

Change settings mid-call with a `TTSUpdateSettingsFrame`:

```ts
// Queue it on the worker, or push it from a processor upstream of the service.
const update = new TTSUpdateSettingsFrame({
  delta: new SpeechifyHttpTTSService.Settings({ voice: "another-voice-id" }),
});
```

## Notes [#notes]

* The sample rate must be one of Speechify's PCM rates (8000, 16000, 22050, 24000, 44100 or 48000). For any other pipeline rate the service asks for 24000 Hz, tags its audio frames with that rate, and the output transport resamples.
* Requests pin Speechify's API version (`Speechify-Version: 2026-09-30`), and send `Speechify-Caller: pipecat` with the pipedog version.
* Speechify marks the pieces of a hyphenated word separately. The service joins marks that touch in the text, so `text-to-speech` is one word timestamp.
* A mid-stream `speech.error` event is reported as an error frame. A non-200 response is too.

## Runtime notes [#runtime-notes]

* It uses the global `fetch` and reads the response as a stream, so it works on Bun, Node, Deno and Cloudflare Workers. Pass `fetch` to use another one (upstream takes an aiohttp session).
* Tested against a local server that replays Speechify's event format, as upstream's tests do for the parsing, not against Speechify itself.

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/speechify/package.json`: ✅ means the compat smoke test passes on that runtime.