# Piper

> Text-to-speech with Piper voices, run locally by PiperTTSService or through a Piper HTTP server by PiperHttpTTSService.

Source: https://polvallverdu.github.io/pipedog/docs/services/tts/piper/



## Overview [#overview]

* **`PiperTTSService`**: runs a [Piper](https://github.com/OHF-Voice/piper1-gpl) voice on your
  machine, on the CPU, with no API key. Its engine (`@pipedog/piper/piper_voice`) is a TypeScript
  reimplementation of what upstream uses from the Python `piper-tts` package (1.8.0), on
  `@pipedog/onnx` and `@pipedog/espeak-ng`.
* **`PiperHttpTTSService`**: posts each sentence to a
  [Piper HTTP server](https://github.com/OHF-Voice/piper1-gpl/blob/main/docs/API_HTTP.md) over
  `fetch`. It needs no model, no ONNX runtime and no eSpeak NG, and runs on Cloudflare Workers.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/tts/piper">
    Pipecat's documentation for the same services.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/piper/src">
    `packages/piper/src/tts.ts` and the engine, `piper_voice.ts`.
  </Card>

  <Card title="Piper voices" href="https://huggingface.co/rhasspy/piper-voices">
    Every voice, by language and quality, with samples.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/piper
# for PiperTTSService (local voices) only:
bun add onnxruntime-node @echogarden/espeak-ng-emscripten
```

* `onnxruntime-node` runs the voice model. `onnxruntime-web@1.30.0` (WebAssembly) works too, slower.
* `@echogarden/espeak-ng-emscripten` is eSpeak NG compiled to WebAssembly, for phonemization. It
  is GPL-3.0 (see [Licenses](#licenses)).

`PiperHttpTTSService` needs neither: the local engine is imported lazily, only by
`PiperTTSService`.

```ts
import { PiperHttpTTSService, PiperTTSService } from "@pipedog/piper/tts";
```

## What you need [#what-you-need]

**`PiperTTSService`**: a Piper voice, `<voice>.onnx` and `<voice>.onnx.json` from Hugging Face
[`rhasspy/piper-voices`](https://huggingface.co/rhasspy/piper-voices). It is downloaded once, when
the pipeline starts, into `downloadDir` if it is missing, as `piper.download_voices` does. Sizes
depend on the quality:

| Quality  | Sample rate | Size      |
| -------- | ----------- | --------- |
| `x_low`  | 16 kHz      | 20-28 MB  |
| `low`    | 16 kHz      | 28-69 MB  |
| `medium` | 22.05 kHz   | 62-81 MB  |
| `high`   | 22.05 kHz   | 63-136 MB |

Audio is resampled to the pipeline's output rate. `downloadDir` defaults to
`$PIPEDOG_CACHE_DIR/piper`, else `$XDG_CACHE_HOME/pipedog/piper`, else `~/.cache/pipedog/piper`.
Put the two files there yourself to skip the download:

```ts
// Voices already on disk: put <voice>.onnx and <voice>.onnx.json in downloadDir.
const fromDisk = new PiperTTSService({
  downloadDir: "/models/piper",
  settings: new PiperTTSService.Settings({ voice: "en_US-lessac-medium" }),
});
```

eSpeak NG adds about 25 MB (the WebAssembly build and its voice data), loaded into memory once per
process. `@pipedog/espeak-ng` wraps it and reproduces piper-tts' clause-by-clause phonemization, so
the phonemes match piper-tts 1.8.0's for the tested languages, except where piper-tts bundles newer
espeak-ng data (`en-gb-x-rp`, `pt-br` and `it` differ slightly).

**`PiperHttpTTSService`**: a running Piper HTTP server, for example:

```bash
uv pip install "piper-tts[http]"
uv run python -m piper.http_server -m en_US-ryan-high
```

## Runtimes [#runtimes]

| Runtime            | Works | What it needs                                                                                                                                                                                                                                                                                            |
| ------------------ | ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Bun                | ✅ yes | PiperTTSService: onnxruntime-node (or onnxruntime-web) and @echogarden/espeak-ng-emscripten, plus a Piper voice (20-136 MB) downloaded on first use. PiperHttpTTSService: only fetch and a Piper HTTP server.                                                                                            |
| Node.js            | ✅ yes | PiperTTSService: onnxruntime-node (or onnxruntime-web) and @echogarden/espeak-ng-emscripten, plus a Piper voice (20-136 MB) downloaded on first use. PiperHttpTTSService: only fetch and a Piper HTTP server.                                                                                            |
| Deno               | ✅ yes | PiperTTSService: onnxruntime-node (or onnxruntime-web) and @echogarden/espeak-ng-emscripten, plus a Piper voice (20-136 MB) downloaded on first use. PiperHttpTTSService: only fetch and a Piper HTTP server.                                                                                            |
| Cloudflare Workers | ✅ yes | PiperTTSService: onnxruntime-node (or onnxruntime-web) and @echogarden/espeak-ng-emscripten, plus a Piper voice (20-136 MB) downloaded on first use. PiperHttpTTSService: only fetch and a Piper HTTP server. PiperHttpTTSService only (PiperTTSService needs a filesystem and exceeds Workers' limits). |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/piper/package.json`: ✅ means the compat smoke test passes on that runtime.

`PiperHttpTTSService` runs on Cloudflare Workers. `PiperTTSService` does not: it needs a filesystem
for the voice cache, and the voice model (20-136 MB) and eSpeak NG's data don't fit a Worker's
128 MB of memory or its 10 MB bundle limit.

## PiperTTSService [#piperttsservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `downloadDir` | `string \| null \| undefined` | no | Directory for storing voice model files. Defaults to pipedog's cache directory (`piperCacheDir()`). |
| `forceRedownload` | `boolean \| undefined` | no | Re-download the voice model even if it already exists. |
| `useCuda` | `boolean \| undefined` | no | Use CUDA for GPU-accelerated inference. DIVERGENCE: accepted but has no effect; inference runs on the ONNX backend's default (CPU) execution provider. |
| `fetch` | `((input: RequestInfo \| URL, init?: RequestInit) => Promise<Response>) \| null \| undefined` | no | `fetch` for downloads (default: the global `fetch`). Not upstream. |
| `backend` | `OnnxBackend \| Promise<OnnxBackend> \| null \| undefined` | no | The ONNX backend (default: `createOnnxBackend()`). Not upstream. |
| `settings` | `PiperTTSSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `textAggregationMode` | `TextAggregationMode \| null \| undefined` | no | How to aggregate incoming text before synthesis. `TextAggregationMode.SENTENCE` (default) buffers until sentence boundaries, `TextAggregationMode.TOKEN` streams tokens directly for lower latency. |
| `pushTextFrames` | `boolean \| undefined` | no | Whether to push TextFrames and LLMFullResponseEndFrames. |
| `pushStopFrames` | `boolean \| undefined` | no | Whether to automatically push TTSStoppedFrames. |
| `pushStartFrame` | `boolean \| undefined` | no | Whether to automatically create audio contexts and push TTSStartedFrames. When true, the base class handles `createAudioContext` and yields `TTSStartedFrame` before each synthesis call, so `runTts` implementations do not need to. |
| `stopFrameTimeoutS` | `number \| undefined` | no | Idle time before pushing TTSStoppedFrame when pushStopFrames is true. |
| `pushSilenceAfterStop` | `boolean \| undefined` | no | Whether to push silence audio after TTSStoppedFrame. |
| `silenceTimeS` | `number \| undefined` | no | Duration of silence to push when pushSilenceAfterStop is true. |
| `pauseFrameProcessing` | `boolean \| undefined` | no | Whether to hold incoming frames after a turn until its audio contexts end. Needed by services whose audio doesn't carry its request's context ID. |
| `maxConsecutiveZeroAudioContexts` | `number \| undefined` | no | How many consecutive TTS contexts may complete without producing any audio before the service is reported unable to do its job. Catches a provider that accepts requests and stays silent — an unknown voice ID, say — which no error ever surfaces. Every silent context reports an error the service can carry on from, whatever the limit is; on reaching the limit the service reports a permanent error instead, stops being given work, and the pipeline worker applies its `ProcessorUnusablePolicy`. Set to 0 to report silent contexts without ever writing the service off. |
| `appendTrailingSpace` | `boolean \| undefined` | no | Whether to append a trailing space to text before sending to TTS. This helps prevent some TTS services from vocalizing trailing punctuation (e.g., "dot"). Only applied in sentence aggregation mode; when streaming tokens, the incoming text's own whitespace is preserved. |
| `sampleRate` | `number \| null \| undefined` | no | Output sample rate for generated audio. |
| `skipTextTypes` | `string[] \| null \| undefined` | no | List of text types that should not be spoken. |
| `textTransforms` | `[string, TextTransform][] \| null \| undefined` | no | A list of TextTransforms to transform text just before sending it to TTS. Each TextTransform takes the aggregated text and its type, and returns the transformed text. To register, provide a list of tuples of `[textType \| "*", transformFunction]`. |
| `transportDestination` | `string \| null \| undefined` | no | Destination for generated audio frames. |
| `reuseContextIdWithinTurn` | `boolean \| undefined` | no | Whether the service should reuse context IDs within the same turn. |
| `audioCache` | `TTSAudioCacheStore \| null \| undefined` | no | Where to keep synthesized audio for reuse (e.g. a shared `TTSAudioCache`). Text whose audio is in the cache is served from it, with no request to the provider. Lines get there through `warmAudioCache()`, and also from live synthesis when `cacheSynthesizedAudio` allows it. Only services that yield their audio from `runTts` (HTTP and streaming-RPC services, see `supportsAudioCache`) use it, and only in sentence aggregation mode. Defaults to null (no cache). pipedog only. |
| `cacheSynthesizedAudio` | `boolean \| ((text: string) => boolean) \| undefined` | no | Whether audio synthesized live is stored in `audioCache`, or a function that decides per text. Defaults to false: only warmed lines are stored, since a generated reply can repeat what the user said. pipedog only. |
| `audioCacheNamespace` | `string \| undefined` | no | Extra part of the cache key, for constructor options that change the audio but are not in `settings`. pipedog only. |

### Settings [#settings]

Settings, passed as `settings: new PiperTTSService.Settings({...})` (`PiperTTSSettings`).

| Setting    | Type     | Default        | Description                                        |
| ---------- | -------- | -------------- | -------------------------------------------------- |
| `model`    | `null`   | `null`         | Not used.                                          |
| `voice`    | `string` | none, required | Piper voice name, for example `"en_US-ryan-high"`. |
| `language` | `null`   | `null`         | Not used: the voice sets the language.             |

The voice is loaded once. Settings changes are stored but not applied, with a warning, as upstream.

### Usage [#usage]

```ts title="tts.ts"
import { PiperHttpTTSService, PiperTTSService } from "@pipedog/piper/tts";

// Downloads en_US-ryan-high.onnx and .onnx.json from rhasspy/piper-voices on first start.
const tts = new PiperTTSService({
  settings: new PiperTTSService.Settings({ voice: "en_US-ryan-high" }),
});
```

The engine works on its own too:

```ts
import { downloadVoice, PiperVoice } from "@pipedog/piper/piper_voice";

await downloadVoice("en_US-lessac-medium", "/models");
const voice = await PiperVoice.load("/models/en_US-lessac-medium.onnx");
for await (const chunk of voice.synthesize("Hello world. How are you?")) {
  chunk.audioInt16Bytes; // one sentence, 16-bit PCM at voice.config.sampleRate
}
```

### Notes [#notes]

* The voice is downloaded and loaded in `start()`. A failure is pushed as an error and retried on
  the first sentence.
* Two services with the same voice in one process share one model session.
* Each sentence is phonemized clause by clause, synthesized, and pushed as 16-bit PCM.
* `forceRedownload: true` downloads the voice again even if it is on disk.
* `useCuda` is accepted but has no effect: inference runs on the ONNX backend's default (CPU)
  execution provider.
* Not supported: voices whose phoneme type isn't `espeak` or `text` (Chinese, Hebrew, Japanese and
  Thai voices need other G2P models), and Arabic diacritization.

## PiperHttpTTSService [#piperhttpttsservice]

Posts `{ text, voice }` to `baseUrl` for each sentence and streams the WAV reply, without its
header.

### Configuration [#configuration-1]

| Option | Type | Required | Description |
|---|---|---|---|
| `baseUrl` | `string` | yes | Base URL for the Piper TTS HTTP server. |
| `fetch` | `((input: RequestInfo \| URL, init?: RequestInit) => Promise<Response>) \| null \| undefined` | no | `fetch` used for HTTP requests. Defaults to the global `fetch`. DIVERGENCE: upstream takes a required `aiohttp_session`. |
| `settings` | `PiperHttpTTSSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `textAggregationMode` | `TextAggregationMode \| null \| undefined` | no | How to aggregate incoming text before synthesis. `TextAggregationMode.SENTENCE` (default) buffers until sentence boundaries, `TextAggregationMode.TOKEN` streams tokens directly for lower latency. |
| `pushTextFrames` | `boolean \| undefined` | no | Whether to push TextFrames and LLMFullResponseEndFrames. |
| `pushStopFrames` | `boolean \| undefined` | no | Whether to automatically push TTSStoppedFrames. |
| `pushStartFrame` | `boolean \| undefined` | no | Whether to automatically create audio contexts and push TTSStartedFrames. When true, the base class handles `createAudioContext` and yields `TTSStartedFrame` before each synthesis call, so `runTts` implementations do not need to. |
| `stopFrameTimeoutS` | `number \| undefined` | no | Idle time before pushing TTSStoppedFrame when pushStopFrames is true. |
| `pushSilenceAfterStop` | `boolean \| undefined` | no | Whether to push silence audio after TTSStoppedFrame. |
| `silenceTimeS` | `number \| undefined` | no | Duration of silence to push when pushSilenceAfterStop is true. |
| `pauseFrameProcessing` | `boolean \| undefined` | no | Whether to hold incoming frames after a turn until its audio contexts end. Needed by services whose audio doesn't carry its request's context ID. |
| `maxConsecutiveZeroAudioContexts` | `number \| undefined` | no | How many consecutive TTS contexts may complete without producing any audio before the service is reported unable to do its job. Catches a provider that accepts requests and stays silent — an unknown voice ID, say — which no error ever surfaces. Every silent context reports an error the service can carry on from, whatever the limit is; on reaching the limit the service reports a permanent error instead, stops being given work, and the pipeline worker applies its `ProcessorUnusablePolicy`. Set to 0 to report silent contexts without ever writing the service off. |
| `appendTrailingSpace` | `boolean \| undefined` | no | Whether to append a trailing space to text before sending to TTS. This helps prevent some TTS services from vocalizing trailing punctuation (e.g., "dot"). Only applied in sentence aggregation mode; when streaming tokens, the incoming text's own whitespace is preserved. |
| `sampleRate` | `number \| null \| undefined` | no | Output sample rate for generated audio. |
| `skipTextTypes` | `string[] \| null \| undefined` | no | List of text types that should not be spoken. |
| `textTransforms` | `[string, TextTransform][] \| null \| undefined` | no | A list of TextTransforms to transform text just before sending it to TTS. Each TextTransform takes the aggregated text and its type, and returns the transformed text. To register, provide a list of tuples of `[textType \| "*", transformFunction]`. |
| `transportDestination` | `string \| null \| undefined` | no | Destination for generated audio frames. |
| `reuseContextIdWithinTurn` | `boolean \| undefined` | no | Whether the service should reuse context IDs within the same turn. |
| `audioCache` | `TTSAudioCacheStore \| null \| undefined` | no | Where to keep synthesized audio for reuse (e.g. a shared `TTSAudioCache`). Text whose audio is in the cache is served from it, with no request to the provider. Lines get there through `warmAudioCache()`, and also from live synthesis when `cacheSynthesizedAudio` allows it. Only services that yield their audio from `runTts` (HTTP and streaming-RPC services, see `supportsAudioCache`) use it, and only in sentence aggregation mode. Defaults to null (no cache). pipedog only. |
| `cacheSynthesizedAudio` | `boolean \| ((text: string) => boolean) \| undefined` | no | Whether audio synthesized live is stored in `audioCache`, or a function that decides per text. Defaults to false: only warmed lines are stored, since a generated reply can repeat what the user said. pipedog only. |
| `audioCacheNamespace` | `string \| undefined` | no | Extra part of the cache key, for constructor options that change the audio but are not in `settings`. pipedog only. |

### Settings [#settings-1]

Settings, passed as `settings: new PiperHttpTTSService.Settings({...})` (`PiperHttpTTSSettings`).

| Setting    | Type             | Default | Description                                                      |
| ---------- | ---------------- | ------- | ---------------------------------------------------------------- |
| `model`    | `null`           | `null`  | Not used.                                                        |
| `voice`    | `string \| null` | `null`  | Voice the server should use. `null` leaves the server's default. |
| `language` | `null`           | `null`  | Not used.                                                        |

### Usage [#usage-1]

```ts
// A Piper HTTP server: python -m piper.http_server -m en_US-ryan-high
const remote = new PiperHttpTTSService({
  baseUrl: "http://localhost:5000",
  settings: new PiperHttpTTSService.Settings({ voice: "en_US-ryan-high" }),
});

```

### Notes [#notes-1]

* A trailing slash on `baseUrl` is removed, with a warning.
* A non-200 reply is pushed as an `ErrorFrame`.
* `fetch` replaces the global `fetch`.

## Differences from upstream [#differences-from-upstream]

* `PiperTTSService` downloads and loads the voice in `start()`, not in the constructor, since
  loading is asynchronous. A missing or blank voice is still rejected by the constructor.
* Default download directory: `~/.cache/pipedog/piper`, where upstream uses
  `~/.cache/pipecat/piper`.
* `useCuda` has no effect (above). `fetch` and `backend` are pipedog options.
* `PiperHttpTTSService`: upstream's required `aiohttp_session` is an optional `fetch`.
* The deprecated `voice_id` argument is not ported. Use `settings`.

## Licenses [#licenses]

* `@pipedog/piper`: BSD 2-Clause.
* Piper voices: each has its own license, in its `MODEL_CARD` on Hugging Face.
* eSpeak NG and `@echogarden/espeak-ng-emscripten`, used by `PiperTTSService`: **GPL-3.0**.
  Upstream's `PiperTTSService` uses `piper-tts`, which is GPL-3.0 itself. To keep GPL code out of
  your application, use `PiperHttpTTSService` with a separately installed Piper server.