# Neuphonic > Text-to-speech with Neuphonic: NeuphonicTTSService over WebSocket and NeuphonicHttpTTSService over server-sent events. Source: https://polvallverdu.github.io/pipedog/docs/services/tts/neuphonic/ ## Overview [#overview] * **`NeuphonicTTSService`**: keeps one WebSocket open, sends each sentence followed by ``, and pushes the audio as it arrives. When the bot is interrupted while speaking, it reconnects (the service is an `InterruptibleTTSService`). * **`NeuphonicHttpTTSService`**: one HTTP request per sentence, read as server-sent events. Pipecat's documentation for the same service. `packages/neuphonic/src/tts.ts`. The Neuphonic documentation. ## Installation [#installation] ```bash bun add @pipedog/neuphonic ``` ```ts import { NeuphonicHttpTTSService, NeuphonicTTSService } from "@pipedog/neuphonic/tts"; ``` ## Prerequisites [#prerequisites] * A Neuphonic account and an API key. * Pass the key as `apiKey`. The examples read it from `NEUPHONIC_API_KEY`. ## NeuphonicTTSService [#neuphonicttsservice] Connects to `wss://api.neuphonic.com/speak/`. ### Configuration [#configuration] | Option | Type | Required | Description | |---|---|---|---| | `apiKey` | `string` | yes | Neuphonic API key for authentication. | | `url` | `string \| undefined` | no | WebSocket URL for the Neuphonic API. | | `encoding` | `string \| undefined` | no | Audio encoding format. Defaults to "pcm_linear". | | `settings` | `NeuphonicTTSSettings \| null \| undefined` | no | Runtime-updatable settings. | | `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. | | `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. | | `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. | | `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. | | `reconnectBackoffMinWait` | `number \| undefined` | no | Minimum time, in seconds, to wait between reconnection attempts. | | `reconnectBackoffMaxWait` | `number \| undefined` | no | Maximum time, in seconds, to wait between reconnection attempts. | | `reconnectOnError` | `boolean \| undefined` | no | Whether to automatically reconnect on connection errors. | | `wsCloseTimeout` | `number \| undefined` | no | Maximum time, in seconds, to wait for the peer to acknowledge the websocket closing handshake before dropping the connection. Applied to connections opened through `_websocketConnect()`. Increase it for peers that need longer to complete a graceful close. | | `wsSendTimeout` | `number \| null \| undefined` | no | Maximum time, in seconds, a single send may wait on a peer that has stopped reading before the connection is dropped and the send throws `ConnectionClosedError`. Applied to connections opened through `_websocketConnect()`. Pass `null` to wait indefinitely. Defaults to `WS_SEND_TIMEOUT`. | | `textAggregationMode` | `TextAggregationMode \| null \| undefined` | no | How to aggregate incoming text before synthesis. `TextAggregationMode.SENTENCE` (default) buffers until sentence boundaries, `TextAggregationMode.TOKEN` streams tokens directly for lower latency. | | `pushTextFrames` | `boolean \| undefined` | no | Whether to push TextFrames and LLMFullResponseEndFrames. | | `pushStopFrames` | `boolean \| undefined` | no | Whether to automatically push TTSStoppedFrames. | | `pushStartFrame` | `boolean \| undefined` | no | Whether to automatically create audio contexts and push TTSStartedFrames. When true, the base class handles `createAudioContext` and yields `TTSStartedFrame` before each synthesis call, so `runTts` implementations do not need to. | | `stopFrameTimeoutS` | `number \| undefined` | no | Idle time before pushing TTSStoppedFrame when pushStopFrames is true. | | `pushSilenceAfterStop` | `boolean \| undefined` | no | Whether to push silence audio after TTSStoppedFrame. | | `silenceTimeS` | `number \| undefined` | no | Duration of silence to push when pushSilenceAfterStop is true. | | `pauseFrameProcessing` | `boolean \| undefined` | no | Whether to hold incoming frames after a turn until its audio contexts end. Needed by services whose audio doesn't carry its request's context ID. | | `maxConsecutiveZeroAudioContexts` | `number \| undefined` | no | How many consecutive TTS contexts may complete without producing any audio before the service is reported unable to do its job. Catches a provider that accepts requests and stays silent — an unknown voice ID, say — which no error ever surfaces. Every silent context reports an error the service can carry on from, whatever the limit is; on reaching the limit the service reports a permanent error instead, stops being given work, and the pipeline worker applies its `ProcessorUnusablePolicy`. Set to 0 to report silent contexts without ever writing the service off. | | `appendTrailingSpace` | `boolean \| undefined` | no | Whether to append a trailing space to text before sending to TTS. This helps prevent some TTS services from vocalizing trailing punctuation (e.g., "dot"). Only applied in sentence aggregation mode; when streaming tokens, the incoming text's own whitespace is preserved. | | `sampleRate` | `number \| null \| undefined` | no | Output sample rate for generated audio. | | `skipTextTypes` | `string[] \| null \| undefined` | no | List of text types that should not be spoken. | | `textTransforms` | `[string, TextTransform][] \| null \| undefined` | no | A list of TextTransforms to transform text just before sending it to TTS. Each TextTransform takes the aggregated text and its type, and returns the transformed text. To register, provide a list of tuples of `[textType \| "*", transformFunction]`. | | `transportDestination` | `string \| null \| undefined` | no | Destination for generated audio frames. | | `reuseContextIdWithinTurn` | `boolean \| undefined` | no | Whether the service should reuse context IDs within the same turn. | | `audioCache` | `TTSAudioCacheStore \| null \| undefined` | no | Where to keep synthesized audio for reuse (e.g. a shared `TTSAudioCache`). Text whose audio is in the cache is served from it, with no request to the provider. Lines get there through `warmAudioCache()`, and also from live synthesis when `cacheSynthesizedAudio` allows it. Only services that yield their audio from `runTts` (HTTP and streaming-RPC services, see `supportsAudioCache`) use it, and only in sentence aggregation mode. Defaults to null (no cache). pipedog only. | | `cacheSynthesizedAudio` | `boolean \| ((text: string) => boolean) \| undefined` | no | Whether audio synthesized live is stored in `audioCache`, or a function that decides per text. Defaults to false: only warmed lines are stored, since a generated reply can repeat what the user said. pipedog only. | | `audioCacheNamespace` | `string \| undefined` | no | Extra part of the cache key, for constructor options that change the audio but are not in `settings`. pipedog only. | ### Settings [#settings] Runtime-updatable settings, passed as `settings: new NeuphonicTTSService.Settings({...})` (`NeuphonicTTSSettings`). Any setting you leave out keeps the default below. A change reconnects the WebSocket, because the settings are query parameters of the connection. | Setting | Type | Default | Description | | ------------- | ------------------ | ------------- | --------------------------------------------------------------------- | | `voice` | `string \| null` | `null` | Voice id. | | `language` | `Language \| null` | `Language.EN` | Language, part of the connection's path. | | `speed` | `number` | `1.0` | Speed multiplier. | | `temperature` | `number \| null` | `null` | Randomness, 0.0 to 1.0. `null` leaves Neuphonic's default. | | `ipaKey` | `string \| null` | `null` | Regional variant for `` tags: `es-castilian` or `pt-br`. | ### Usage [#usage] ```ts title="tts.ts" import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames"; import { Language } from "@pipedog/core/transcriptions/language"; import { NeuphonicHttpTTSService, NeuphonicTTSService } from "@pipedog/neuphonic/tts"; const tts = new NeuphonicTTSService({ apiKey: process.env.NEUPHONIC_API_KEY ?? "", settings: new NeuphonicTTSService.Settings({ voice: "your-voice-id", language: Language.EN, speed: 1.0, }), }); ``` Change settings mid-call with a `TTSUpdateSettingsFrame`: ```ts // Reconnects the WebSocket with the new settings. const update = new TTSUpdateSettingsFrame({ delta: new NeuphonicTTSService.Settings({ speed: 1.2 }), }); ``` ### Event handlers [#event-handlers] | Event | Description | | --------------------- | ------------------------ | | `on_connected` | The WebSocket connected. | | `on_disconnected` | The WebSocket closed. | | `on_connection_error` | The connection failed. | ## NeuphonicHttpTTSService [#neuphonichttpttsservice] Posts each sentence to `https://api.neuphonic.com/sse/speak/`. ### Configuration [#configuration-1] | Option | Type | Required | Description | |---|---|---|---| | `apiKey` | `string` | yes | Neuphonic API key for authentication. | | `fetch` | `{ (input: RequestInfo \| URL, init?: RequestInit): Promise; (input: string \| URL \| Request, init?: RequestInit): Promise; } \| null \| undefined` | no | `fetch` used for HTTP requests. Defaults to the global `fetch`. DIVERGENCE: upstream takes a required shared `aiohttp_session`. | | `url` | `string \| undefined` | no | Base URL for the Neuphonic HTTP API. | | `encoding` | `string \| null \| undefined` | no | Audio encoding format. Defaults to "pcm_linear". | | `settings` | `NeuphonicTTSSettings \| null \| undefined` | no | Runtime-updatable settings. | | `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. | | `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. | | `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. | | `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. | | `textAggregationMode` | `TextAggregationMode \| null \| undefined` | no | How to aggregate incoming text before synthesis. `TextAggregationMode.SENTENCE` (default) buffers until sentence boundaries, `TextAggregationMode.TOKEN` streams tokens directly for lower latency. | | `pushTextFrames` | `boolean \| undefined` | no | Whether to push TextFrames and LLMFullResponseEndFrames. | | `pushStopFrames` | `boolean \| undefined` | no | Whether to automatically push TTSStoppedFrames. | | `pushStartFrame` | `boolean \| undefined` | no | Whether to automatically create audio contexts and push TTSStartedFrames. When true, the base class handles `createAudioContext` and yields `TTSStartedFrame` before each synthesis call, so `runTts` implementations do not need to. | | `stopFrameTimeoutS` | `number \| undefined` | no | Idle time before pushing TTSStoppedFrame when pushStopFrames is true. | | `pushSilenceAfterStop` | `boolean \| undefined` | no | Whether to push silence audio after TTSStoppedFrame. | | `silenceTimeS` | `number \| undefined` | no | Duration of silence to push when pushSilenceAfterStop is true. | | `pauseFrameProcessing` | `boolean \| undefined` | no | Whether to hold incoming frames after a turn until its audio contexts end. Needed by services whose audio doesn't carry its request's context ID. | | `maxConsecutiveZeroAudioContexts` | `number \| undefined` | no | How many consecutive TTS contexts may complete without producing any audio before the service is reported unable to do its job. Catches a provider that accepts requests and stays silent — an unknown voice ID, say — which no error ever surfaces. Every silent context reports an error the service can carry on from, whatever the limit is; on reaching the limit the service reports a permanent error instead, stops being given work, and the pipeline worker applies its `ProcessorUnusablePolicy`. Set to 0 to report silent contexts without ever writing the service off. | | `appendTrailingSpace` | `boolean \| undefined` | no | Whether to append a trailing space to text before sending to TTS. This helps prevent some TTS services from vocalizing trailing punctuation (e.g., "dot"). Only applied in sentence aggregation mode; when streaming tokens, the incoming text's own whitespace is preserved. | | `sampleRate` | `number \| null \| undefined` | no | Output sample rate for generated audio. | | `skipTextTypes` | `string[] \| null \| undefined` | no | List of text types that should not be spoken. | | `textTransforms` | `[string, TextTransform][] \| null \| undefined` | no | A list of TextTransforms to transform text just before sending it to TTS. Each TextTransform takes the aggregated text and its type, and returns the transformed text. To register, provide a list of tuples of `[textType \| "*", transformFunction]`. | | `transportDestination` | `string \| null \| undefined` | no | Destination for generated audio frames. | | `reuseContextIdWithinTurn` | `boolean \| undefined` | no | Whether the service should reuse context IDs within the same turn. | | `audioCache` | `TTSAudioCacheStore \| null \| undefined` | no | Where to keep synthesized audio for reuse (e.g. a shared `TTSAudioCache`). Text whose audio is in the cache is served from it, with no request to the provider. Lines get there through `warmAudioCache()`, and also from live synthesis when `cacheSynthesizedAudio` allows it. Only services that yield their audio from `runTts` (HTTP and streaming-RPC services, see `supportsAudioCache`) use it, and only in sentence aggregation mode. Defaults to null (no cache). pipedog only. | | `cacheSynthesizedAudio` | `boolean \| ((text: string) => boolean) \| undefined` | no | Whether audio synthesized live is stored in `audioCache`, or a function that decides per text. Defaults to false: only warmed lines are stored, since a generated reply can repeat what the user said. pipedog only. | | `audioCacheNamespace` | `string \| undefined` | no | Extra part of the cache key, for constructor options that change the audio but are not in `settings`. pipedog only. | It takes the same settings as the WebSocket service. ### Usage [#usage-1] ```ts const httpTts = new NeuphonicHttpTTSService({ apiKey: process.env.NEUPHONIC_API_KEY ?? "", settings: new NeuphonicHttpTTSService.Settings({ voice: "your-voice-id" }), }); ``` ## Notes [#notes] * The sample rate defaults to 22050 Hz and the encoding to `pcm_linear`. * The WebSocket sends a keepalive (`{"text": ""}`) every 10 seconds. * An SSE event that carries `errors` is reported as an error frame, and the stream keeps going. ## Runtime notes [#runtime-notes] * The WebSocket opens through the core `websocketConnect`, with the key in an `x-api-key` header (through `fetch` on Cloudflare Workers). The HTTP service uses the global `fetch`; pass `fetch` to use another one (upstream takes an aiohttp session). Both work on Bun, Node, Deno and Cloudflare Workers. * Upstream talks to Neuphonic's API directly rather than through the `pyneuphonic` SDK, and so does this port. * The deprecated `params` (`InputParams`), `voiceId` and `aggregateSentences` options are not ported. * Upstream has no Neuphonic tests. Ours run against local fake servers, not against Neuphonic itself. | Runtime | Works | What it needs | | ------------------ | ----- | ------------- | | Bun | ✅ yes | Nothing. | | Node.js | ✅ yes | Nothing. | | Deno | ✅ yes | Nothing. | | Cloudflare Workers | ✅ yes | Nothing. | Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/neuphonic/package.json`: ✅ means the compat smoke test passes on that runtime.