Fish Audio

OverviewCopy Anchor Link

Fish Audio turns text into speech, including with cloned voices. pipedog ports Pipecat's FishAudioTTSService: it streams over Fish Audio's real-time WebSocket (wss://api.fish.audio/v1/tts/live), speaking MessagePack, and reconnects when the user interrupts the bot.

InstallationCopy Anchor Link

bun add @pipedog/fish

import { FishAudioTTSService } from "@pipedog/fish/tts";

PrerequisitesCopy Anchor Link

• A Fish Audio account and an API key.
• A voice model's reference id, set as voice. It is sent as reference_id.
• Pass the key as apiKey. The examples read it from FISH_API_KEY.

FishAudioTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new FishAudioTTSService.Settings({...}).

Settings left null are not sent, so Fish Audio applies its own defaults. Settings.fromMapping() also accepts upstream's nested prosody dict.

UsageCopy Anchor Link

import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { FishAudioTTSService } from "@pipedog/fish/tts";

const tts = new FishAudioTTSService({
  apiKey: process.env.FISH_API_KEY ?? "",
  settings: new FishAudioTTSService.Settings({
    // A voice model's reference id from the Fish Audio library.
    voice: "your-reference-id",
    model: "s2.1-pro",
    latency: "balanced",
    prosodySpeed: 1.1,
  }),
});

Change settings mid-call with a TTSUpdateSettingsFrame:

// Queue it on the worker, or push it from a processor upstream of the service.
// A change to any setting reconnects, because they are all sent in the start message.
const update = new TTSUpdateSettingsFrame({
  delta: new FishAudioTTSService.Settings({ prosodySpeed: 0.9, prosodyVolume: 2 }),
});

Event handlersCopy Anchor Link

Inherited from TTSService:

NotesCopy Anchor Link

• Each text chunk is sent with a flush, which makes Fish Audio generate the audio right away.
• The service reconnects on an interruption while the bot is speaking, and on any settings change.
• outputFormat is "pcm" by default. The service emits the audio as it arrives and does not produce word timestamps.

Runtime notesCopy Anchor Link

• The socket is the global WebSocket. The Authorization and model headers go on the handshake, which works on Bun, Node and Deno, and on Cloudflare Workers, where the socket is opened through fetch.
• Upstream packs messages with ormsgpack. pipedog uses @msgpack/msgpack, which is pure TypeScript. It writes a whole-number float such as prosodySpeed: 1.0 as a MessagePack integer.
• Not ported: the deprecated InputParams and the referenceId, modelId and params constructor options. Use settings.

Generated from pipecat.runtimes and pipecat.compat in packages/fish/package.json: ✅ means the compat smoke test passes on that runtime.