Azure Speech

OverviewCopy Anchor Link

pipedog ports both of Pipecat's Azure text-to-speech services:

• AzureTTSService: streams each sentence's audio over the Speech service's WebSocket synthesis protocol, with word boundaries turned into word-timed TTSTextFrames.
• AzureHttpTTSService: one POST to the REST endpoint per sentence, returning the whole sentence's audio.

Both build the same SSML from their settings (voice, style, prosody, emphasis, HD voice parameters, locale pinning). Upstream drives Azure through the native Speech SDK; pipedog speaks the service's WebSocket and REST protocols directly, so no SDK is needed.

InstallationCopy Anchor Link

bun add @pipedog/azure

import { AzureHttpTTSService, AzureTTSService } from "@pipedog/azure/tts";

PrerequisitesCopy Anchor Link

• An Azure AI Speech (or Azure AI Foundry) resource, with its key and region from the portal's Keys and Endpoint page.
• Pass the key as apiKey and the region (e.g. "eastus") as region, or a Private Link or custom-domain URL as privateEndpoint. The examples read them from AZURE_SPEECH_API_KEY and AZURE_SPEECH_REGION.

AzureTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new AzureTTSService.Settings({...}). Any setting you leave out keeps the default below. AzureHttpTTSService uses the same settings class.

UsageCopy Anchor Link

import { AzureHttpTTSService, AzureTTSService } from "@pipedog/azure/tts";
import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";

const tts = new AzureTTSService({
  apiKey: process.env.AZURE_SPEECH_API_KEY ?? "",
  region: process.env.AZURE_SPEECH_REGION ?? "eastus",
  settings: new AzureTTSService.Settings({
    voice: "en-US-AvaMultilingualNeural",
    language: "en-US",
    style: "cheerful",
    rate: "1.1",
  }),
});

Change settings mid-call with a TTSUpdateSettingsFrame. The SSML is built per sentence, so the change applies from the next one:

// Queue it on the worker, or push it from a processor upstream of the service.
// SSML settings apply from the next sentence.
const update = new TTSUpdateSettingsFrame({
  delta: new AzureTTSService.Settings({ language: "en-GB", forceLocale: true }),
});

NotesCopy Anchor Link

• The synthesis connection opens on the first sentence and is reused. Audio is raw 16-bit mono PCM at the pipeline's output rate (8, 16, 22.05, 24, 44.1 or 48 kHz; other rates fall back to 24 kHz).
• Word boundaries become TTSTextFrames timed against the audio. Trailing punctuation is merged into the word before it, and Chinese, Japanese and Korean characters are merged up to the next punctuation or space. The last word always arrives before the TTSStoppedFrame.
• On an interruption the connection is closed, which stops Azure from sending more audio for the interrupted sentence; the next sentence opens a new one. Upstream calls the SDK's stop_speaking_async() instead.

AzureHttpTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

UsageCopy Anchor Link

const httpTts = new AzureHttpTTSService({
  apiKey: process.env.AZURE_SPEECH_API_KEY ?? "",
  region: process.env.AZURE_SPEECH_REGION ?? "eastus",
  sampleRate: 24000,
});

NotesCopy Anchor Link

• Posts the SSML to https://<region>.tts.speech.microsoft.com/cognitiveservices/v1 (or <privateEndpoint>/tts/cognitiveservices/v1) and yields the sentence as one audio frame.
• It requests a raw-*-16bit-mono-pcm format, which has no WAV header. Upstream strips the 44 header bytes of the SDK's RIFF result instead.
• Pass fetch to send the request through your own client.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/azure/package.json: ✅ means the compat smoke test passes on that runtime.

Both services work on Bun, Node, Deno and Cloudflare Workers: AzureTTSService uses core's websocketConnect, AzureHttpTTSService uses fetch. The key is sent as the Ocp-Apim-Subscription-Key header, and AzureTTSService also sends it as a query parameter, as the Speech SDK does, for runtimes whose WebSocket can't send headers. With a custom-domain privateEndpoint, AzureTTSService first asks the endpoint for its regional redirect, as the SDK does.

The deprecated voice, params (InputParams) and aggregateSentences options aren't ported; use settings and textAggregationMode.