Twilio

OverviewCopy Anchor Link

TwilioFrameSerializer speaks Twilio's Media Streams WebSocket protocol. Paired with FastAPIWebsocketTransport, it lets a bot answer phone calls:

• Caller audio (μ-law at 8 kHz) becomes InputAudioRawFrames at the pipeline's input rate.
• Bot audio is resampled to 8 kHz, μ-law encoded and sent as media events.
• Touch-tone digits become InputDTMFFrames.
• An InterruptionFrame sends clear, so Twilio drops the audio it has buffered.
• An EndFrame or CancelFrame hangs up the call through Twilio's REST API (autoHangUp).

New to phone bots? Start with How phone calls connect and the Twilio quickstart.

InstallationCopy Anchor Link

bun add @pipedog/serializer-twilio @pipedog/transport-websocket @pipedog/runner

import { TwilioFrameSerializer } from "@pipedog/serializer-twilio/twilio";

PrerequisitesCopy Anchor Link

• A Twilio account and a phone number with voice, whose incoming-call webhook returns TwiML with <Connect><Stream url="wss://your.host/ws" />.
• For auto hang-up, your account SID and one credential: the auth token, or an API key (its SID, which starts with SK, and its secret). The examples read them from TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_API_KEY and TWILIO_API_SECRET. Each Twilio region has its own auth token and API keys, so use the ones that belong to the region you hang up through.
• The stream SID and call SID, which Twilio sends in its start message. parseTelephonyWebsocket reads them for you.

TwilioFrameSerializerCopy Anchor Link

ConfigurationCopy Anchor Link

InputParamsCopy Anchor Link

params is a TwilioFrameSerializer.InputParams (TwilioFrameSerializerInputParams):

UsageCopy Anchor Link

Read Twilio's handshake to get the call details, then build the serializer and the transport:

import { parseTelephonyWebsocket } from "@pipedog/runner/utils";
import { TwilioFrameSerializer } from "@pipedog/serializer-twilio/twilio";
import {
  FastAPIWebsocketParams,
  FastAPIWebsocketTransport,
} from "@pipedog/transport-websocket/fastapi";
import { websocketFetchHandler } from "@pipedog/transport-websocket/fetch";
import type { WebsocketConnection } from "@pipedog/transport-websocket/socket";

async function runCall(connection: WebsocketConnection): Promise<void> {
  // Twilio sends "connected", then "start" with the stream and call SIDs.
  const [, callData] = await parseTelephonyWebsocket(connection);

  const serializer = new TwilioFrameSerializer({
    streamSid: callData.streamId ?? "",
    callSid: callData.callId,
    accountSid: process.env.TWILIO_ACCOUNT_SID,
    authToken: process.env.TWILIO_AUTH_TOKEN,
  });

  const transport = new FastAPIWebsocketTransport(
    connection,
    new FastAPIWebsocketParams({
      audioInEnabled: true,
      audioOutEnabled: true,
      addWavHeader: false,
      serializer,
    }),
  );
  // ... build and run the pipeline with transport.input() and transport.output()
}

// The <Stream url="wss://..."> in your TwiML points here.
const handler = websocketFetchHandler(runCall);

To hang up with an API key instead of the auth token, pass apiKeySid and apiKeySecret:

const withApiKey = new TwilioFrameSerializer({
  streamSid: "MZ...",
  callSid: "CA...",
  accountSid: process.env.TWILIO_ACCOUNT_SID,
  // An API key replaces the auth token. Pass both parts, and no authToken.
  apiKeySid: process.env.TWILIO_API_KEY,
  apiKeySecret: process.env.TWILIO_API_SECRET,
});

To hang up through a regional API host, pass both region and edge:

const regional = new TwilioFrameSerializer({
  streamSid: "MZ...",
  callSid: "CA...",
  accountSid: "AC...",
  authToken: "...",
  // Hang up through api.dublin.ie1.twilio.com. Both are required together.
  region: "ie1",
  edge: "dublin",
});

Without auto hang-up, only the stream SID is needed:

const noHangUp = new TwilioFrameSerializer({
  streamSid: "MZ...",
  params: new TwilioFrameSerializer.InputParams({ autoHangUp: false }),
});

NotesCopy Anchor Link

• With autoHangUp on (the default), the constructor throws if callSid or accountSid is missing, if there is neither an authToken nor an API key, if both an authToken and any part of an API key are given, if only one of apiKeySid and apiKeySecret is set, or if only one of region and edge is set (unless baseUrl is).
• The hang-up is a POST of Status=completed to https://api.twilio.com/2010-04-01/Accounts/{accountSid}/Calls/{callSid}.json with Basic auth (accountSid and authToken, or apiKeySid and apiKeySecret), attempted once. A 404 with code 20404 (call already ended) is logged at debug level; other failures are logged as errors and don't throw. baseUrl replaces the host, for a Twilio-compatible API.
• When the caller hangs up first, the socket is already closed, so no hang-up request is made.
• Set addWavHeader: false on the transport params: Twilio takes raw μ-law.
• OutputTransportMessageFrames are sent as JSON, except RTVI messages while ignoreRtviMessages is on.
• A DTMF event whose digit is not a keypad key is ignored. Malformed JSON and media events without a payload are ignored too.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/serializer-twilio/package.json: ✅ means the compat smoke test passes on that runtime.

The serializer uses the global fetch for the hang-up and runs on Bun, Node, Deno and Cloudflare Workers. Accept the Twilio WebSocket with websocketFetchHandler (Deno, Workers) or any of the WebSocket server adapters.

Resampling between 8 kHz and the pipeline rate goes through core's resampler engine (@pipedog/resampler-wasm by default) in place of libsoxr, so resampled samples differ slightly from upstream's. At 8 kHz the wire output is byte-identical; the package's golden tests replay vectors recorded from upstream.