# Twilio

> Phone calls over Twilio Media Streams with TwilioFrameSerializer, used with FastAPIWebsocketTransport.

Source: https://polvallverdu.github.io/pipedog/docs/serializers/twilio/



## Overview [#overview]

**`TwilioFrameSerializer`** speaks Twilio's Media Streams WebSocket protocol. Paired with
[`FastAPIWebsocketTransport`](https://polvallverdu.github.io/pipedog/docs/transports/websocket.md), it lets a bot answer phone calls:

* Caller audio (μ-law at 8 kHz) becomes `InputAudioRawFrame`s at the pipeline's input rate.
* Bot audio is resampled to 8 kHz, μ-law encoded and sent as `media` events.
* Touch-tone digits become `InputDTMFFrame`s.
* An `InterruptionFrame` sends `clear`, so Twilio drops the audio it has buffered.
* An `EndFrame` or `CancelFrame` hangs up the call through Twilio's REST API (`autoHangUp`).

New to phone bots? Start with [How phone calls connect](https://polvallverdu.github.io/pipedog/docs/telephony.md) and the
[Twilio quickstart](https://polvallverdu.github.io/pipedog/docs/telephony/twilio-quickstart.md).

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/serializers/twilio">
    Pipecat's documentation for the same serializer.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/serializer-twilio/src">
    `packages/serializer-twilio/src/twilio.ts`.
  </Card>

  <Card title="Twilio docs" href="https://www.twilio.com/docs/voice/media-streams/websocket-messages">
    Media Streams WebSocket messages.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/serializer-twilio @pipedog/transport-websocket @pipedog/runner
```

```ts
import { TwilioFrameSerializer } from "@pipedog/serializer-twilio/twilio";
```

## Prerequisites [#prerequisites]

* A Twilio account and a phone number with voice, whose incoming-call webhook returns TwiML with
  `<Connect><Stream url="wss://your.host/ws" />`.
* For auto hang-up, your account SID and one credential: the auth token, or an API key (its SID,
  which starts with `SK`, and its secret). The examples read them from `TWILIO_ACCOUNT_SID`,
  `TWILIO_AUTH_TOKEN`, `TWILIO_API_KEY` and `TWILIO_API_SECRET`. Each Twilio region has its own
  auth token and API keys, so use the ones that belong to the region you hang up through.
* The stream SID and call SID, which Twilio sends in its `start` message.
  [`parseTelephonyWebsocket`](https://polvallverdu.github.io/pipedog/docs/telephony/#reading-the-handshake) reads them for you.

## TwilioFrameSerializer [#twilioframeserializer]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `streamSid` | `string` | yes | The Twilio Media Stream SID. |
| `callSid` | `string \| null \| undefined` | no | The associated Twilio Call SID (optional, but required for auto hang-up). |
| `accountSid` | `string \| null \| undefined` | no | Twilio account SID (required for auto hang-up). |
| `authToken` | `string \| null \| undefined` | no | Twilio Auth Token for auto hang-up. Mutually exclusive with apiKeySid/apiKeySecret. |
| `apiKeySid` | `string \| null \| undefined` | no | Twilio API key SID (starts with "SK"). Alternative to authToken for auto hang-up. Must be provided with apiKeySecret. |
| `apiKeySecret` | `string \| null \| undefined` | no | Twilio API key secret. Must be provided with apiKeySid. |
| `region` | `string \| null \| undefined` | no | Twilio region (e.g., "au1", "ie1"). Must be specified with edge. |
| `edge` | `string \| null \| undefined` | no | Twilio edge location (e.g., "sydney", "dublin"). Must be specified with region. |
| `baseUrl` | `string \| null \| undefined` | no | Optional REST API base URL (scheme + host, e.g. `https://api.twilio.com`) used for auto hang-up. Defaults to the host derived from region/edge. Set this to target a Twilio-API-compatible backend or a self-hosted server. |
| `params` | `TwilioFrameSerializerInputParams \| null \| undefined` | no | Configuration parameters. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |

### InputParams [#inputparams]

`params` is a `TwilioFrameSerializer.InputParams` (`TwilioFrameSerializerInputParams`):

| Param                     | Type             | Default | Description                                                                                    |
| ------------------------- | ---------------- | ------- | ---------------------------------------------------------------------------------------------- |
| `twilioSampleRate`        | `number`         | `8000`  | Sample rate Twilio uses, in Hz.                                                                |
| `sampleRate`              | `number \| null` | `null`  | Override for the pipeline input rate. `null` uses the pipeline's.                              |
| `autoHangUp`              | `boolean`        | `true`  | Hang up the call on `EndFrame` or `CancelFrame`.                                               |
| `resamplerClearAfterSecs` | `number \| null` | `0.2`   | Seconds of silence after which the stream resamplers clear their history. `null` never clears. |
| `ignoreRtviMessages`      | `boolean`        | `true`  | Drop RTVI messages instead of sending them to Twilio.                                          |

### Usage [#usage]

Read Twilio's handshake to get the call details, then build the serializer and the transport:

```ts title="twilio.ts"
import { parseTelephonyWebsocket } from "@pipedog/runner/utils";
import { TwilioFrameSerializer } from "@pipedog/serializer-twilio/twilio";
import {
  FastAPIWebsocketParams,
  FastAPIWebsocketTransport,
} from "@pipedog/transport-websocket/fastapi";
import { websocketFetchHandler } from "@pipedog/transport-websocket/fetch";
import type { WebsocketConnection } from "@pipedog/transport-websocket/socket";

async function runCall(connection: WebsocketConnection): Promise<void> {
  // Twilio sends "connected", then "start" with the stream and call SIDs.
  const [, callData] = await parseTelephonyWebsocket(connection);

  const serializer = new TwilioFrameSerializer({
    streamSid: callData.streamId ?? "",
    callSid: callData.callId,
    accountSid: process.env.TWILIO_ACCOUNT_SID,
    authToken: process.env.TWILIO_AUTH_TOKEN,
  });

  const transport = new FastAPIWebsocketTransport(
    connection,
    new FastAPIWebsocketParams({
      audioInEnabled: true,
      audioOutEnabled: true,
      addWavHeader: false,
      serializer,
    }),
  );
  // ... build and run the pipeline with transport.input() and transport.output()
}

// The <Stream url="wss://..."> in your TwiML points here.
const handler = websocketFetchHandler(runCall);
```

To hang up with an API key instead of the auth token, pass `apiKeySid` and `apiKeySecret`:

```ts
const withApiKey = new TwilioFrameSerializer({
  streamSid: "MZ...",
  callSid: "CA...",
  accountSid: process.env.TWILIO_ACCOUNT_SID,
  // An API key replaces the auth token. Pass both parts, and no authToken.
  apiKeySid: process.env.TWILIO_API_KEY,
  apiKeySecret: process.env.TWILIO_API_SECRET,
});
```

To hang up through a regional API host, pass both `region` and `edge`:

```ts
const regional = new TwilioFrameSerializer({
  streamSid: "MZ...",
  callSid: "CA...",
  accountSid: "AC...",
  authToken: "...",
  // Hang up through api.dublin.ie1.twilio.com. Both are required together.
  region: "ie1",
  edge: "dublin",
});
```

Without auto hang-up, only the stream SID is needed:

```ts
const noHangUp = new TwilioFrameSerializer({
  streamSid: "MZ...",
  params: new TwilioFrameSerializer.InputParams({ autoHangUp: false }),
});
```

### Notes [#notes]

* With `autoHangUp` on (the default), the constructor throws if `callSid` or `accountSid` is
  missing, if there is neither an `authToken` nor an API key, if both an `authToken` and any part
  of an API key are given, if only one of `apiKeySid` and `apiKeySecret` is set, or if only one of
  `region` and `edge` is set (unless `baseUrl` is).
* The hang-up is a `POST` of `Status=completed` to
  `https://api.twilio.com/2010-04-01/Accounts/{accountSid}/Calls/{callSid}.json` with Basic auth
  (`accountSid` and `authToken`, or `apiKeySid` and `apiKeySecret`), attempted once. A 404 with code 20404 (call already ended) is logged at debug level; other
  failures are logged as errors and don't throw. `baseUrl` replaces the host, for a
  Twilio-compatible API.
* When the caller hangs up first, the socket is already closed, so no hang-up request is made.
* Set `addWavHeader: false` on the transport params: Twilio takes raw μ-law.
* `OutputTransportMessageFrame`s are sent as JSON, except RTVI messages while
  `ignoreRtviMessages` is on.
* A DTMF event whose digit is not a keypad key is ignored. Malformed JSON and `media` events
  without a payload are ignored too.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/serializer-twilio/package.json`: ✅ means the compat smoke test passes on that runtime.

The serializer uses the global `fetch` for the hang-up and runs on Bun, Node, Deno and Cloudflare
Workers. Accept the Twilio WebSocket with `websocketFetchHandler` (Deno, Workers) or any of the
[WebSocket](https://polvallverdu.github.io/pipedog/docs/transports/websocket.md) server adapters.

Resampling between 8 kHz and the pipeline rate goes through core's resampler engine
(`@pipedog/resampler-wasm` by default) in place of libsoxr, so resampled samples differ slightly
from upstream's. At 8 kHz the wire output is byte-identical; the package's golden tests replay
vectors recorded from upstream.