How phone calls connect

The path of a callCopy Anchor Link

A phone bot is a pipeline whose transport is a WebSocket that a phone provider opens to your server. The provider handles the phone network; your server handles the conversation.

caller ─► phone network ─► provider (Twilio, Telnyx, Plivo, Exotel, ...)
                               │ 1. webhook: "a call came in"  ──►  your server answers with
                               │                                    "stream it to wss://…/ws"
                               │ 2. WebSocket to wss://…/ws    ◄─►  transport + serializer
                               ▼                                    + your pipeline

1. The call arrives. The provider calls your webhook (Twilio and Telnyx with TwiML/TeXML, Plivo with XML, Exotel and Genesys through a flow you configure in their console). The answer tells the provider to open a bidirectional media stream to your WebSocket URL, optionally with parameters such as the caller's number.
2. The stream opens. The provider connects to your WebSocket and sends a handshake: a connected message, then a start message with the stream ID and call ID.
3. The conversation. Audio flows both ways as JSON messages (base64 μ-law at 8 kHz for most providers) or binary frames (Vonage, Genesys). The provider's serializer turns them into InputAudioRawFrames at your pipeline's rate, and your bot's audio back into the provider's format. Touch-tone keys become InputDTMFFrames; an interruption tells the provider to drop buffered audio.
4. The end. When the caller hangs up, the provider closes the socket and the transport fires on_client_disconnected. When your bot ends the call (an EndFrame), serializers that support it hang up through the provider's REST API, so the caller isn't left on a silent line.

All of this runs over a plain WebSocket, so the same bot runs on Bun, Node, Deno and Cloudflare Workers. Accept the socket with websocketFetchHandler from @pipedog/transport-websocket: Deno and Workers upgrade from fetch on their own, Bun upgrades with the server it passes to fetch, and serveFetch() serves the handler on Bun, Node and Deno. See WebSocket.

SerializersCopy Anchor Link

Set addWavHeader: false on FastAPIWebsocketParams for all of them.

Reading the handshakeCopy Anchor Link

@pipedog/runner reads the handshake for you. parseTelephonyWebsocket(websocket) is upstream's parse_telephony_websocket: it reads the first two messages, detects Twilio, Telnyx, Plivo or Exotel, and returns [transportType, callData]. callData.callId is the provider's call identifier (Twilio's callSid, Telnyx's call_control_id, Plivo's callId, Exotel's call_sid), and callData.fromNumber and callData.toNumber are filled where the provider sends them (on Twilio, from the from_number and to_number stream parameters).

telephonyHandshake(websocket) returns the same parse as a typed outcome, so a caller who hangs up while the call is still connecting is an ordinary result rather than an exception:

import type { FrameSerializer } from "@pipedog/core/serializers/base_serializer";
import type { CallData, TelnyxCallData } from "@pipedog/runner/types";
import { telephonyHandshake } from "@pipedog/runner/utils";
import { ExotelFrameSerializer } from "@pipedog/serializer-exotel/exotel";
import { PlivoFrameSerializer } from "@pipedog/serializer-plivo/plivo";
import { TelnyxFrameSerializer } from "@pipedog/serializer-telnyx/telnyx";
import { TwilioFrameSerializer } from "@pipedog/serializer-twilio/twilio";
import { websocketFetchHandler } from "@pipedog/transport-websocket/fetch";

const env = process.env;

/** One serializer per provider, from the parsed handshake. */
function serializerFor(transportType: string, callData: CallData): FrameSerializer {
  const streamId = callData.streamId ?? "";
  switch (transportType) {
    case "twilio":
      return new TwilioFrameSerializer({
        streamSid: streamId,
        callSid: callData.callId,
        accountSid: env.TWILIO_ACCOUNT_SID,
        authToken: env.TWILIO_AUTH_TOKEN,
      });
    case "telnyx":
      return new TelnyxFrameSerializer({
        streamId,
        callControlId: callData.callId,
        outboundEncoding: (callData as TelnyxCallData).outboundEncoding ?? "PCMU",
        inboundEncoding: "PCMU",
        apiKey: env.TELNYX_API_KEY,
      });
    case "plivo":
      return new PlivoFrameSerializer({
        streamId,
        callId: callData.callId,
        authId: env.PLIVO_AUTH_ID,
        authToken: env.PLIVO_AUTH_TOKEN,
      });
    default:
      return new ExotelFrameSerializer({ streamSid: streamId, callSid: callData.callId });
  }
}

const handler = websocketFetchHandler(async (websocket) => {
  const handshake = await telephonyHandshake(websocket);
  if (handshake.status === "closed") return; // the caller hung up while connecting
  if (handshake.status === "unrecognized") throw new Error("not a phone provider");

  const { transportType, callData } = handshake;
  console.log(`${transportType} call ${callData.callId} from ${callData.fromNumber}`);
  const serializer = serializerFor(transportType, callData);
  // new FastAPIWebsocketTransport(websocket, new FastAPIWebsocketParams({ serializer, ... }))
});

bun add @pipedog/runner

The same package is the development runner: main() serves the webhook for Twilio (TwiML), Telnyx (TeXML) and Plivo (XML) and the /ws media stream for those and Exotel, and createTransport() builds the right serializer from the handshake, so a phone bot is one bot(runnerArgs) function.

NextCopy Anchor Link

• Twilio quickstart: a working phone bot from examples/twilio-phone-bot.
• Testing without a phone: the example's tests drive the bot with a scripted fake Twilio that speaks the Media Streams protocol, against mock OpenAI APIs.