SmallWebRTC

OverviewCopy Anchor Link

SmallWebRTC connects one browser (or any WebRTC peer) straight to the bot, with no media server in between. Audio goes over RTP as Opus, and RTVI and app messages go over a data channel. It works with the Pipecat browser SDK's SmallWebRTCTransport.

• SmallWebRTCTransport: the pipeline transport for one peer connection: audio in and out, optional camera and screen share, and app messages.
• SmallWebRTCConnection: the peer connection itself (upstream on aiortc, here on werift), with ICE servers, tracks, the data channel and renegotiation.
• SmallWebRTCRequestHandler: answers SDP offers, creates or reuses connections and applies ICE candidates. Framework-agnostic.
• smallWebRTCOfferHandler: the signalling endpoint (POST and PATCH /api/offer) as a fetch handler, (Request) => Promise<Response>.
• nodeRequestListener: adapts a fetch handler to Node's http.createServer().

InstallationCopy Anchor Link

bun add @pipedog/transport-smallwebrtc

import { SmallWebRTCTransport } from "@pipedog/transport-smallwebrtc/transport";
import { IceServer, SmallWebRTCConnection } from "@pipedog/transport-smallwebrtc/connection";
import { SmallWebRTCRequestHandler } from "@pipedog/transport-smallwebrtc/request_handler";
import { nodeRequestListener, smallWebRTCOfferHandler } from "@pipedog/transport-smallwebrtc/http";

The package depends on werift (a pure TypeScript WebRTC stack: ICE, DTLS, SRTP, SCTP, RTP) and libopus-wasm (libopus compiled to WebAssembly). Neither needs native code.

PrerequisitesCopy Anchor Link

• A long-lived server process that can send and receive UDP. Serverless functions and Cloudflare Workers can't run the media.
• A client: the Pipecat browser SDK with @pipecat-ai/small-webrtc-transport, pointed at your offer endpoint.
• For peers behind restrictive NATs, a TURN server (see ICE servers).

SmallWebRTCTransportCopy Anchor Link

ConfigurationCopy Anchor Link

new SmallWebRTCTransport(connection, params, options?). connection is the SmallWebRTCConnection from the request handler. params is a plain TransportParams (audioInEnabled, audioOutEnabled, videoInEnabled, videoOutEnabled and so on). options:

UsageCopy Anchor Link

Serve the offer endpoint and start a bot for each new connection:

import { createServer } from "node:http";
import { createTask } from "@pipedog/asyncio";
import { Pipeline } from "@pipedog/core/pipeline/pipeline";
import { PipelineWorker } from "@pipedog/core/pipeline/worker";
import { TransportParams } from "@pipedog/core/transports/base_transport";
import { WorkerRunner } from "@pipedog/core/workers/runner";
import { IceServer, type SmallWebRTCConnection } from "@pipedog/transport-smallwebrtc/connection";
import { nodeRequestListener, smallWebRTCOfferHandler } from "@pipedog/transport-smallwebrtc/http";
import {
  ConnectionMode,
  SmallWebRTCRequestHandler,
} from "@pipedog/transport-smallwebrtc/request_handler";
import { SmallWebRTCTransport } from "@pipedog/transport-smallwebrtc/transport";

/** Runs one bot on a new peer connection. */
async function runBot(connection: SmallWebRTCConnection): Promise<void> {
  const transport = new SmallWebRTCTransport(
    connection,
    new TransportParams({ audioInEnabled: true, audioOutEnabled: true }),
  );

  const pipeline = new Pipeline([
    transport.input(),
    // ... STT, LLM context aggregators, LLM, TTS
    transport.output(),
  ]);
  const worker = new PipelineWorker(pipeline);
  const runner = new WorkerRunner({ handleSigint: false });

  transport.eventHandler("on_client_connected", async () => {
    console.log("client connected");
  });
  transport.eventHandler("on_client_disconnected", async () => {
    await runner.cancel();
  });

  await runner.addWorkers(worker);
  await runner.run();
}

// POST /api/offer takes the browser's SDP offer and answers it; PATCH sends ICE candidates.
const offer = smallWebRTCOfferHandler(new SmallWebRTCRequestHandler(), (connection) => {
  // Start the bot in the background: the answer is sent once this returns.
  createTask(() => runBot(connection), "voice-bot");
});

Bun.serve({
  port: 7860,
  fetch(request) {
    if (new URL(request.url).pathname === "/api/offer") return offer(request);
    return new Response("not found", { status: 404 });
  },
});

On Node, wrap the same handler with nodeRequestListener:

// On Node, serve the same handler with the http module instead of Bun.serve.
createServer(nodeRequestListener(offer)).listen(7860);

App messages from the peer arrive on the data channel:

async function runWithMessages(connection: SmallWebRTCConnection): Promise<void> {
  const transport = new SmallWebRTCTransport(
    connection,
    new TransportParams({ audioInEnabled: true, audioOutEnabled: true }),
  );

  transport.eventHandler("on_app_message", async (_transport, message: unknown, sender: string) => {
    console.log(`message from ${sender}:`, message);
    // Reply on the data channel.
    connection.sendAppMessage({ type: "ack" });
  });
}

Event handlersCopy Anchor Link

NotesCopy Anchor Link

• After on_client_connected runs, the input pushes a ClientConnectedFrame. App messages are also pushed into the pipeline as InputTransportMessageFrames.
• Video sources listed in params.videoInSources are captured before on_client_connected runs, so a handler that captures a source itself wins. captureParticipantVideo() captures the camera ("camera") or the screen share ("screenVideo"), at 30 frames per second by default.
• getClientId(client) returns the peer connection's pcId.
• When the peer replaces the connection (restart_pc), the old track completes its pending writes instead of leaving them waiting. Upstream leaves them pending, so the output transport's write timeout fails the pipeline.

SmallWebRTCConnectionCopy Anchor Link

You rarely build one yourself: SmallWebRTCRequestHandler creates one per offer, with the options from its connectionOptions and iceServers.

ConfigurationCopy Anchor Link

Event handlersCopy Anchor Link

NotesCopy Anchor Link

• If the connection stays in the connecting state longer than connectionTimeoutSecs (default 60), it is closed.
• sendAppMessage(message) sends a JSON message on the data channel.
• Audio is decoded straight to the pipeline's input rate and channel count when Opus supports it (8, 12, 16, 24 or 48 kHz). aiortc always decodes to 48 kHz stereo.
• Opus decoding uses packet loss concealment and in-band FEC. Pass opusCodec, or call setOpusCodec(), to replace libopus-wasm (with a native binding, for example).
• There is no built-in video codec. Without videoDecoderFactory, incoming video is negotiated but produces no frames; without videoEncoderFactory, outgoing video frames are dropped. A warning is logged once in each case. VP8 and H.264 depayloading and packetization are built in.
• The SCTP data chunk payload is 1100 bytes, so data channel packets fit a 1280-byte MTU. Change it with the PIPECAT_SCTP_MAX_CHUNK_SIZE environment variable.

SmallWebRTCRequestHandlerCopy Anchor Link

ConfigurationCopy Anchor Link

ICE serversCopy Anchor Link

Without ICE servers, only host candidates are gathered, which works on a LAN or localhost. Add a STUN server to cross NATs, and a TURN server for networks that block direct UDP:

// A TURN server, for peers behind NATs that block direct connections.
const handler = new SmallWebRTCRequestHandler({
  iceServers: [
    new IceServer({ urls: "stun:stun.l.google.com:19302" }),
    new IceServer({
      urls: "turn:turn.example.com:3478",
      username: process.env.TURN_USERNAME ?? "",
      credential: process.env.TURN_CREDENTIAL ?? "",
    }),
  ],
  // Accept one peer at a time.
  connectionMode: ConnectionMode.SINGLE,
});

updateIceServers() changes the list for connections created afterwards.

NotesCopy Anchor Link

• ConnectionMode.MULTIPLE (the default) accepts several peers. ConnectionMode.SINGLE keeps one: an offer with a different pc_id, or with none while a connection exists, gets HTTP 400.
• An offer with the pc_id of an existing connection renegotiates it instead of creating a new one.
• esp32Mode removes the SHA-384 and SHA-512 fingerprints ESP32 peers don't support and, with host, keeps only the ICE candidates that match it.

smallWebRTCOfferHandlerCopy Anchor Link

smallWebRTCOfferHandler(handler, onConnection) serves one URL, such as /api/offer, with the same contract as Pipecat's development runner:

onConnection(connection, request) runs for each new connection, with the parsed SmallWebRTCRequest (its requestData carries the client's custom data). Start the bot in the background and return: the answer is sent once it returns.

Errors follow FastAPI: an HTTPException becomes its status with {"detail": ...}, a malformed body a 400, another method a 405 and anything else a 500.

It is a standard fetch handler, so it mounts in Bun.serve, Deno.serve, Hono and most frameworks; see Frameworks. On Node, nodeRequestListener(fetchHandler) turns it into a listener for http.createServer().

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/transport-smallwebrtc/package.json: ✅ means the compat smoke test passes on that runtime.

WebRTC media runs over UDP. Bun, Node and Deno can open UDP sockets; Cloudflare Workers can't, so the package does not run there. Use the WebSocket transport on Workers. See Runtimes.

The signalling endpoint is plain HTTP, but the connection it creates lives in the same process, so the offer endpoint and the media must run on the same long-lived server.

Upstream encodes and decodes with PyAV (FFmpeg). Here Opus comes from libopus-wasm and video codecs are pluggable, as described above. werift's API wears an aiortc-shaped adapter (rtc.ts), so the ported code keeps upstream's structure.