Cloudflare Workers

Workers upgrade a WebSocket from fetch, so websocketFetchHandler works there as it is. Workers have no UDP, so the WebRTC offer doesn't: audio goes over the WebSocket, as protobuf frames. For WebRTC users, Cloudflare Realtime carries the media and hands the bot PCM over WebSockets.

A voice session is a long-lived connection with state, so each one runs in its own Durable Object. The Worker routes each request to an object by session name. The full guide, with turn detection, reconnection and limits, is Voice agents on Workers and Durable Objects.

InstallCopy Anchor Link

bun add @pipedog/core @pipedog/asyncio @pipedog/transport-websocket @pipedog/serializer-protobuf @pipedog/openai

To run it from a checkout instead, see Installation. The Worker needs the nodejs_compat compatibility flag.

The shapeCopy Anchor Link

client ──WebSocket──► Worker ──getByName(id)──► VoiceSession (Durable Object)
                                                   │ FastAPIWebsocketTransport
                                                   │ OpenAI realtime transcription (turns)
                                                   │ chat, speech synthesis

The Durable Object turns a connection into a fetch handler and returns its result from fetch:

import { DurableObject } from "cloudflare:workers";
import { websocketFetchHandler } from "@pipedog/transport-websocket/fetch";

export class VoiceSession extends DurableObject<Env> {
  readonly #handler = websocketFetchHandler((connection) => runVoiceAgent(connection, this.env));

  override fetch(request: Request): Promise<Response> {
    return this.#handler(request);
  }
}

The Worker routes each session to an object by name. The same name always reaches the same object:

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const id = new URL(request.url).pathname.split("/")[3];
    return env.VOICE_SESSION.getByName(id).fetch(request);
  },
} satisfies ExportedHandler<Env>;

runVoiceAgent builds the pipeline on the connection with FastAPIWebsocketTransport. It detects turns on the provider's side, not with Silero and Smart Turn: the local models don't fit in a Worker's size limit.

Run itCopy Anchor Link

The complete example is examples/workers-durable-voice-agent:

cd examples/workers-durable-voice-agent
echo "OPENAI_API_KEY=sk-..." > .dev.vars
bun run dev

Its tests run against workerd through wrangler dev. To deploy, see Run it in the full guide.

AuthenticationCopy Anchor Link

The Worker has no auth. Check the session in the Worker's fetch before it reaches the Durable Object, as in Authentication. Browsers can't set headers on a WebSocket, so take the token from the URL.