Voice bot over SmallWebRTC

This guide runs a voice assistant end to end: you speak in the browser, OpenAI transcribes it and writes a reply, and the bot speaks the reply back. It is the same bot as examples/smallwebrtc-voice-bot in the repository, and its test runs it in headless Chromium.

The browser side uses the unchanged Pipecat client SDK, @pipecat-ai/client-js with @pipecat-ai/small-webrtc-transport.

git clone https://github.com/polvallverdu/pipedog.git
cd pipedog
bun install

OPENAI_API_KEY=sk-... bun run --cwd examples/smallwebrtc-voice-bot start

VOICE_PROVIDER=google GOOGLE_API_KEY=... bun run --cwd examples/smallwebrtc-voice-bot start

const transport = new SmallWebRTCTransport(
  connection,
  new TransportParams({ audioInEnabled: true, audioOutEnabled: true }),
);

const stt = new OpenAISTTService({ apiKey, baseUrl });
const llm = new OpenAILLMService({
  apiKey,
  baseUrl,
  settings: new OpenAILLMSettings({ systemInstruction: SYSTEM_INSTRUCTION }),
});
const tts = new OpenAITTSService({ apiKey, baseUrl, sampleRate: 24000 });

const context = new LLMContext();
const aggregators = new LLMContextAggregatorPair(context, {
  userParams: new LLMUserAggregatorParams({
    vadAnalyzer: new SileroVADAnalyzer(),
    userTurnStrategies: new UserTurnStrategies({
      stop: [new TurnAnalyzerUserTurnStopStrategy({ turnAnalyzer: createLocalSmartTurnAnalyzerV3() })],
    }),
  }),
});

const pipeline = new Pipeline([
  transport.input(),
  stt,
  aggregators.user(),
  llm,
  tts,
  transport.output(),
  aggregators.assistant(),
]);

bun test examples/smallwebrtc-voice-bot

What the bot does on connectCopy Anchor Link

When the browser connects, the bot adds a developer message asking the model to introduce itself, and queues an LLMRunFrame. That first reply is the greeting. From then on, each user turn runs through the pipeline above.

Configure itCopy Anchor Link

Tracing and alertsCopy Anchor Link

Both are off until you set them:

Alerts are sent once per distinct error, and at most five per conversation. Warnings and errors in the logs have credentials masked, so an error that quotes a URL with ?key=... doesn't leak the key. The wiring is in the example's src/observability.ts.

ReconnectionCopy Anchor Link

The browser SDK reconnects by itself. When the connection fails, or stays disconnected for five seconds, it opens a new peer connection for the same session (restart_pc). The bot keeps the pipeline and the conversation through it, so the call picks up where it was. This is how Pipecat behaves too. A session ends when the browser closes the connection, or when reconnecting gives up.

Next stepsCopy Anchor Link

• Frames and frame processors explain the pipeline's building blocks.
• Runtimes & compatibility lists where SmallWebRTC runs: Bun, Node and Deno, not Cloudflare Workers, which have no UDP.