# Voice bot over SmallWebRTC > A voice assistant you talk to in the browser, with OpenAI speech-to-text, chat and text-to-speech. Source: https://polvallverdu.github.io/pipedog/docs/voice-bot/ This guide runs a voice assistant end to end: you speak in the browser, OpenAI transcribes it and writes a reply, and the bot speaks the reply back. It is the same bot as [`examples/smallwebrtc-voice-bot`](https://github.com/polvallverdu/pipedog/tree/main/examples/smallwebrtc-voice-bot) in the repository, and its test runs it in headless Chromium. The browser side uses the unchanged Pipecat client SDK, `@pipecat-ai/client-js` with `@pipecat-ai/small-webrtc-transport`. ### Get the example [#get-the-example] Clone the repository and install the workspace: ```bash git clone https://github.com/polvallverdu/pipedog.git cd pipedog bun install ``` ### Run it [#run-it] Set an OpenAI API key and start the server: ```bash OPENAI_API_KEY=sk-... bun run --cwd examples/smallwebrtc-voice-bot start ``` Open [http://localhost:7860](http://localhost:7860), click **Connect**, allow the microphone, and talk. The bot introduces itself, then answers your question. To use Google Gemini instead of OpenAI, set `VOICE_PROVIDER=google` and `GOOGLE_API_KEY`. The pipeline is the same, with Gemini Live for transcription, Gemini for the reply and Gemini speech synthesis for the voice. ```bash VOICE_PROVIDER=google GOOGLE_API_KEY=... bun run --cwd examples/smallwebrtc-voice-bot start ``` ### Read the pipeline [#read-the-pipeline] The whole bot is one function, `runVoiceBot`, which builds a pipeline for each connection: ```ts const transport = new SmallWebRTCTransport( connection, new TransportParams({ audioInEnabled: true, audioOutEnabled: true }), ); const stt = new OpenAISTTService({ apiKey, baseUrl }); const llm = new OpenAILLMService({ apiKey, baseUrl, settings: new OpenAILLMSettings({ systemInstruction: SYSTEM_INSTRUCTION }), }); const tts = new OpenAITTSService({ apiKey, baseUrl, sampleRate: 24000 }); const context = new LLMContext(); const aggregators = new LLMContextAggregatorPair(context, { userParams: new LLMUserAggregatorParams({ vadAnalyzer: new SileroVADAnalyzer(), userTurnStrategies: new UserTurnStrategies({ stop: [new TurnAnalyzerUserTurnStopStrategy({ turnAnalyzer: createLocalSmartTurnAnalyzerV3() })], }), }), }); const pipeline = new Pipeline([ transport.input(), stt, aggregators.user(), llm, tts, transport.output(), aggregators.assistant(), ]); ``` Audio from the browser arrives at 16 kHz. Silero VAD detects when the user starts speaking, and Smart Turn decides when they have finished, so the bot does not answer mid-sentence. The transcript goes to the chat model, and the reply goes to speech synthesis at 24 kHz, the rate OpenAI's speech endpoint returns. ### Check it with a scripted user [#check-it-with-a-scripted-user] The example's test plays a recorded question into a headless browser's microphone and checks that the bot answers it. OpenAI is replaced by a mock, so the test needs no key: ```bash bun test examples/smallwebrtc-voice-bot ``` The test needs Chromium for Playwright (`bunx playwright-core install chromium` in the example's directory). ## What the bot does on connect [#what-the-bot-does-on-connect] When the browser connects, the bot adds a developer message asking the model to introduce itself, and queues an `LLMRunFrame`. That first reply is the greeting. From then on, each user turn runs through the pipeline above. ## Configure it [#configure-it] | Setting | Where | Default | | ------------------------ | ------------------------------- | --------------------------------- | | Provider | `VOICE_PROVIDER` | `openai` (or `google`) | | OpenAI API key | `OPENAI_API_KEY` | required for OpenAI | | Google API key | `GOOGLE_API_KEY` | required for Google | | OpenAI-compatible server | `OPENAI_BASE_URL` | `https://api.openai.com/v1` | | Port | `PORT` | `7860` | | Chat model and prompt | `OpenAILLMSettings` in `bot.ts` | the service default, see the code | | Voice | `OpenAITTSService` settings | `alloy` | ## Tracing and alerts [#tracing-and-alerts] Both are off until you set them: | Variable | What it does | | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `OTEL_EXPORTER_OTLP_ENDPOINT` | An OTLP/HTTP traces endpoint, such as `http://localhost:4318/v1/traces`. Each conversation is traced: its turns, and the speech-to-text, LLM and speech synthesis calls in them. | | `ERROR_WEBHOOK_URL` | Each pipeline error is POSTed there as JSON, with the conversation id, the processor and the error. The body has a `text` field, so a Slack-style incoming webhook takes it as is. | Alerts are sent once per distinct error, and at most five per conversation. Warnings and errors in the logs have credentials masked, so an error that quotes a URL with `?key=...` doesn't leak the key. The wiring is in the example's `src/observability.ts`. ## Reconnection [#reconnection] The browser SDK reconnects by itself. When the connection fails, or stays disconnected for five seconds, it opens a new peer connection for the same session (`restart_pc`). The bot keeps the pipeline and the conversation through it, so the call picks up where it was. This is how Pipecat behaves too. A session ends when the browser closes the connection, or when reconnecting gives up. ## Next steps [#next-steps] * [Frames](https://polvallverdu.github.io/pipedog/docs/concepts/frames.md) and [frame processors](https://polvallverdu.github.io/pipedog/docs/concepts/frame-processors.md) explain the pipeline's building blocks. * [Runtimes & compatibility](https://polvallverdu.github.io/pipedog/docs/guides/runtimes.md) lists where SmallWebRTC runs: Bun, Node and Deno, not Cloudflare Workers, which have no UDP.