# pipedog > pipedog is an unofficial TypeScript port of Pipecat, the framework for real-time voice and multimodal AI agents, for Bun, Node, Deno and Cloudflare Workers. Every page is also available as Markdown: append `.md` to its URL. All pages in one file: https://polvallverdu.github.io/pipedog/llms-full.txt - **Get started** - [Introduction](https://polvallverdu.github.io/pipedog/docs.md): pipedog is an unofficial TypeScript port of Pipecat, the framework for real-time voice and multimodal conversational AI. - [Installation](https://polvallverdu.github.io/pipedog/docs/installation.md): Add pipedog to a Bun, Node, Deno or Cloudflare Workers project. - [CLI](https://polvallverdu.github.io/pipedog/docs/cli.md): Scaffold a TypeScript voice bot with pipedog init, set up your coding agent with the docs MCP server, and run evals, with the pipedog command. - [Quickstart](https://polvallverdu.github.io/pipedog/docs/quickstart.md): Build and run your first pipedog pipeline with a custom frame processor. - [Voice bot over SmallWebRTC](https://polvallverdu.github.io/pipedog/docs/voice-bot.md): A voice assistant you talk to in the browser, with OpenAI speech-to-text, chat and text-to-speech. - [Development runner](https://polvallverdu.github.io/pipedog/docs/runner.md): Run a bot with WebRTC in the browser and phone calls over Twilio, Telnyx, Plivo or Exotel, with the same bot() entry point as Pipecat's runner. - [Evals](https://polvallverdu.github.io/pipedog/docs/evals.md): Test a bot end to end with scripted or LLM-simulated users, judged by an LLM, from YAML scenarios and the pipedog-eval CLI. - [Examples](https://polvallverdu.github.io/pipedog/docs/examples.md): Runnable voice bots in the repository, over WebRTC, over a WebSocket, on the phone, on Cloudflare Durable Objects, and mounted in each major framework. - **Core concepts** - [Frames](https://polvallverdu.github.io/pipedog/docs/concepts/frames.md): Frame categories, processing order, interruption behavior and custom frames. - [Frame processors](https://polvallverdu.github.io/pipedog/docs/concepts/frame-processors.md): Write a custom FrameProcessor - handle frames, push them on, pick a direction, set up resources and listen to events. - [Pipelines & workers](https://polvallverdu.github.io/pipedog/docs/concepts/pipelines-and-workers.md): How a Pipeline links processors, how a PipelineWorker runs it, and how a WorkerRunner runs workers. - [Turn detection](https://polvallverdu.github.io/pipedog/docs/concepts/turn-detection.md): Decide when the user has finished speaking, with local models, with VAD only, or on the provider's side. - [Turn strategies](https://polvallverdu.github.io/pipedog/docs/concepts/turn-strategies.md): The start and stop strategies that decide when a user turn begins and ends, the presets that bundle them, and how to write your own. - [User muting](https://polvallverdu.github.io/pipedog/docs/concepts/user-muting.md): Ignore the user's audio and transcripts while the bot greets them, runs a tool, or speaks, with built-in or custom mute strategies. - [Turn events & idle users](https://polvallverdu.github.io/pipedog/docs/concepts/turn-events.md): The events the context aggregators emit around each turn, prompting a user who has gone quiet, and the observers that track turns and response latency. - [Interruptions & cancellation](https://polvallverdu.github.io/pipedog/docs/concepts/interruptions-and-cancellation.md): How the bot stops when the user speaks, and how @pipedog/asyncio gives JavaScript Python's asyncio cancellation. - [TTS audio cache](https://polvallverdu.github.io/pipedog/docs/concepts/tts-audio-cache.md): Keep the audio of fixed lines (a greeting, a disclaimer) and play it without asking the TTS provider again, with TTSAudioCache and warmAudioCache(). - [Pipecat Tone](https://polvallverdu.github.io/pipedog/docs/concepts/tone.md): Read how the user meant each turn, and voice the bot's plain replies with expression tags, using Pipecat Tone's small local models. - [Typed tools](https://polvallverdu.github.io/pipedog/docs/concepts/typed-tools.md): Define LLM function calls with a Zod, Valibot or ArkType schema (Standard Schema) and get the JSON schema, inferred handler types and argument validation. - [Metrics & tracing](https://polvallverdu.github.io/pipedog/docs/concepts/metrics-and-tracing.md): Performance and usage metrics as MetricsFrames, and OpenTelemetry tracing of conversations, turns and services. - [Sentry metrics](https://polvallverdu.github.io/pipedog/docs/concepts/sentry-metrics.md): Report TTFB and processing time of any service to Sentry as transactions with SentryMetrics, using the Sentry SDK for Bun, Node, Deno or Cloudflare Workers. - **Flows** - [Flows](https://polvallverdu.github.io/pipedog/docs/flows.md): Build structured conversations from nodes, functions and actions with @pipedog/flows, a port of Pipecat Flows. - [Nodes](https://polvallverdu.github.io/pipedog/docs/flows/nodes.md): A node is one step of a flow, with a role, task messages, functions and actions. - [Functions](https://polvallverdu.github.io/pipedog/docs/flows/functions.md): Tools the LLM calls at a node, written as direct functions or explicit schemas, and how they choose the next node. - [Actions](https://polvallverdu.github.io/pipedog/docs/flows/actions.md): Run speech, hang-ups and your own code before or after the LLM responds at a node. - [Context strategies](https://polvallverdu.github.io/pipedog/docs/flows/context-strategies.md): Choose whether a node keeps, resets or summarizes the conversation history on entry. - [Flow configs](https://polvallverdu.github.io/pipedog/docs/flows/flow-configs.md): Describe a flow as YAML or JSON and join it to your handlers with Flow. - [Dynamic flows](https://polvallverdu.github.io/pipedog/docs/flows/dynamic-flows.md): Build each node at runtime from what the conversation has learned. - **Frameworks** - [Overview](https://polvallverdu.github.io/pipedog/docs/frameworks.md): Mount a pipedog voice agent in SvelteKit, Hono, Bun, Next.js, TanStack Start, Cloudflare Workers, Express, Elysia, Nuxt, Deno or Vercel. - [SvelteKit](https://polvallverdu.github.io/pipedog/docs/frameworks/sveltekit.md): Mount the pipedog WebRTC offer endpoint in a SvelteKit +server.ts route. - [Hono](https://polvallverdu.github.io/pipedog/docs/frameworks/hono.md): Mount the pipedog WebRTC offer and WebSocket endpoints in a Hono app on Bun. - [Bun](https://polvallverdu.github.io/pipedog/docs/frameworks/bun.md): Serve the pipedog WebRTC offer and WebSocket endpoints with Bun.serve. - [Next.js](https://polvallverdu.github.io/pipedog/docs/frameworks/nextjs.md): Mount the pipedog WebRTC offer endpoint in a Next.js route handler on the Node.js runtime. - [TanStack Start](https://polvallverdu.github.io/pipedog/docs/frameworks/tanstack-start.md): Mount the pipedog WebRTC offer endpoint in a TanStack Start server route. - [Cloudflare Workers](https://polvallverdu.github.io/pipedog/docs/frameworks/cloudflare-workers.md): Serve the pipedog WebSocket endpoint from a Worker, with one Durable Object per session. - [Express](https://polvallverdu.github.io/pipedog/docs/frameworks/express.md): Mount the pipedog WebRTC offer endpoint in an Express 5 app. - [Elysia](https://polvallverdu.github.io/pipedog/docs/frameworks/elysia.md): Mount the pipedog WebRTC offer endpoint in an Elysia app on Bun. - [Nuxt](https://polvallverdu.github.io/pipedog/docs/frameworks/nuxt.md): Mount the pipedog WebRTC offer endpoint in a Nuxt (Nitro) server route. - [Deno](https://polvallverdu.github.io/pipedog/docs/frameworks/deno.md): Serve the pipedog WebRTC offer and WebSocket endpoints with Deno.serve. - [Vercel](https://polvallverdu.github.io/pipedog/docs/frameworks/vercel.md): Serve the pipedog endpoints from a Bun server on Vercel Functions. - **Services** - [Supported services](https://polvallverdu.github.io/pipedog/docs/services.md): Every pipedog service by category: ported, and coming soon from Pipecat. - LLM - [LLM](https://polvallverdu.github.io/pipedog/docs/services/llm.md): Chat models that turn the conversation context into a response. - [OpenAI](https://polvallverdu.github.io/pipedog/docs/services/llm/openai.md): Streaming chat completions with OpenAILLMService, the base for every OpenAI-compatible API. - [OpenAI Responses](https://polvallverdu.github.io/pipedog/docs/services/llm/openai-responses.md): OpenAI's Responses API with OpenAIResponsesLLMService (WebSocket) and OpenAIResponsesHttpLLMService (HTTP), including reasoning models. - [Azure OpenAI](https://polvallverdu.github.io/pipedog/docs/services/llm/azure.md): Streaming chat completions from OpenAI models deployed on Azure, with AzureLLMService. - [Anthropic](https://polvallverdu.github.io/pipedog/docs/services/llm/anthropic.md): Claude models with streaming, function calling, thinking and prompt caching through AnthropicLLMService. - [Google Gemini](https://polvallverdu.github.io/pipedog/docs/services/llm/google.md): Streaming Gemini chat completions with GoogleLLMService, with thinking, safety settings and Google Search grounding. - [Google Vertex AI](https://polvallverdu.github.io/pipedog/docs/services/llm/google-vertex.md): Gemini models on Vertex AI with GoogleVertexLLMService, authenticated with a service account or Application Default Credentials. - [AWS Bedrock](https://polvallverdu.github.io/pipedog/docs/services/llm/aws.md): Foundation models on Amazon Bedrock (Claude, Nova, Llama and others) through AWSBedrockLLMService and the Converse API. - [AWS AgentCore](https://polvallverdu.github.io/pipedog/docs/services/llm/aws-agentcore.md): Run an Amazon Bedrock AgentCore agent in the pipeline, in place of an LLM, with AWSAgentCoreProcessor. - [LangChain](https://polvallverdu.github.io/pipedog/docs/services/llm/langchain.md): Run a LangChain.js runnable as the pipeline's LLM step with LangchainProcessor. - [Vercel AI SDK](https://polvallverdu.github.io/pipedog/docs/services/llm/ai-sdk.md): Run any Vercel AI SDK language model (OpenAI, Google, Anthropic, Ollama, the AI Gateway, ...) as the pipeline's LLM with AISDKLLMService. - [Cloudflare Workers AI](https://polvallverdu.github.io/pipedog/docs/services/llm/cloudflare.md): WorkersAILLMService: Workers AI chat models through Cloudflare's OpenAI-compatible endpoint. - [Perplexity](https://polvallverdu.github.io/pipedog/docs/services/llm/perplexity.md): PerplexityAgentHttpLLMService answers with web search through Perplexity's Agent API, over the OpenAI Responses protocol. - [NVIDIA](https://polvallverdu.github.io/pipedog/docs/services/llm/nvidia.md): NvidiaLLMService streams chat completions from NVIDIA NIM, and NvidiaOmniLLMService hears the user's speech with Nemotron Omni. - [Baseten](https://polvallverdu.github.io/pipedog/docs/services/llm/baseten.md): BasetenLLMService streams chat completions from Baseten through its OpenAI-compatible API. - [Cerebras](https://polvallverdu.github.io/pipedog/docs/services/llm/cerebras.md): CerebrasLLMService streams chat completions from Cerebras through its OpenAI-compatible API. - [Crusoe](https://polvallverdu.github.io/pipedog/docs/services/llm/crusoe.md): CrusoeLLMService streams chat completions from Crusoe through its OpenAI-compatible API. - [DeepSeek](https://polvallverdu.github.io/pipedog/docs/services/llm/deepseek.md): DeepSeekLLMService streams chat completions from DeepSeek through its OpenAI-compatible API. - [Fireworks AI](https://polvallverdu.github.io/pipedog/docs/services/llm/fireworks.md): FireworksLLMService streams chat completions from Fireworks AI through its OpenAI-compatible API. - [Groq](https://polvallverdu.github.io/pipedog/docs/services/llm/groq.md): GroqLLMService streams chat completions from Groq through its OpenAI-compatible API. - [Inception](https://polvallverdu.github.io/pipedog/docs/services/llm/inception.md): InceptionLLMService streams chat completions from Inception through its OpenAI-compatible API. - [Mistral](https://polvallverdu.github.io/pipedog/docs/services/llm/mistral.md): MistralLLMService streams chat completions from Mistral through its OpenAI-compatible API. - [Nebius](https://polvallverdu.github.io/pipedog/docs/services/llm/nebius.md): NebiusLLMService streams chat completions from Nebius through its OpenAI-compatible API. - [Novita AI](https://polvallverdu.github.io/pipedog/docs/services/llm/novita.md): NovitaLLMService streams chat completions from Novita AI through its OpenAI-compatible API. - [Ollama](https://polvallverdu.github.io/pipedog/docs/services/llm/ollama.md): OLLamaLLMService streams chat completions from Ollama through its OpenAI-compatible API. - [OpenRouter](https://polvallverdu.github.io/pipedog/docs/services/llm/openrouter.md): OpenRouterLLMService streams chat completions from OpenRouter through its OpenAI-compatible API. - [Qwen](https://polvallverdu.github.io/pipedog/docs/services/llm/qwen.md): QwenLLMService streams chat completions from Qwen through its OpenAI-compatible API. - [SambaNova](https://polvallverdu.github.io/pipedog/docs/services/llm/sambanova.md): SambaNovaLLMService streams chat completions from SambaNova through its OpenAI-compatible API. - [Sarvam AI](https://polvallverdu.github.io/pipedog/docs/services/llm/sarvam.md): SarvamLLMService streams chat completions from Sarvam AI through its OpenAI-compatible API. - [Smallest AI](https://polvallverdu.github.io/pipedog/docs/services/llm/smallest.md): SmallestLLMService streams chat completions from Smallest AI through its OpenAI-compatible API. - [Together AI](https://polvallverdu.github.io/pipedog/docs/services/llm/together.md): TogetherLLMService streams chat completions from Together AI through its OpenAI-compatible API. - [xAI Grok](https://polvallverdu.github.io/pipedog/docs/services/llm/grok.md): GrokLLMService streams chat completions from xAI Grok through its OpenAI-compatible API. - Speech-to-text - [Speech-to-text](https://polvallverdu.github.io/pipedog/docs/services/stt.md): Transcribe the user's speech into TranscriptionFrames. - [AssemblyAI](https://polvallverdu.github.io/pipedog/docs/services/stt/assemblyai.md): Streaming speech-to-text with AssemblyAI Universal-Streaming (v3), plus the sync HTTP service. - [Gladia](https://polvallverdu.github.io/pipedog/docs/services/stt/gladia.md): Realtime multilingual speech-to-text with Gladia. - [Speechmatics](https://polvallverdu.github.io/pipedog/docs/services/stt/speechmatics.md): Realtime speech-to-text with Speechmatics, with diarization and server-side turn detection. - [Soniox](https://polvallverdu.github.io/pipedog/docs/services/stt/soniox.md): Realtime multilingual speech-to-text with Soniox. - [Deepgram](https://polvallverdu.github.io/pipedog/docs/services/stt/deepgram.md): Streaming speech-to-text with DeepgramSTTService (Nova) and DeepgramFluxSTTService (Flux, with model-side turn detection). - [Cloudflare Workers AI](https://polvallverdu.github.io/pipedog/docs/services/stt/cloudflare.md): Speech-to-text on Workers AI: Deepgram Flux with turn detection, Deepgram Nova-3 streaming, and Whisper, over the AI binding or the REST API. - [OpenAI](https://polvallverdu.github.io/pipedog/docs/services/stt/openai.md): Speech-to-text with OpenAISTTService (segmented, Audio API) and OpenAIRealtimeSTTService (streaming, Realtime API, optional server VAD). - [Google](https://polvallverdu.github.io/pipedog/docs/services/stt/google.md): Speech-to-text with GeminiSTTService (Gemini Live transcription models) and GoogleSTTService (Google Cloud Speech-to-Text V2). - [Cartesia](https://polvallverdu.github.io/pipedog/docs/services/stt/cartesia.md): Streaming speech-to-text with CartesiaSTTService (Ink) and CartesiaTurnsSTTService (Ink-2, with server-side turn detection and eager end of turn). - [ElevenLabs](https://polvallverdu.github.io/pipedog/docs/services/stt/elevenlabs.md): Speech-to-text with ElevenLabsSTTService (segmented uploads) and ElevenLabsRealtimeSTTService (WebSocket streaming). - [AWS Transcribe](https://polvallverdu.github.io/pipedog/docs/services/stt/aws.md): Streaming speech-to-text from Amazon Transcribe with AWSTranscribeSTTService, over Transcribe's WebSocket API. - [AWS SageMaker (bidirectional streaming)](https://polvallverdu.github.io/pipedog/docs/services/stt/aws-sagemaker.md): SageMakerBidiClient, the HTTP/2 bidirectional streaming client for speech models hosted on Amazon SageMaker endpoints. - [Azure Speech](https://polvallverdu.github.io/pipedog/docs/services/stt/azure.md): Continuous speech recognition from Azure AI Speech with AzureSTTService, over the Speech service's WebSocket protocol. - [Whisper](https://polvallverdu.github.io/pipedog/docs/services/stt/whisper.md): Speech-to-text with Whisper. WhisperSTTService and WhisperSTTServiceMLX run Whisper locally on ONNX Runtime, and BaseWhisperSTTService is the base for Whisper-compatible APIs. - [Moonshine](https://polvallverdu.github.io/pipedog/docs/services/stt/moonshine.md): Local speech-to-text with MoonshineSTTService, running Moonshine's small, fast ASR models on the CPU. No API key. - [FunASR](https://polvallverdu.github.io/pipedog/docs/services/stt/funasr.md): Local multilingual speech-to-text with FunASRSTTService, running SenseVoiceSmall on ONNX Runtime. Strong on Chinese, with emotion and audio-event tags. No API key. - [Groq](https://polvallverdu.github.io/pipedog/docs/services/stt/groq.md): Speech-to-text with GroqSTTService, Whisper models served by Groq. - [Mistral](https://polvallverdu.github.io/pipedog/docs/services/stt/mistral.md): Realtime speech-to-text with MistralSTTService, Mistral's Voxtral over WebSocket. - [Together AI](https://polvallverdu.github.io/pipedog/docs/services/stt/together.md): Realtime speech-to-text with TogetherSTTService, over Together AI's WebSocket API. - [xAI](https://polvallverdu.github.io/pipedog/docs/services/stt/xai.md): Realtime speech-to-text with XAISTTService, over xAI's WebSocket API. - [Sarvam AI](https://polvallverdu.github.io/pipedog/docs/services/stt/sarvam.md): Speech-to-text for Indian languages with SarvamSTTService and SarvamRealtimeSTTService, over Sarvam's WebSocket APIs. - [Smallest AI](https://polvallverdu.github.io/pipedog/docs/services/stt/smallest.md): Realtime speech-to-text with SmallestSTTService, over Smallest AI's Pulse WebSocket API. - [Gradium](https://polvallverdu.github.io/pipedog/docs/services/stt/gradium.md): Realtime speech-to-text with GradiumSTTService, over Gradium's WebSocket API, with optional server-side turn detection. - [Meta](https://polvallverdu.github.io/pipedog/docs/services/stt/meta.md): Realtime speech-to-text with MetaSTTService, over Meta's Muse Voice WebSocket API. - [Zoom AI](https://polvallverdu.github.io/pipedog/docs/services/stt/zoom-ai.md): Streaming speech-to-text with Zoom's Scribe Live, which ends each turn with its own voice activity detection. - **Coming soon** - [NVIDIA Riva](https://polvallverdu.github.io/pipedog/docs/services/stt/nvidia.md): Speech recognition from NVIDIA Riva. Not ported to pipedog yet. - Text-to-speech - [Text-to-speech](https://polvallverdu.github.io/pipedog/docs/services/tts.md): Turn the bot's text into audio frames. - [OpenAI](https://polvallverdu.github.io/pipedog/docs/services/tts/openai.md): Text-to-speech with OpenAITTSService, streaming 24 kHz PCM from OpenAI's speech API. - [Google](https://polvallverdu.github.io/pipedog/docs/services/tts/google.md): Text-to-speech with GoogleTTSService (Cloud TTS streaming), GoogleHttpTTSService (Cloud TTS with SSML) and GeminiTTSService (Gemini TTS models). - [Deepgram](https://polvallverdu.github.io/pipedog/docs/services/tts/deepgram.md): Text-to-speech with DeepgramTTSService (Aura over WebSocket), DeepgramHttpTTSService (Aura over HTTP) and DeepgramFluxTTSService (Flux). - [Cloudflare Workers AI](https://polvallverdu.github.io/pipedog/docs/services/tts/cloudflare.md): Text-to-speech on Workers AI: Deepgram Aura streaming over a WebSocket or per sentence over HTTP, and MeloTTS, over the AI binding or the REST API. - [Speechmatics](https://polvallverdu.github.io/pipedog/docs/services/tts/speechmatics.md): Text-to-speech with Speechmatics. - [Soniox](https://polvallverdu.github.io/pipedog/docs/services/tts/soniox.md): Streaming text-to-speech with Soniox. - [Cartesia](https://polvallverdu.github.io/pipedog/docs/services/tts/cartesia.md): Text-to-speech with CartesiaTTSService (WebSocket, word timestamps) and CartesiaHttpTTSService (HTTP), configured with GenerationConfig. - [ElevenLabs](https://polvallverdu.github.io/pipedog/docs/services/tts/elevenlabs.md): Text-to-speech with ElevenLabsTTSService (multi-context WebSocket), ElevenLabsHttpTTSService (HTTP streaming) and ElevenLabsDialogueTTSService (Text to Dialogue), all with word timestamps. - [Groq](https://polvallverdu.github.io/pipedog/docs/services/tts/groq.md): Text-to-speech with GroqTTSService, over Groq's OpenAI-compatible speech API. - [Mistral](https://polvallverdu.github.io/pipedog/docs/services/tts/mistral.md): Text-to-speech with MistralTTSService, Mistral's Voxtral TTS streamed over Server-Sent Events. - [Together AI](https://polvallverdu.github.io/pipedog/docs/services/tts/together.md): Text-to-speech with TogetherTTSService, streaming over Together AI's WebSocket API. - [xAI](https://polvallverdu.github.io/pipedog/docs/services/tts/xai.md): Text-to-speech with xAI: XAITTSService over WebSocket, with word timestamps, and XAIHttpTTSService over HTTP. - [Sarvam AI](https://polvallverdu.github.io/pipedog/docs/services/tts/sarvam.md): Text-to-speech for Indian languages: SarvamTTSService over WebSocket and SarvamHttpTTSService over HTTP. - [Smallest AI](https://polvallverdu.github.io/pipedog/docs/services/tts/smallest.md): Realtime text-to-speech with SmallestTTSService, over Smallest AI's WebSocket API, with word timestamps. - [Gradium](https://polvallverdu.github.io/pipedog/docs/services/tts/gradium.md): Text-to-speech with GradiumTTSService, over Gradium's WebSocket API. - [Inworld](https://polvallverdu.github.io/pipedog/docs/services/tts/inworld.md): Text-to-speech with InworldTTSService (WebSocket) and InworldHttpTTSService (HTTP). - [AWS Polly](https://polvallverdu.github.io/pipedog/docs/services/tts/aws.md): Text-to-speech from Amazon Polly with AWSPollyTTSService, with SSML prosody, engines and lexicons. - [Azure Speech](https://polvallverdu.github.io/pipedog/docs/services/tts/azure.md): Neural voices from Azure AI Speech with AzureTTSService (streaming, with word timestamps) and AzureHttpTTSService (REST). - [Kokoro](https://polvallverdu.github.io/pipedog/docs/services/tts/kokoro.md): Local text-to-speech with KokoroTTSService, running the Kokoro-82M ONNX model on your machine with eSpeak NG phonemization. No API key. - [Piper](https://polvallverdu.github.io/pipedog/docs/services/tts/piper.md): Text-to-speech with Piper voices, run locally by PiperTTSService or through a Piper HTTP server by PiperHttpTTSService. - [Pocket TTS](https://polvallverdu.github.io/pipedog/docs/services/tts/pocket-tts.md): Local text-to-speech with PocketTTSService, running Kyutai's 100M-parameter Pocket TTS on ONNX Runtime, faster than real time on a CPU. No API key. - [Rime](https://polvallverdu.github.io/pipedog/docs/services/tts/rime.md): Text-to-speech with RimeTTSService (WebSocket, word timestamps), RimeHttpTTSService (HTTP) and RimeV1TTSService (WebSocket v1, protobuf). - [Fish Audio](https://polvallverdu.github.io/pipedog/docs/services/tts/fish.md): Text-to-speech with FishAudioTTSService, Fish Audio's MessagePack WebSocket API, with cloned-voice support. - [Hume](https://polvallverdu.github.io/pipedog/docs/services/tts/hume.md): Text-to-speech with HumeTTSService, Hume's Octave voices over HTTP streaming, with word timestamps. - [Neuphonic](https://polvallverdu.github.io/pipedog/docs/services/tts/neuphonic.md): Text-to-speech with Neuphonic: NeuphonicTTSService over WebSocket and NeuphonicHttpTTSService over server-sent events. - [Speechify](https://polvallverdu.github.io/pipedog/docs/services/tts/speechify.md): Text-to-speech with Speechify: SpeechifyHttpTTSService streams audio and word timestamps over Server-Sent Events. - [MiniMax](https://polvallverdu.github.io/pipedog/docs/services/tts/minimax.md): Text-to-speech with MiniMax's T2A API: MiniMaxHttpTTSService streams PCM audio over HTTP. - [Resemble AI](https://polvallverdu.github.io/pipedog/docs/services/tts/resembleai.md): Text-to-speech with Resemble AI: ResembleAITTSService streams audio and word timestamps over WebSocket. - [Camb.ai](https://polvallverdu.github.io/pipedog/docs/services/tts/camb.md): Text-to-speech with Camb.ai's MARS models: CambTTSService streams PCM audio over HTTP. - [Async](https://polvallverdu.github.io/pipedog/docs/services/tts/asyncai.md): Text-to-speech with Async: AsyncAITTSService over WebSocket and AsyncAIHttpTTSService over HTTP streaming. - [Bland](https://polvallverdu.github.io/pipedog/docs/services/tts/bland.md): Text-to-speech with Bland: BlandTTSService streams LLM tokens over a realtime WebSocket, and BlandHttpTTSService posts whole texts. - [AssemblyAI](https://polvallverdu.github.io/pipedog/docs/services/tts/assemblyai.md): Streaming text-to-speech with per-word timings over AssemblyAI's Streaming TTS WebSocket API. - **Coming soon** - [NVIDIA Riva](https://polvallverdu.github.io/pipedog/docs/services/tts/nvidia.md): Speech synthesis from NVIDIA Riva. Not ported to pipedog yet. - Speech-to-speech - [Speech-to-speech](https://polvallverdu.github.io/pipedog/docs/services/s2s.md): Realtime models that take audio in and give audio out, in one service. - [OpenAI Realtime](https://polvallverdu.github.io/pipedog/docs/services/s2s/openai.md): Speech-to-speech with OpenAIRealtimeLLMService, over OpenAI's Realtime API WebSocket. - [OpenAI Live](https://polvallverdu.github.io/pipedog/docs/services/s2s/openai-live.md): Full-duplex speech-to-speech with OpenAILiveLLMService, which delegates work to a backend model through ResponsesDelegation or ClientDelegation. - [Gemini Live](https://polvallverdu.github.io/pipedog/docs/services/s2s/gemini-live.md): Real-time speech-to-speech with GeminiLiveLLMService over the Gemini Live API, with server or local VAD, tools, thinking and the Gemini File API. - [Gemini Live on Vertex AI](https://polvallverdu.github.io/pipedog/docs/services/s2s/gemini-live-vertex.md): Real-time speech-to-speech with GeminiLiveVertexLLMService, the Gemini Live API through Vertex AI with service account or ADC authentication. - [Inworld Realtime](https://polvallverdu.github.io/pipedog/docs/services/s2s/inworld.md): Speech-to-speech with InworldRealtimeLLMService, over Inworld's Realtime API WebSocket. - [Ultravox](https://polvallverdu.github.io/pipedog/docs/services/s2s/ultravox.md): Speech-to-speech with UltravoxRealtimeLLMService, Ultravox's audio-native realtime model. - [xAI Grok Realtime](https://polvallverdu.github.io/pipedog/docs/services/s2s/grok.md): Speech-to-speech with GrokRealtimeLLMService, over xAI's Grok Voice Agent WebSocket. - [AWS Nova Sonic](https://polvallverdu.github.io/pipedog/docs/services/s2s/aws.md): Real-time speech-to-speech with Amazon Nova Sonic on Bedrock through AWSNovaSonicLLMService, with tools and session continuation. - [Azure Voice Live](https://polvallverdu.github.io/pipedog/docs/services/s2s/azure-voice-live.md): Speech-to-speech with AzureVoiceLiveLLMService, over Azure's Voice Live API WebSocket. - [Azure OpenAI Realtime](https://polvallverdu.github.io/pipedog/docs/services/s2s/azure-realtime.md): Speech-to-speech with OpenAI's realtime models deployed on Azure, with AzureRealtimeLLMService. - Image generation - [Image generation](https://polvallverdu.github.io/pipedog/docs/services/image.md): Generate images from text prompts. - [OpenAI](https://polvallverdu.github.io/pipedog/docs/services/image/openai.md): Image generation with OpenAIImageGenService, using OpenAI's GPT Image models. - [Google](https://polvallverdu.github.io/pipedog/docs/services/image/google.md): Text-to-image generation with GoogleImageGenService and Gemini image models. - [Azure OpenAI](https://polvallverdu.github.io/pipedog/docs/services/image/azure.md): Image generation through Azure OpenAI's REST API, with AzureImageGenServiceREST. - Video avatars - [Video avatars](https://polvallverdu.github.io/pipedog/docs/services/video.md): Animated avatars that speak the bot's audio. - [HeyGen](https://polvallverdu.github.io/pipedog/docs/services/video/heygen.md): HeyGen video avatars (LiveAvatar and Interactive Avatar) with HeyGenVideoService and HeyGenTransport. - **Coming soon** - [Simli](https://polvallverdu.github.io/pipedog/docs/services/video/simli.md): Simli's realtime lip-synced avatars. Not ported to pipedog yet. - [Tavus](https://polvallverdu.github.io/pipedog/docs/services/video/tavus.md): Tavus' realtime video avatars. Blocked in pipedog: not ported yet. - [MCP client](https://polvallverdu.github.io/pipedog/docs/services/mcp-client.md): Give the bot the tools of an MCP server (stdio, SSE or streamable HTTP) with MCPClient. - [Other upstream providers](https://polvallverdu.github.io/pipedog/docs/services/other-upstream.md): Pipecat services that aren't ported to pipedog yet and have no page of their own here. - **Audio** - [Audio filters & mixers](https://polvallverdu.github.io/pipedog/docs/audio.md): Clean up the user's audio before it reaches VAD and speech-to-text, and mix background sound into the bot's audio. - [RNNoise filter](https://polvallverdu.github.io/pipedog/docs/audio/rnnoise-filter.md): Suppress background noise in the user's audio with RNNoise, compiled to WebAssembly so it runs on Bun, Node, Deno and Cloudflare Workers. - [Soundfile mixer](https://polvallverdu.github.io/pipedog/docs/audio/soundfile-mixer.md): Mix WAV background sound into the bot's output audio with SoundfileMixer, in pure TypeScript on every runtime. - [Ducking](https://polvallverdu.github.io/pipedog/docs/audio/ducking.md): Lower the bot's voice while the user talks over it, with AudioDuckingFilter, on any transport and every runtime. - [TEN VAD](https://polvallverdu.github.io/pipedog/docs/audio/ten-vad.md): Detect the user's voice with TEN VAD's WebAssembly build, which you obtain from TEN and pass in, as an opt-in alternative to Silero. - **Coming soon** - [ai-coustics](https://polvallverdu.github.io/pipedog/docs/audio/aic-filter.md): ai-coustics speech enhancement (AICFilter) and the Quail voice activity detector (AICQuailVADAnalyzer). Deferred in pipedog: not ported yet. - [Koala](https://polvallverdu.github.io/pipedog/docs/audio/koala-filter.md): Picovoice Koala noise suppression (KoalaFilter). Deferred in pipedog: not ported yet. - [Krisp VIVA](https://polvallverdu.github.io/pipedog/docs/audio/krisp-viva-filter.md): Krisp's voice isolation filter, VAD analyzer and turn analyzer. Deferred in pipedog: not ported yet. - **Transports** - [Transports](https://polvallverdu.github.io/pipedog/docs/transports.md): Move audio, video and messages between the user and the pipeline. - [WebSocket](https://polvallverdu.github.io/pipedog/docs/transports/websocket.md): WebSocket transports with FastAPIWebsocketTransport (bring your own socket), SingleClientWebsocketServerTransport, WebsocketClientTransport and RTVIClientTransport, plus endpoint helpers for each runtime. - [Cloudflare Realtime](https://polvallverdu.github.io/pipedog/docs/transports/cloudflare-realtime.md): CloudflareRealtimeTransport connects a bot to the Cloudflare Realtime SFU over its WebSocket adapters, so users talk over WebRTC while the bot runs in a Worker, a Durable Object or any server. - [SmallWebRTC](https://polvallverdu.github.io/pipedog/docs/transports/smallwebrtc.md): Peer-to-peer WebRTC with SmallWebRTCTransport, SmallWebRTCConnection, SmallWebRTCRequestHandler and the smallWebRTCOfferHandler signalling endpoint. - [LiveKit](https://polvallverdu.github.io/pipedog/docs/transports/livekit.md): WebRTC rooms on a LiveKit server or LiveKit Cloud, with LiveKitTransport, LiveKitRESTHelper and the runner's token helpers. - [WhatsApp](https://polvallverdu.github.io/pipedog/docs/transports/whatsapp.md): WhatsApp Business voice calls with WhatsAppClient, over SmallWebRTC. Handles Meta's calling webhooks, signature checks, and accepting, rejecting and ending calls. - [Local audio](https://polvallverdu.github.io/pipedog/docs/transports/local-audio.md): The machine's microphone and speakers with LocalAudioTransport, through sox or ffmpeg, for developing a bot without a browser. No native addon. - **Coming soon** - [Daily](https://polvallverdu.github.io/pipedog/docs/transports/daily.md): WebRTC rooms on Daily, with telephony (PSTN and SIP) and recording. Blocked in pipedog: not ported yet. - [LemonSlice](https://polvallverdu.github.io/pipedog/docs/transports/lemonslice.md): LemonSlice's realtime video avatars, delivered through Daily rooms. Blocked in pipedog: not ported yet. - [MoQ](https://polvallverdu.github.io/pipedog/docs/transports/moq.md): Media over QUIC (MoQ) relays. Deferred in pipedog: not ported yet. - [Vonage Video](https://polvallverdu.github.io/pipedog/docs/transports/vonage.md): Vonage Video API sessions, through the Video Connector. Blocked in pipedog: not ported yet. - **Serializers** - [Serializers](https://polvallverdu.github.io/pipedog/docs/serializers.md): Convert frames to and from a wire format, for WebSocket and telephony transports. - [Protobuf](https://polvallverdu.github.io/pipedog/docs/serializers/protobuf.md): Binary frame serialization for WebSocket transports with ProtobufFrameSerializer, wire-compatible with Pipecat's frames.proto. - [RTVI client](https://polvallverdu.github.io/pipedog/docs/serializers/rtvi-client.md): RTVIClientSerializer lets a pipedog pipeline act as an RTVI client, turning a bot's RTVI server messages into frames. - [Telnyx](https://polvallverdu.github.io/pipedog/docs/serializers/telnyx.md): Phone calls over Telnyx media streaming with TelnyxFrameSerializer, used with FastAPIWebsocketTransport. - [Twilio](https://polvallverdu.github.io/pipedog/docs/serializers/twilio.md): Phone calls over Twilio Media Streams with TwilioFrameSerializer, used with FastAPIWebsocketTransport. - [Plivo](https://polvallverdu.github.io/pipedog/docs/serializers/plivo.md): Phone calls over Plivo audio streaming with PlivoFrameSerializer, used with FastAPIWebsocketTransport. - [Exotel](https://polvallverdu.github.io/pipedog/docs/serializers/exotel.md): Phone calls over Exotel's Voicebot stream with ExotelFrameSerializer, used with FastAPIWebsocketTransport. - [Vonage](https://polvallverdu.github.io/pipedog/docs/serializers/vonage.md): Vonage Audio Connector streams with VonageFrameSerializer, used with FastAPIWebsocketTransport. - [Genesys](https://polvallverdu.github.io/pipedog/docs/serializers/genesys.md): Genesys Cloud AudioHook sessions with GenesysAudioHookSerializer, used with FastAPIWebsocketTransport. - **Telephony** - [How phone calls connect](https://polvallverdu.github.io/pipedog/docs/telephony.md): How a phone call reaches a pipedog bot over a WebSocket media stream, and which serializer speaks each provider's protocol. - [Twilio quickstart](https://polvallverdu.github.io/pipedog/docs/telephony/twilio-quickstart.md): Run a phone bot on Twilio Media Streams from examples/twilio-phone-bot, and test it without a phone. - [Telephony extensions](https://polvallverdu.github.io/pipedog/docs/telephony/extensions.md): IVRNavigator walks a phone menu with DTMF, and VoicemailDetector tells a person from a voicemail on outbound calls. Port of pipecat.extensions. - **Deployment** - [Deployment overview](https://polvallverdu.github.io/pipedog/docs/deployment.md): Where a pipedog voice agent can run in production, and which transport each host allows. - [Node, Bun and Deno servers](https://polvallverdu.github.io/pipedog/docs/deployment/servers.md): Run a pipedog voice agent as a long-lived server process, with SmallWebRTC or WebSocket sessions. - [Bundling](https://polvallverdu.github.io/pipedog/docs/deployment/bundling.md): Ship a pipedog bot as one file with bun build, with the models and WebAssembly files inside, and know what has to stay outside. - [Voice agents on Workers and Durable Objects](https://polvallverdu.github.io/pipedog/docs/deployment/cloudflare-workers.md): Run one voice agent per session on Cloudflare Durable Objects, and serve pipedog endpoints from any framework with fetch handlers. - [Voice agent fully on Cloudflare](https://polvallverdu.github.io/pipedog/docs/deployment/cloudflare-workers-ai.md): Run the whole voice stack on Cloudflare, with Deepgram Flux speech-to-text, a Workers AI LLM and Aura text-to-speech in a Durable Object. - [Vercel](https://polvallverdu.github.io/pipedog/docs/deployment/vercel.md): Run WebSocket voice sessions on Vercel Functions with the Bun runtime, and what Vercel can't host. - **Guides** - [Coming from Pipecat (Python)](https://polvallverdu.github.io/pipedog/docs/guides/coming-from-pipecat.md): How Pipecat's Python API maps to pipedog - naming, options objects, enums, async and cancellation, frames, and what is not ported. - [Differences from upstream](https://polvallverdu.github.io/pipedog/docs/guides/differences-from-upstream.md): Where pipedog behaves differently from Pipecat (Python), and why. - [Runtimes & compatibility](https://polvallverdu.github.io/pipedog/docs/guides/runtimes.md): Running pipedog on Bun, Node, Deno and Cloudflare Workers, and which packages are verified where. - [Effect](https://polvallverdu.github.io/pipedog/docs/guides/effect.md): Use pipedog from Effect with @pipedog/effect. Run a pipeline as a scoped resource, stream its frames, feed it from a Stream, and write frame processors as Effect functions with services from Layers. - [Docs for AI tools](https://polvallverdu.github.io/pipedog/docs/guides/llms.md): Read these docs from an LLM or coding agent, with llms.txt, llms-full.txt and a Markdown version of every page. - **API reference** - [Overview](https://polvallverdu.github.io/pipedog/docs/api.md): API reference generated from the pipedog TypeScript sources. - [Frames](https://polvallverdu.github.io/pipedog/docs/api/frames.md): The Frame base class, frame categories and common frame options, from @pipedog/core/frames/frames. - [FrameProcessor](https://polvallverdu.github.io/pipedog/docs/api/frame-processor.md): FrameProcessor options, FrameProcessorSetup and the FrameProcessor class, from @pipedog/core/processors/frame_processor. - [PipelineWorker & WorkerRunner](https://polvallverdu.github.io/pipedog/docs/api/pipeline-worker.md): PipelineParams, PipelineWorker options, WorkerRunner options and runTest options. - [@pipedog/asyncio](https://polvallverdu.github.io/pipedog/docs/api/asyncio.md): Tasks, cooperative cancellation, timeouts, queues and events with Python asyncio semantics. - **Project** - [Performance](https://polvallverdu.github.io/pipedog/docs/performance.md): pipedog on Bun, Node, Deno and Cloudflare Workers against Pipecat on CPython, measured on the same machine with the same scenarios. - [Roadmap & status](https://polvallverdu.github.io/pipedog/docs/roadmap.md): What is ported from Pipecat, what isn't yet, and how to ask for the next port. - [Porting workflow](https://polvallverdu.github.io/pipedog/docs/contributing.md): How pipedog tracks upstream Pipecat, picks the next work, ports modules and tests, and moves to a new upstream commit. - [Porting status](https://polvallverdu.github.io/pipedog/docs/contributing/status.md): Progress per package and runtime compatibility, generated from porting/manifest.json.