Local audio

OverviewCopy Anchor Link

LocalAudioTransport records from the machine's microphone and plays the bot's voice on its speakers: the quickest way to talk to a bot while you build it, with no browser, phone or media server. It is a port of Pipecat's LocalAudioTransport. Upstream uses PyAudio (PortAudio, native). pipedog has no native addon: it spawns sox or ffmpeg and pipes raw 16-bit PCM through the child's stdin and stdout.

InstallationCopy Anchor Link

bun add @pipedog/transport-local

and one of the tools:

With neither on the PATH, setup() throws LocalAudioToolNotFoundError, with the install commands in its message. detectAudioTool() does the same lookup on its own.

import { LocalAudioTransport, LocalAudioTransportParams } from "@pipedog/transport-local";

RuntimesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/transport-local/package.json: ✅ means the compat smoke test passes on that runtime.

Processes are started with Bun.spawn, Deno.Command or node:child_process, whichever runtime you are on. Cloudflare Workers can't start processes.

UsageCopy Anchor Link

import { Pipeline } from "@pipedog/core/pipeline/pipeline";
import { LocalAudioTransport, LocalAudioTransportParams } from "@pipedog/transport-local";

const transport = new LocalAudioTransport(
  new LocalAudioTransportParams({ audioInEnabled: true, audioOutEnabled: true }),
);

const pipeline = new Pipeline([
  transport.input(),
  stt,
  userAggregator,
  llm,
  tts,
  transport.output(),
  assistantAggregator,
]);

Wear headphones: with speakers, the microphone hears the bot, and the bot interrupts itself.

ConfigurationCopy Anchor Link

LocalAudioTransportParams takes everything TransportParams does (sample rates, channels, VAD, filters, mixers), plus:

DevicesCopy Anchor Link

PortAudio's device indices (upstream's input_device_index) don't exist here. Name a device in the tool's own terms:

• sox: the device name for the platform's driver: CoreAudio on macOS, ALSA on Linux, WaveAudio on Windows. The default is the system's default device (sox -d, which is what rec and play run).
• ffmpeg: on macOS an AVFoundation index or name for input and an AudioToolbox index for output; on Linux PulseAudio source and sink names (PipeWire provides them). On Windows input needs a DirectShow name, and ffmpeg has no playback device there, so use sox.

How it worksCopy Anchor Link

• Input. A recorder process writes PCM to stdout. The transport cuts it into 20 ms InputAudioRawFrames and pushes them downstream. If the recorder dies, it pushes an error frame with the tool's stderr.
• Output. Each OutputAudioRawFrame is written to the player's stdin, and the write then waits for the audio's duration, like the other non-blocking output transports. The player stays at most a chunk ahead of real time, so interruptions cut the speech quickly.
• Both processes are stopped at cleanup and on cancel.

Differences from upstreamCopy Anchor Link

• sox or ffmpeg instead of PyAudio, and input_device_index and output_device_index become inputDevice and outputDevice.
• The recorder and player start in start(), where upstream opens the streams in setup().
• The desktop window transport, transports/local/tk, is not ported.
• Not exercised in the project's tests: a real microphone and speakers. The tests run against a fake process spawner, and the compat check against a real child process.