Zoom AI

OverviewCopy Anchor Link

ZoomAISTTService streams 16 kHz PCM audio to Zoom's Scribe Live over a WebSocket. Scribe's own voice activity detection decides when the user starts and stops speaking, and each completed turn comes back as one final transcript. It is a port of Pipecat's ZoomAISTTService.

InstallationCopy Anchor Link

bun add @pipedog/zoom-ai

PrerequisitesCopy Anchor Link

• A Zoom AI Services API key or JWT, passed as apiKey (the examples read ZOOM_AI_API_KEY). It is sent on every connect and reconnect, so it must stay valid for as long as the service runs.

ZoomAISTTServiceCopy Anchor Link

ConfigurationCopy Anchor Link

The default URL is wss://api.zoom.us/v2/aiservices/scribe/live. A plaintext ws:// URL to a host other than localhost logs a warning.

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new ZoomAISTTService.Settings({...}) (ZoomAISTTSettings). Settings you leave out keep their defaults. Changing a setting reconnects with a new Scribe session.

UsageCopy Anchor Link

import { STTUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { Language } from "@pipedog/core/transcriptions/language";
import { ZoomAISTTService } from "@pipedog/zoom-ai/stt";

const stt = new ZoomAISTTService({
  apiKey: process.env.ZOOM_AI_API_KEY ?? "",
  settings: new ZoomAISTTService.Settings({
    language: Language.EN_US,
    vocabulary: ["pipedog", "Pipecat"],
  }),
});

Turn on Scribe's other features:

// Speaker labels, text polishing, and a Scribe field the settings don't cover, through `extra`.
const transcriber = new ZoomAISTTService({
  apiKey: process.env.ZOOM_AI_API_KEY ?? "",
  settings: new ZoomAISTTService.Settings({
    silenceDurationMs: 600,
    diarization: true,
    textPolishing: true,
    extra: { config: { profanity_filter: true } },
  }),
});

Change settings mid-call with an STTUpdateSettingsFrame:

// A settings change reconnects with a new Scribe session.
const update = new STTUpdateSettingsFrame({
  delta: new ZoomAISTTService.Settings({ language: Language.ES }),
});

NotesCopy Anchor Link

• Scribe's speech_started event broadcasts a ProposedUserStartedSpeakingFrame, and each completed transcript is pushed as a TranscriptionFrame followed by a ProposedUserStoppedSpeakingFrame. The service recommends ExternalUserTurnStrategies, so the user aggregator follows Scribe's turns instead of running a local VAD.
• Only final transcripts are pushed. There are no interim transcriptions, and no time to final segment (TTFS) metric, since Scribe sets the turn boundary.
• shouldInterrupt (default true) decides whether a user turn started by Scribe interrupts the bot.
• Audio at another sample rate is resampled to 16 kHz. When input stops mid-turn (the user muted, say), the service sends enough silence after a second to end the turn.
• When Zoom refuses the connection, the error carries Zoom's reason when the WebSocket error exposes the handshake response. A standard WebSocket doesn't, so the reason may be missing.

RuntimesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/zoom-ai/package.json: ✅ means the compat smoke test passes on that runtime.

The service uses the global WebSocket (through core's websocketConnect, which sends the auth headers on the handshake on Cloudflare Workers too). No Zoom SDK is needed.