Cloudflare Workers AI

OverviewCopy Anchor Link

Workers AI hosts speech-to-text models on Cloudflare's network. @pipedog/cloudflare has a service for each kind:

• WorkersAIFluxSTTService: Deepgram Flux (@cf/deepgram/flux), conversational STT that also detects the end of the user's turn. It replaces VAD and Smart Turn.
• WorkersAINova3STTService: Deepgram Nova-3 (@cf/deepgram/nova-3) over a WebSocket, with interim results. Use it with a VAD, as with Deepgram.
• WorkersAIWhisperSTTService: Whisper (@cf/openai/whisper-large-v3-turbo, @cf/openai/whisper, @cf/openai/whisper-tiny-en), one request per speech segment.

This package is pipedog's own: Pipecat has no Workers AI services. The Flux and Nova-3 services are the Deepgram services with a different way to open the socket, since Workers AI carries Deepgram's WebSocket protocol, so they behave and emit frames the same way.

InstallationCopy Anchor Link

bun add @pipedog/cloudflare

import { WorkersAIFluxSTTService } from "@pipedog/cloudflare/flux_stt";
import { WorkersAINova3STTService } from "@pipedog/cloudflare/stt";
import { WorkersAIWhisperSTTService } from "@pipedog/cloudflare/whisper";

PrerequisitesCopy Anchor Link

Every Workers AI service reaches the models one of two ways:

• The AI binding, inside a Worker or Durable Object. Add "ai": { "binding": "AI" } to wrangler.jsonc and pass ai: env.AI. No token is needed. The realtime models open their socket with env.AI.run(model, input, { websocket: true }).
• The REST API, anywhere: pass accountId and apiToken (a Cloudflare API token with the Workers AI permission). Requests go to https://api.cloudflare.com/client/v4/accounts/{accountId}/ai/run/{model}, and the realtime models open wss:// on the same URL, with the session parameters in the query string. The examples read them from CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN.

// Anywhere else: an account ID and an API token with Workers AI access.
const fluxRest = new WorkersAIFluxSTTService({
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID ?? "",
  apiToken: process.env.CLOUDFLARE_API_TOKEN ?? "",
});

To share one connection setup between services, build a WorkersAIClient and pass it as client:

import { WorkersAIClient } from "@pipedog/cloudflare/client";

// One client shared by several services.
const client = new WorkersAIClient({ ai: env.AI });
const fluxShared = new WorkersAIFluxSTTService({ client });

WorkersAIFluxSTTServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

WorkersAIFluxSTTService.Settings is Deepgram Flux's settings class; see Deepgram Flux settings. The ones Workers AI documents:

UsageCopy Anchor Link

import { WorkersAIFluxSTTService } from "@pipedog/cloudflare/flux_stt";

// Inside a Worker or Durable Object: the AI binding, no token.
const flux = new WorkersAIFluxSTTService({
  ai: env.AI,
  settings: new WorkersAIFluxSTTService.Settings({
    eotThreshold: 0.75,
    keyterm: ["pipedog", "Cloudflare"],
  }),
});

Flux owns turn detection: use it with no VAD analyzer, and the user aggregator follows its turn events, as with DeepgramFluxSTTService (see Turn detection).

Change settings mid-call with an STTUpdateSettingsFrame:

// Queue it on the worker, or push it from a processor upstream of the service.
const update = new STTUpdateSettingsFrame({
  delta: new WorkersAIFluxSTTService.Settings({ eotThreshold: 0.85 }),
});

Event handlersCopy Anchor Link

The same as DeepgramFluxSTTService: on_start_of_turn, on_end_of_turn, on_eager_end_of_turn, on_turn_resumed, on_update, plus on_connected, on_disconnected and on_connection_error.

NotesCopy Anchor Link

• The session parameters are the model's inputs (encoding, sample_rate, eot_threshold, eager_eot_threshold, eot_timeout_ms, keyterm, mip_opt_out, tag). The model is the URL path, not a model parameter.
• Workers AI's turn messages carry an event and no type; they are read as Deepgram's TurnInfo. Workers AI doesn't document Deepgram's Connected message, so an open socket counts as confirmed.
• Audio is 16-bit PCM (linear16) at the pipeline's input rate.

WorkersAINova3STTServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

WorkersAINova3STTService.Settings is Deepgram's settings class (see DeepgramSTTService settings), with model defaulting to "@cf/deepgram/nova-3".

UsageCopy Anchor Link

import { WorkersAINova3STTService } from "@pipedog/cloudflare/stt";

const nova = new WorkersAINova3STTService({
  ai: env.AI,
  settings: new WorkersAINova3STTService.Settings({ language: Language.EN, smartFormat: true }),
});

NotesCopy Anchor Link

• Every Deepgram parameter but the model is sent as a session parameter. Workers AI documents interim_results, endpointing, vad_events, utterance_end_ms, smart_format, punctuate, keyterm, language and more on the model page.

WorkersAIWhisperSTTServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

UsageCopy Anchor Link

import { WorkersAIWhisperModel, WorkersAIWhisperSTTService } from "@pipedog/cloudflare/whisper";

const whisper = new WorkersAIWhisperSTTService({
  ai: env.AI,
  settings: new WorkersAIWhisperSTTService.Settings({
    model: WorkersAIWhisperModel.LARGE_V3_TURBO,
    language: Language.EN,
    prompt: "A conversation about Cloudflare Workers.",
  }),
});

NotesCopy Anchor Link

• A segmented service: it needs a VAD analyzer. The audio between the VAD's start and stop is sent as a WAV file, base64 for turbo and an array of bytes for the original models.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/cloudflare/package.json: ✅ means the compat smoke test passes on that runtime.

• No dependencies beyond pipedog's own packages. HTTP goes through fetch and WebSockets through the core websocketConnect (REST) or the binding's socket (Workers).
• Tested against a local fake of Workers AI (both paths), not against Workers AI itself. A live test (packages/cloudflare/test/live_workers_ai.test.ts) runs when CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN are set.