Cloudflare Workers AI

OverviewCopy Anchor Link

WorkersAILLMService runs Workers AI's chat models (Llama, GLM, Kimi, Qwen, gpt-oss and others) through Workers AI's OpenAI-compatible endpoint. It is an OpenAILLMService subclass, like the other OpenAI-compatible providers: streaming, function calling and usage metrics work the same way.

This package is pipedog's own: Pipecat has no Workers AI services.

InstallationCopy Anchor Link

bun add @pipedog/cloudflare

import { WorkersAILLMService } from "@pipedog/cloudflare/llm";

PrerequisitesCopy Anchor Link

• A Cloudflare account ID and an API token with the Workers AI permission. The examples read them from CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN.
• The service needs them inside a Worker too: the AI binding answers chat requests in Workers AI's own format, not OpenAI's, so the service calls the REST endpoint with fetch, which works from a Worker like from any other runtime.

ConfigurationCopy Anchor Link

The base URL is https://api.cloudflare.com/client/v4/accounts/{accountId}/ai/v1. A REST WorkersAIClient can be passed as client instead of accountId and apiToken.

SettingsCopy Anchor Link

The OpenAI settings (model, systemInstruction, temperature, maxTokens, ...), passed as settings: new WorkersAILLMService.Settings({...}). model defaults to "@cf/meta/llama-3.3-70b-instruct-fp8-fast". Others to try: "@cf/zai-org/glm-4.7-flash", "@cf/meta/llama-4-scout-17b-16e-instruct", "@cf/openai/gpt-oss-20b".

UsageCopy Anchor Link

import { WorkersAILLMService } from "@pipedog/cloudflare/llm";
import { LLMUpdateSettingsFrame } from "@pipedog/core/frames/frames";

const llm = new WorkersAILLMService({
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID ?? "",
  apiToken: process.env.CLOUDFLARE_API_TOKEN ?? "",
  settings: new WorkersAILLMService.Settings({
    model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
    systemInstruction: "You are a helpful assistant in a voice conversation.",
    temperature: 0.7,
  }),
});

Through an AI Gateway, for caching, logs and rate limits, set baseUrl to the gateway's Workers AI URL:

// Through an AI Gateway: the gateway's Workers AI OpenAI-compatible URL.
const viaGateway = new WorkersAILLMService({
  apiToken: process.env.CLOUDFLARE_API_TOKEN ?? "",
  baseUrl: `https://gateway.ai.cloudflare.com/v1/${process.env.CLOUDFLARE_ACCOUNT_ID}/my-gateway/workers-ai/v1`,
});

Change settings mid-call with an LLMUpdateSettingsFrame:

// Queue it on the worker, or push it from a processor upstream of the service.
const update = new LLMUpdateSettingsFrame({
  delta: new WorkersAILLMService.Settings({ temperature: 0.2, maxTokens: 300 }),
});

NotesCopy Anchor Link

• Not every Workers AI model takes function calls; check the model page before giving it tools.
• The same package has Workers AI speech-to-text, text-to-speech and Smart Turn v2 (turn detection). For a whole agent on Cloudflare, see Voice agent fully on Cloudflare.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/cloudflare/package.json: ✅ means the compat smoke test passes on that runtime.

• The client is the official openai JS SDK (through @pipedog/openai), which talks through fetch. It works on Bun, Node, Deno and Cloudflare Workers.
• Tested against a local OpenAI-compatible mock server, not against Workers AI itself.