OpenAI

OverviewCopy Anchor Link

OpenAILLMService streams replies from OpenAI's Chat Completions API, with function calling, token usage metrics and out-of-band inference (runInference).

• OpenAILLMService: the OpenAI chat service. It extends BaseOpenAILLMService, which builds the client and the request. Point baseUrl at any server that speaks the Chat Completions API (Ollama, vLLM, OpenRouter, and so on) to use it with that provider.

For the Responses API, see OpenAI Responses.

InstallationCopy Anchor Link

bun add @pipedog/openai

import { OpenAILLMService } from "@pipedog/openai/llm";
import { OpenAILLMSettings } from "@pipedog/openai/base_llm";

PrerequisitesCopy Anchor Link

• An OpenAI account and an API key from the OpenAI dashboard.
• Pass the key as apiKey. If you leave it out, the OpenAI SDK reads OPENAI_API_KEY. The examples read it from OPENAI_API_KEY.

OpenAILLMServiceCopy Anchor Link

ConfigurationCopy Anchor Link

The client options (apiKey, baseUrl, organization, project, defaultHeaders) come from OpenAIClientOptions:

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new OpenAILLMService.Settings({...}) (OpenAILLMSettings). Any setting you leave out keeps the default below. Fields marked "not sent" hold the OpenAI SDK's sentinel and are left out of the request, so OpenAI's own default applies.

UsageCopy Anchor Link

import { LLMUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { OpenAILLMService } from "@pipedog/openai/llm";

const llm = new OpenAILLMService({
  apiKey: process.env.OPENAI_API_KEY ?? "",
  settings: new OpenAILLMService.Settings({
    model: "gpt-4.1",
    systemInstruction: "You are a helpful assistant in a voice conversation.",
    temperature: 0.7,
  }),
});

An OpenAI-compatible server, through baseUrl:

// Any server that speaks the Chat Completions API, here a local Ollama.
const local = new OpenAILLMService({
  apiKey: "ollama",
  baseUrl: "http://localhost:11434/v1",
  settings: new OpenAILLMService.Settings({ model: "llama3.2" }),
});

Re-issue a request that produces no output in time:

const withRetry = new OpenAILLMService({
  apiKey: process.env.OPENAI_API_KEY ?? "",
  retryOnTimeout: true,
  retryTimeoutSecs: 3,
});

Change settings mid-call with an LLMUpdateSettingsFrame. The next completion uses them:

// Queue it on the worker, or push it from a processor upstream of the service.
const update = new LLMUpdateSettingsFrame({
  delta: new OpenAILLMService.Settings({ temperature: 0.2, maxCompletionTokens: 300 }),
});

Event handlersCopy Anchor Link

These come from the LLMService base class.

NotesCopy Anchor Link

• Runs a completion for each LLMContextFrame, between an LLMFullResponseStartFrame and an LLMFullResponseEndFrame, and streams the reply as LLMTextFrames. Other frames pass through.
• Requests always stream, with stream_options: { include_usage: true }, so token usage metrics are reported.
• serviceTier is sent as service_tier when set.
• retryOnTimeout is off by default. With it on, an attempt that produces nothing within retryTimeoutSecs (5 seconds by default) is abandoned and re-issued once.
• Models older than gpt-4o-mini and gpt-4o-2024-08-06 can't enforce a response schema, so runInference ignores a responseSchema for them, with a warning.
• OpenAI-compatible subclasses override createClient() to build the client their own way, and can set supportsDeveloperRole = false to send "developer" messages as "user" messages.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/openai/package.json: ✅ means the compat smoke test passes on that runtime.

• The client is the official openai JS SDK, which talks through fetch. It works on Bun, Node, Deno and Cloudflare Workers.
• Upstream builds the client with an httpx connection pool. fetch has no such object, so connection reuse is left to the runtime.
• The JS SDK has no extra_body option. Provider-specific extra_body fields are merged into the request JSON just before it is sent, as the Python client does.
• The deprecated model and params constructor options and BaseOpenAILLMService.InputParams are not ported. Use settings.
• For Azure OpenAI, use AzureLLMService. Other OpenAI-compatible providers have their own subclasses where ported (see LLM); for the rest, OpenAILLMService with baseUrl covers most of them.