Vercel AI SDK

OverviewCopy Anchor Link

AISDKLLMService runs a Vercel AI SDK language model as the pipeline's LLM service. The AI SDK gives one interface to many providers, so one service covers all of them: @ai-sdk/openai, @ai-sdk/google, @ai-sdk/anthropic, @ai-sdk/mistral, @ai-sdk/amazon-bedrock, Ollama and the other community providers, and model ID strings routed through the AI Gateway.

• AISDKLLMService: an LLMService that streams the model's reply with the AI SDK's streamText(). It behaves like pipedog's native LLM services: it consumes LLMContextFrames, pushes LLMFullResponseStartFrame, LLMTextFrames and LLMFullResponseEndFrame, runs function calls through pipedog's function calling, and reports TTFB, processing time and token usage.

This is a pipedog addition: Pipecat (Python) has no AI SDK service.

InstallationCopy Anchor Link

ai (version 7) is a peer dependency, so you pick its version. Add the provider package you use, and zod, which ai requires.

bun add @pipedog/ai-sdk ai zod @ai-sdk/google

import { AISDKLLMService } from "@pipedog/ai-sdk";

PrerequisitesCopy Anchor Link

• Whatever the provider needs. @ai-sdk/google reads GOOGLE_GENERATIVE_AI_API_KEY, or takes createGoogleGenerativeAI({ apiKey }). A model ID string uses the AI Gateway, which reads AI_GATEWAY_API_KEY.

AISDKLLMServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

AISDKLLMService.Settings (AISDKLLMSettings) holds the runtime-updatable settings. The common fields map to the AI SDK's call settings: temperature, maxTokens (maxOutputTokens), topP, topK, frequencyPenalty, presencePenalty and seed. null, the default, leaves a field to the provider.

systemInstruction is sent as the AI SDK's instructions. extra is merged into the streamText() call as is.

UsageCopy Anchor Link

import { google } from "@ai-sdk/google";
import { AISDKLLMService } from "@pipedog/ai-sdk";

const llm = new AISDKLLMService({
  model: google("gemini-3.5-flash-lite"),
  settings: new AISDKLLMService.Settings({
    systemInstruction: "You are a helpful voice assistant. Keep answers short.",
    temperature: 0.7,
    maxTokens: 1024,
  }),
});

Function callingCopy Anchor Link

Tools come from the context's ToolsSchema, as with every LLM service. Each FunctionSchema becomes an AI SDK tool with a JSON schema and no execute, so the AI SDK hands the model's calls back to pipedog, which runs the registered handler (or the schema's own handler, or a direct function), adds the result to the context and asks the model again.

import { FunctionSchema } from "@pipedog/core/adapters/llm/schemas/function_schema";
import { ToolsSchema } from "@pipedog/core/adapters/llm/schemas/tools_schema";
import { LLMContext } from "@pipedog/core/processors/aggregators/llm_context";
import type { FunctionCallParams } from "@pipedog/core/services/llm_service";

const weather = new FunctionSchema({
  name: "get_weather",
  description: "Get the current weather for a city",
  properties: { city: { type: "string" } },
  required: ["city"],
});

// pipedog runs the handler and sends the result back to the model.
llm.registerFunction("get_weather", async (params: FunctionCallParams) => {
  await params.resultCallback({ conditions: "sunny", celsius: 21 });
});

const context = new LLMContext({ tools: new ToolsSchema({ standardTools: [weather] }) });

Provider-defined tools, such as Google Search grounding or OpenAI web search, go in the tools option. They are sent on every request and run on the provider's side.

Switching modelsCopy Anchor Link

import { LLMUpdateSettingsFrame } from "@pipedog/core/frames/frames";

// With `provider`, a model ID in a settings update switches the model.
const switchable = new AISDKLLMService({
  model: google("gemini-3.5-flash-lite"),
  provider: google,
});
const update = new LLMUpdateSettingsFrame({
  delta: new AISDKLLMService.Settings({ model: "gemini-3.5-flash" }),
});

// Or switch to a model from any provider directly.
switchable.setLanguageModel(google("gemini-3.5-flash"));

Without provider, a model ID in a settings update can't be turned into a model, so it is ignored with a warning (unless the model was given as an ID string in the first place).

Provider optionsCopy Anchor Link

Provider-specific options go in providerOptions, keyed by provider, as in the AI SDK:

const thinkingOff = new AISDKLLMService({
  model: google("gemini-3.5-flash-lite"),
  settings: new AISDKLLMService.Settings({
    providerOptions: { google: { thinkingConfig: { thinkingBudget: 0 } } },
  }),
});

NotesCopy Anchor Link

• Consumes LLMContextFrame. For each response it pushes LLMFullResponseStartFrame, one LLMTextFrame per streamed text delta and LLMFullResponseEndFrame, even when the request fails.
• Reasoning is pushed as LLMThoughtStartFrame / LLMThoughtTextFrame / LLMThoughtEndFrame and kept in the context, so providers that need it back (Anthropic thinking blocks) get it.
• Provider metadata that comes with a tool call or the reply text, such as Gemini's thought signatures, is kept in the context and sent back with that output on later turns.
• An initial "system" context message becomes the instructions when the service has no systemInstruction. Later "system" and "developer" messages are sent as user messages.
• Images (image_url), files (file_url, file_base64) and audio (input_audio) in user messages become AI SDK image and file parts. File URLs are fetched by pipedog's file resolver and sent inline.
• A failed request is pushed upstream as an ErrorFrame (Error during completion: ...). The AI SDK retries failed requests itself (maxRetries, 2 by default).
• runInference() runs a one-shot generateText(). A responseSchema is passed as Output.object(), so the provider enforces it where it can.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/ai-sdk/package.json: ✅ means the compat smoke test passes on that runtime.

• An interruption cancels the response: the request's abortSignal is aborted and each stream part is a cancellation point.
• The service uses no runtime-specific APIs. Whether a provider package runs on a given runtime is up to that package; the official ones use fetch.