SambaNova

OverviewCopy Anchor Link

SambaNova Cloud serves open models with fast inference on its own hardware.

• SambaNovaLLMService: an OpenAILLMService subclass with SambaNova's base URL and default model.

InstallationCopy Anchor Link

bun add @pipedog/sambanova

It depends on @pipedog/openai, which is installed with it.

import { SambaNovaLLMService } from "@pipedog/sambanova/llm";

PrerequisitesCopy Anchor Link

• A SambaNova Cloud account and an API key from SambaNova.
• Pass the key as apiKey. It is required, and the service never falls back to OPENAI_API_KEY. The examples read it from SAMBANOVA_API_KEY.

SambaNovaLLMServiceCopy Anchor Link

ConfigurationCopy Anchor Link

The other options (organization, project, defaultHeaders, serviceTier, retryOnTimeout, retryTimeoutSecs, function calling) are OpenAILLMService's. See OpenAI.

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new SambaNovaLLMService.Settings({...}) (SambaNovaLLMSettings). They are OpenAI's settings, with these defaults:

Every other setting keeps OpenAI's default. Unset sampling settings are left out of the request, so the provider's own defaults apply. extra carries provider-specific request fields.

UsageCopy Anchor Link

import { LLMUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { SambaNovaLLMService } from "@pipedog/sambanova/llm";

const llm = new SambaNovaLLMService({
  apiKey: process.env.SAMBANOVA_API_KEY ?? "",
  settings: new SambaNovaLLMService.Settings({
    model: "Meta-Llama-3.3-70B-Instruct",
    systemInstruction: "You are a helpful assistant in a voice conversation.",
    temperature: 0.7,
  }),
});

Change settings mid-call with an LLMUpdateSettingsFrame. The next completion uses them:

// Queue it on the worker, or push it from a processor upstream of the service.
const update = new LLMUpdateSettingsFrame({
  delta: new SambaNovaLLMService.Settings({ temperature: 0.2, maxTokens: 300 }),
});

NotesCopy Anchor Link

• Runs a completion for each LLMContextFrame, between an LLMFullResponseStartFrame and an LLMFullResponseEndFrame, and streams the reply as LLMTextFrames, as OpenAILLMService does. Function calls, token usage metrics and runInference work the same way.
• "developer" messages are sent as "user" messages, since the API doesn't support that role (supportsDeveloperRole is false).
• service_tier is not sent: SambaNova doesn't support it.
• SambaNova's tool calls may come without an index. The service skips tool calls whose arguments are empty, and runs function calls only when the last one has arguments.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/sambanova/package.json: ✅ means the compat smoke test passes on that runtime.

• The client is the official openai JS SDK (through @pipedog/openai), which talks through fetch. It works on Bun, Node, Deno and Cloudflare Workers.
• The deprecated model constructor option is not ported. Use settings.
• Tested against a local OpenAI-compatible mock server, not against SambaNova itself.