# Sarvam AI

> SarvamLLMService streams chat completions from Sarvam AI through its OpenAI-compatible API.

Source: https://polvallverdu.github.io/pipedog/docs/services/llm/sarvam/



{/* Generated by scripts/generate-providers.ts from data/providers.ts. */}

## Overview [#overview]

Sarvam AI's models for Indian languages, over an OpenAI-compatible API (`sarvam-105b`, plus a few hosted open models).

* **`SarvamLLMService`**: an `OpenAILLMService` subclass with Sarvam AI's base URL and default
  model.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/llm/sarvam">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/sarvam/src">
    `packages/sarvam/src/llm.ts`.
  </Card>

  <Card title="Sarvam docs" href="https://docs.sarvam.ai/">
    Models and the API reference.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/sarvam
```

It depends on `@pipedog/openai`, which is installed with it.

```ts
import { SarvamLLMService } from "@pipedog/sarvam/llm";
```

## Prerequisites [#prerequisites]

* A Sarvam account and an API key from the [Sarvam dashboard](https://dashboard.sarvam.ai/).
* Pass the key as `apiKey`. It is required, and the service never falls back to
  `OPENAI_API_KEY`. The examples read it from `SARVAM_API_KEY`.

## SarvamLLMService [#sarvamllmservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `apiKey` | `string` | yes | Sarvam API key used for both OpenAI auth and Sarvam subscription header. |
| `baseUrl` | `string \| null \| undefined` | no | Sarvam OpenAI-compatible base URL. When null, resolved from the model: `/v1` for `sarvam-105b-conversations`, `/v2` for all other models. |
| `settings` | `SarvamLLMSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `defaultHeaders` | `Record<string, string> \| null \| undefined` | no | Additional HTTP headers to include in requests. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `runInParallel` | `boolean \| undefined` | no | Whether to run function calls in parallel or sequentially. Defaults to true. |
| `groupParallelTools` | `boolean \| undefined` | no | Whether to group parallel function calls so the LLM is triggered exactly once after all calls in the batch complete. When false, each function call result triggers the LLM independently as it arrives. Defaults to true. |
| `functionCallTimeoutSecs` | `number \| null \| undefined` | no | Optional timeout in seconds for function calls. A call that runs past it is cancelled: its handler is cancelled (it gets a `CancelledError`), the call is settled as cancelled, and inference runs so the LLM can report that it didn't complete. |
| `fileResolver` | `FileResolver \| null \| undefined` | no | Resolves file URLs in the context that this provider can't fetch itself (see `FileResolver`). Pass one configured with a `fileStorage` to resolve uploaded-file URLs (`new FileResolver({ fileStorage: runnerArgs.fileStorage })` when a runner hosts the bot), and share one instance across services that may consume the same files (e.g. services switched between mid-session) so downloads are reused. Defaults to a resolver that handles `data:` and `http(s)` URLs only. |
| `organization` | `string \| null \| undefined` | no | OpenAI organization ID. |
| `project` | `string \| null \| undefined` | no | OpenAI project ID. |
| `serviceTier` | `string \| null \| undefined` | no | Service tier to use (e.g., "auto", "flex", "priority"). |
| `retryTimeoutSecs` | `number \| null \| undefined` | no | How long an inference may go without producing output before it is abandoned and re-issued, when `retryOnTimeout` is set. Defaults to 5.0 seconds. |
| `retryOnTimeout` | `boolean \| null \| undefined` | no | Whether to re-issue the request once if the first attempt produces no output within `retryTimeoutSecs`. The retry is unbounded. |

The other options (`organization`, `project`, `defaultHeaders`, `serviceTier`,
`retryOnTimeout`, `retryTimeoutSecs`, function calling) are `OpenAILLMService`'s. See
[OpenAI](https://polvallverdu.github.io/pipedog/docs/services/llm/openai/#configuration).

|               |                                                                                                                                |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| Base URL      | `https://api.sarvam.ai/v2`                                                                                                     |
| Default model | `sarvam-105b`                                                                                                                  |
| Models to try | `sarvam-105b`, `sarvam-105b-conversations`, `gemma4`, `glm5.3`, `deepseekv4.1-flash`. See the provider's catalog for the rest. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new SarvamLLMService.Settings({...})`
(`SarvamLLMSettings`). They are [OpenAI's settings](https://polvallverdu.github.io/pipedog/docs/services/llm/openai/#settings), with these
defaults:

| Setting           | Type                                  | Default         | Description                                                                    |
| ----------------- | ------------------------------------- | --------------- | ------------------------------------------------------------------------------ |
| `model`           | `string`                              | `"sarvam-105b"` | Model name.                                                                    |
| `wikiGrounding`   | `boolean \| null`                     | `null`          | Sent as `wiki_grounding` when set. Only `sarvam-105b` and `gemma4` support it. |
| `reasoningEffort` | `"low" \| "medium" \| "high" \| null` | `null`          | Sent as `reasoning_effort` when set, on the models that reason.                |

Every other setting keeps OpenAI's default. Unset sampling settings are left out of the request, so
the provider's own defaults apply. `extra` carries provider-specific request fields.

### Usage [#usage]

```ts title="llm.ts"
import { LLMUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { SarvamLLMService } from "@pipedog/sarvam/llm";

const llm = new SarvamLLMService({
  apiKey: process.env.SARVAM_API_KEY ?? "",
  settings: new SarvamLLMService.Settings({
    model: "sarvam-105b",
    systemInstruction: "You are a helpful assistant in a voice conversation.",
  }),
});
```

Ground answers in Wikipedia and set the reasoning effort:

```ts
const grounded = new SarvamLLMService({
  apiKey: process.env.SARVAM_API_KEY ?? "",
  settings: new SarvamLLMService.Settings({
    model: "sarvam-105b",
    wikiGrounding: true,
    reasoningEffort: "low",
  }),
});
```

Change settings mid-call with an `LLMUpdateSettingsFrame`. The next completion uses them:

```ts
// Queue it on the worker, or push it from a processor upstream of the service.
const update = new LLMUpdateSettingsFrame({
  delta: new SarvamLLMService.Settings({ temperature: 0.2 }),
});
```

### Notes [#notes]

* Runs a completion for each `LLMContextFrame`, between an `LLMFullResponseStartFrame` and an
  `LLMFullResponseEndFrame`, and streams the reply as `LLMTextFrame`s, as
  [`OpenAILLMService`](https://polvallverdu.github.io/pipedog/docs/services/llm/openai.md) does. Function calls, token usage metrics and
  `runInference` work the same way.
* "developer" messages are sent as "user" messages, since the API doesn't support that role (`supportsDeveloperRole` is `false`).
* Only `deepseekv4.1-flash`, `gemma4`, `glm5.3`, `sarvam-105b` and `sarvam-105b-conversations` are accepted as the model. Any other raises when the service is constructed.
* When you don't pass `baseUrl`, it is chosen from the model: `/v1` for `sarvam-105b-conversations` and `/v2` for the rest.
* A context that a model can't serve (an image for a model without vision, say) fails with a non-fatal `ErrorFrame`, not an exception, so an `LLMSwitcher` can fall back to another service.
* The same package has Sarvam's [speech-to-text](https://polvallverdu.github.io/pipedog/docs/services/stt/sarvam.md) and [text-to-speech](https://polvallverdu.github.io/pipedog/docs/services/tts/sarvam.md) services.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/sarvam/package.json`: ✅ means the compat smoke test passes on that runtime.

* The client is the official `openai` JS SDK (through `@pipedog/openai`), which talks through
  `fetch`. It works on Bun, Node, Deno and Cloudflare Workers.
* The deprecated `model` constructor option is not ported. Use `settings`.
* Tested against a local OpenAI-compatible mock server, not against Sarvam AI itself.