# Google Gemini

> Streaming Gemini chat completions with GoogleLLMService, with thinking, safety settings and Google Search grounding.

Source: https://polvallverdu.github.io/pipedog/docs/services/llm/google/



## Overview [#overview]

`GoogleLLMService` runs Gemini models through the Gemini API (Google AI Studio keys). It
converts an `LLMContext` to Gemini's `Content`/`Part` format, streams the reply and handles
function calls.

* **`GoogleLLMService`**: streaming Gemini inference with tools, thinking (`GoogleThinkingConfig`),
  safety settings, image output and search grounding.
* **`LLMSearchResponseFrame`** (`frames.ts`): pushed when a reply carries grounding metadata, for
  example from the Google Search tool.
* **`GoogleRTVIProcessor` / `GoogleRTVIObserver`** (`rtvi.ts`): an RTVI processor whose observer
  also forwards `LLMSearchResponseFrame`s to the client.

For Vertex AI, see [Google Vertex AI](https://polvallverdu.github.io/pipedog/docs/services/llm/google-vertex.md). For speech-to-speech,
see [Gemini Live](https://polvallverdu.github.io/pipedog/docs/services/s2s/gemini-live.md).

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/llm/google">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/google/src">
    `packages/google/src/llm.ts`, `frames.ts` and `rtvi.ts`.
  </Card>

  <Card title="Gemini API docs" href="https://ai.google.dev/gemini-api/docs">
    Models, thinking, safety settings and grounding.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/google
```

```ts
import { GoogleLLMService, GoogleThinkingConfig } from "@pipedog/google/llm";
import { LLMSearchResponseFrame } from "@pipedog/google/frames";
import { GoogleRTVIProcessor } from "@pipedog/google/rtvi";
```

## Prerequisites [#prerequisites]

* A Gemini API key from [Google AI Studio](https://aistudio.google.com/apikey).
* Pass the key as `apiKey`. The examples read it from `GOOGLE_API_KEY`.

## GoogleLLMService [#googlellmservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `runInParallel` | `boolean \| undefined` | no | Whether to run function calls in parallel or sequentially. Defaults to true. |
| `apiKey` | `string` | yes | Google AI API key for authentication. |
| `settings` | `GoogleLLMSettings \| null \| undefined` | no | Runtime-updatable settings for this service. |
| `tools` | `Record<string, unknown>[] \| null \| undefined` | no | List of available tools/functions. |
| `toolConfig` | `Record<string, unknown> \| null \| undefined` | no | Configuration for tool usage. |
| `httpOptions` | `HttpOptions \| null \| undefined` | no | HTTP options for the client. |
| `streamIdleTimeoutSecs` | `number \| null \| undefined` | no | Gap allowed between streamed chunks, in seconds. `null` waits forever. Default 20. |
| `retryTimeoutSecs` | `number \| null \| undefined` | no | First-chunk wait before re-issuing, in seconds. Default 5. |
| `retryOnTimeout` | `boolean \| null \| undefined` | no | Whether to re-issue the request once on a first-chunk timeout. Default false. |
| `imageDecoder` | `ImageDecoder \| null \| undefined` | no | Decodes inline images to pixels (upstream: Pillow). |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `groupParallelTools` | `boolean \| undefined` | no | Whether to group parallel function calls so the LLM is triggered exactly once after all calls in the batch complete. When false, each function call result triggers the LLM independently as it arrives. Defaults to true. |
| `functionCallTimeoutSecs` | `number \| null \| undefined` | no | Optional timeout in seconds for function calls. A call that runs past it is cancelled: its handler is cancelled (it gets a `CancelledError`), the call is settled as cancelled, and inference runs so the LLM can report that it didn't complete. |
| `fileResolver` | `FileResolver \| null \| undefined` | no | Resolves file URLs in the context that this provider can't fetch itself (see `FileResolver`). Pass one configured with a `fileStorage` to resolve uploaded-file URLs (`new FileResolver({ fileStorage: runnerArgs.fileStorage })` when a runner hosts the bot), and share one instance across services that may consume the same files (e.g. services switched between mid-session) so downloads are reused. Defaults to a resolver that handles `data:` and `http(s)` URLs only. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new GoogleLLMService.Settings({...})`. Any
setting you leave out keeps the default below. Extra Gemini `GenerateContentConfig` fields can go
in `extra`, with the Python SDK's snake_case keys (they are converted to camelCase when the
request is sent).

| Setting                      | Type                           | Default              | Description                                                                    |
| ---------------------------- | ------------------------------ | -------------------- | ------------------------------------------------------------------------------ |
| `model`                      | `string`                       | `"gemini-3.6-flash"` | Gemini model.                                                                  |
| `systemInstruction`          | `string \| null`               | `null`               | System prompt.                                                                 |
| `maxTokens`                  | `number \| null`               | `4096`               | Sent as `maxOutputTokens`.                                                     |
| `temperature`                | `number \| null`               | `null`               | Sampling temperature.                                                          |
| `topK`                       | `number \| null`               | `null`               | Top-k sampling.                                                                |
| `topP`                       | `number \| null`               | `null`               | Nucleus sampling.                                                              |
| `seed`                       | `number \| null`               | `null`               | Random seed. Gemini treats it as best effort.                                  |
| `frequencyPenalty`           | `number \| null`               | `null`               | Frequency penalty. Not every Gemini model accepts it.                          |
| `presencePenalty`            | `number \| null`               | `null`               | Presence penalty. Not every Gemini model accepts it.                           |
| `thinking`                   | `GoogleThinkingConfig \| null` | `null`               | Thinking configuration. See below.                                             |
| `safetySettings`             | `SafetySetting[] \| null`      | `null`               | Content safety filters. Categories you leave out keep the Gemini API defaults. |
| `useProviderThinkingDefault` | `boolean`                      | `false`              | Send no thinking default; run at Google's default.                             |

`GoogleThinkingConfig` takes `thinkingLevel` (Gemini 3 and later: `"minimal"`, `"low"`,
`"medium"`, `"high"`), `thinkingBudget` (Gemini 2.5: a token count, `0` off, `-1` dynamic) and
`includeThoughts`. Set a level or a budget, not both.

When `thinking` is `null`, the service sends the lowest thinking setting Google documents for
the model, for the fastest replies, and logs what it sent. For example `gemini-2.5-flash` gets
`thinking_budget: 0`, `gemini-2.5-pro` gets `128`, `gemini-3.6-flash` gets
`thinking_level: "minimal"` and `gemini-3.7-flash` gets `"low"`. Models not in that table
(including aliases such as `gemini-flash-latest`) run at Google's default.

### Usage [#usage]

```ts title="llm.ts"
import { LLMUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { GoogleLLMService, GoogleThinkingConfig } from "@pipedog/google/llm";
import { GoogleRTVIProcessor } from "@pipedog/google/rtvi";

const llm = new GoogleLLMService({
  apiKey: process.env.GOOGLE_API_KEY ?? "",
  settings: new GoogleLLMService.Settings({
    model: "gemini-3.6-flash",
    systemInstruction: "You are a helpful voice assistant. Keep answers short.",
    temperature: 0.7,
  }),
});
```

Thinking, for Gemini 3 and Gemini 2.5 models:

```ts
// Gemini 3 models take a thinking level.
const gemini3 = new GoogleLLMService({
  apiKey: process.env.GOOGLE_API_KEY ?? "",
  settings: new GoogleLLMService.Settings({
    model: "gemini-3.6-flash",
    thinking: new GoogleThinkingConfig({ thinkingLevel: "low", includeThoughts: true }),
  }),
});

// Gemini 2.5 models take a token budget (0 turns thinking off).
const gemini25 = new GoogleLLMService({
  apiKey: process.env.GOOGLE_API_KEY ?? "",
  settings: new GoogleLLMService.Settings({
    model: "gemini-2.5-flash",
    thinking: new GoogleThinkingConfig({ thinkingBudget: 1024 }),
  }),
});
```

Google Search grounding, with the RTVI processor that forwards search results to the client:

```ts
// Tools in the SDK's camelCase shape are sent as given.
const grounded = new GoogleLLMService({
  apiKey: process.env.GOOGLE_API_KEY ?? "",
  tools: [{ googleSearch: {} }],
});

// Pass it to PipelineWorker as `rtviProcessor` to forward each
// LLMSearchResponseFrame to the client as a "bot-llm-search-response" message.
const rtvi = new GoogleRTVIProcessor();
```

Change settings mid-call with an `LLMUpdateSettingsFrame`:

```ts
// Queue it on the worker, or push it from a processor upstream of the service.
const update = new LLMUpdateSettingsFrame({
  delta: new GoogleLLMService.Settings({ temperature: 0.2, maxTokens: 1024 }),
});
```

### Event handlers [#event-handlers]

Inherited from `LLMService`.

| Event                         | Description                                                            |
| ----------------------------- | ---------------------------------------------------------------------- |
| `on_completion_timeout`       | The stream stalled past `streamIdleTimeoutSecs` (or the retry window). |
| `on_function_calls_started`   | Function calls were received and are about to run.                     |
| `on_function_calls_cancelled` | In-progress function calls were cancelled.                             |

### Notes [#notes]

* Emits `LLMFullResponseStartFrame`, `LLMTextFrame`s and `LLMFullResponseEndFrame`. Thoughts
  (with `includeThoughts`) arrive whole, each as an `LLMThoughtStartFrame`,
  `LLMThoughtTextFrame` and `LLMThoughtEndFrame`.
* When a reply has grounding metadata, the service pushes an `LLMSearchResponseFrame` with the
  reply text (`searchResult`), the search entry point's `renderedContent`, and `origins`
  (site URI, title and the supported text segments with confidence scores).
  `GoogleRTVIObserver` sends it to the client as a `bot-llm-search-response` message.
* Gemini thought signatures are saved into the context (through an `LLMMessagesAppendFrame`),
  so they go back to the model on the next turn.
* Inline image output becomes an `AssistantImageRawFrame`, but only with an `imageDecoder`.
  The default decoder throws, so without one an image reply surfaces as a pushed error.
* `streamIdleTimeoutSecs` (default 20) bounds the gap between chunks, including the wait for
  the first one. With `retryOnTimeout`, a first chunk that doesn't arrive within
  `retryTimeoutSecs` (default 5) re-issues the request once.
* Newer models reject a request that ends with a model turn. For any model whose name doesn't
  start with `gemini-2.`, `gemini-3-`, `gemini-3.1-` or `gemini-pro-latest`, the service adds a
  trailing user message when needed.
* A reply that stops for a reason other than `STOP` (safety, recitation, token limit, ...)
  logs a warning with the finish reason. The text that did arrive is still pushed.
* Setting `thinkingBudget` on a Gemini 3, Gemini 4 or Gemma 4 model logs a warning: use
  `thinkingLevel` there.
* `runInference()` accepts a `responseSchema` (JSON schema) on Gemini 2.5 and later.
* An `INVALID_ARGUMENT` rejection removes a pending invalid file message from the context and is
  reported as an application error.
* Token usage includes tool-use prompt tokens in the prompt count and thinking tokens in the
  completion count.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/google/package.json`: ✅ means the compat smoke test passes on that runtime.

The client is `GoogleGenAI` from `@google/genai`, which uses `fetch`, so the service runs on
Bun, Node, Deno and Cloudflare Workers. Subclasses can override `createClient()`. Image
decoding (upstream uses Pillow) is not ported: pass an `imageDecoder` built on `sharp`,
`@jsquash/*` or similar to handle image output.

The deprecated upstream `InputParams` class and the `model`, `params` and `systemInstruction`
constructor options are not ported; use `settings`.