# Google Vertex AI

> Gemini models on Vertex AI with GoogleVertexLLMService, authenticated with a service account or Application Default Credentials.

Source: https://polvallverdu.github.io/pipedog/docs/services/llm/google-vertex/



## Overview [#overview]

`GoogleVertexLLMService` runs Gemini models through Vertex AI instead of the Gemini API. It
extends [`GoogleLLMService`](https://polvallverdu.github.io/pipedog/docs/services/llm/google.md), so streaming, tools, thinking, safety
settings and grounding work the same way. Only authentication and the endpoint differ.

* **`GoogleVertexLLMService`**: Gemini on Vertex AI, with a Google Cloud project, a location and
  service account credentials or Application Default Credentials (ADC).

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/llm/google-vertex">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/google/src/vertex">
    `packages/google/src/vertex/llm.ts`.
  </Card>

  <Card title="Vertex AI docs" href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference">
    The Vertex AI generative inference API.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/google
```

```ts
import { GoogleVertexLLMService } from "@pipedog/google/vertex/llm";
```

## Prerequisites [#prerequisites]

* A Google Cloud project with the Vertex AI API enabled.
* A service account with access to Vertex AI, as a key file or a JSON string, or ADC
  (`gcloud auth application-default login`, or the attached service account on GCP).
* The examples read `GOOGLE_CLOUD_PROJECT_ID`, and `GOOGLE_APPLICATION_CREDENTIALS` (a key file
  path) or `GOOGLE_CREDENTIALS_JSON` (the key itself).

## GoogleVertexLLMService [#googlevertexllmservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `credentials` | `string \| null \| undefined` | no | JSON string of service account credentials. |
| `credentialsPath` | `string \| null \| undefined` | no | Path to the service account JSON file. |
| `location` | `string \| undefined` | no | Location for the Vertex AI endpoint. Defaults to "global", the only location that serves the Gemini 3 series; the regional endpoints serve the 2.5 series. |
| `projectId` | `string` | yes | Google Cloud project ID. |
| `settings` | `GoogleVertexLLMSettings \| null \| undefined` | no | Runtime-updatable settings for this service. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `runInParallel` | `boolean \| undefined` | no | Whether to run function calls in parallel or sequentially. Defaults to true. |
| `groupParallelTools` | `boolean \| undefined` | no | Whether to group parallel function calls so the LLM is triggered exactly once after all calls in the batch complete. When false, each function call result triggers the LLM independently as it arrives. Defaults to true. |
| `functionCallTimeoutSecs` | `number \| null \| undefined` | no | Optional timeout in seconds for function calls. A call that runs past it is cancelled: its handler is cancelled (it gets a `CancelledError`), the call is settled as cancelled, and inference runs so the LLM can report that it didn't complete. |
| `fileResolver` | `FileResolver \| null \| undefined` | no | Resolves file URLs in the context that this provider can't fetch itself (see `FileResolver`). Pass one configured with a `fileStorage` to resolve uploaded-file URLs (`new FileResolver({ fileStorage: runnerArgs.fileStorage })` when a runner hosts the bot), and share one instance across services that may consume the same files (e.g. services switched between mid-session) so downloads are reused. Defaults to a resolver that handles `data:` and `http(s)` URLs only. |
| `tools` | `Record<string, unknown>[] \| null \| undefined` | no | List of available tools/functions. |
| `toolConfig` | `Record<string, unknown> \| null \| undefined` | no | Configuration for tool usage. |
| `httpOptions` | `HttpOptions \| null \| undefined` | no | HTTP options for the client. |
| `streamIdleTimeoutSecs` | `number \| null \| undefined` | no | Gap allowed between streamed chunks, in seconds. `null` waits forever. Default 20. |
| `retryTimeoutSecs` | `number \| null \| undefined` | no | First-chunk wait before re-issuing, in seconds. Default 5. |
| `retryOnTimeout` | `boolean \| null \| undefined` | no | Whether to re-issue the request once on a first-chunk timeout. Default false. |
| `imageDecoder` | `ImageDecoder \| null \| undefined` | no | Decodes inline images to pixels (upstream: Pillow). |

### Settings [#settings]

`GoogleVertexLLMService.Settings` has the same fields and defaults as
[`GoogleLLMService.Settings`](https://polvallverdu.github.io/pipedog/docs/services/llm/google/#settings): the default model is
`"gemini-3.6-flash"`, `maxTokens` is `4096`, and the other settings start as `null`.

### Usage [#usage]

```ts title="vertex.ts"
import { GoogleVertexLLMService } from "@pipedog/google/vertex/llm";

const llm = new GoogleVertexLLMService({
  credentialsPath: process.env.GOOGLE_APPLICATION_CREDENTIALS ?? "",
  projectId: process.env.GOOGLE_CLOUD_PROJECT_ID ?? "",
  settings: new GoogleVertexLLMService.Settings({
    model: "gemini-3.6-flash",
    systemInstruction: "You are a helpful voice assistant.",
  }),
});
```

With the service account key as a JSON string:

```ts
// A service account key as a JSON string, e.g. from a secret store.
const fromJson = new GoogleVertexLLMService({
  credentials: process.env.GOOGLE_CREDENTIALS_JSON ?? "",
  projectId: process.env.GOOGLE_CLOUD_PROJECT_ID ?? "",
});
```

With Application Default Credentials and a regional endpoint:

```ts
// No credentials option: Application Default Credentials, resolved on the
// first request. Gemini 2.5 models are served from regional endpoints.
const regional = new GoogleVertexLLMService({
  projectId: process.env.GOOGLE_CLOUD_PROJECT_ID ?? "",
  location: "us-east4",
  settings: new GoogleVertexLLMService.Settings({ model: "gemini-2.5-flash" }),
});
```

### Authentication [#authentication]

The service picks credentials in this order:

1. `credentials`: a service account key as a JSON string. It is parsed in the constructor.
2. `credentialsPath`: the path to a key file, read by the client.
3. Neither: Application Default Credentials.

All three go to `@google/genai` as `googleAuthOptions` with the `cloud-platform` scope, and
google-auth-library inside the SDK loads the key file, finds ADC and refreshes the access token
as it expires.

<Callout type="info">
  Upstream loads the credentials and refreshes a token in the constructor, so missing ADC fails
  there with "No valid credentials provided.". Here loading a file, finding ADC and refreshing a
  token are asynchronous, so they happen on the first request, and missing credentials surface
  as an error from that request.
</Callout>

### Event handlers [#event-handlers]

The same as [`GoogleLLMService`](https://polvallverdu.github.io/pipedog/docs/services/llm/google/#event-handlers).

### Notes [#notes]

* Passing `apiKey` throws: Vertex AI takes `credentials`, `credentialsPath` or ADC.
* `projectId` is required. `location` defaults to `"global"`, the only location that serves
  the Gemini 3 series. Regional endpoints (`"us-east4"`, `"us-central1"`, ...) serve the
  Gemini 2.5 series.
* The Vertex adapter also accepts `gs://` file URLs in the context: Vertex reads them through
  its own IAM.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/google/package.json`: ✅ means the compat smoke test passes on that runtime.

Authentication goes through google-auth-library, which only the Node build of `@google/genai`
supports. `GoogleVertexLLMService` runs on Bun, Node and Deno, not on Cloudflare Workers. On
Workers, use [`GoogleLLMService`](https://polvallverdu.github.io/pipedog/docs/services/llm/google.md) with a Gemini API key.

The deprecated upstream `model`, `params` and `systemInstruction` constructor options are not
ported; use `settings`.