# Azure OpenAI

> Streaming chat completions from OpenAI models deployed on Azure, with AzureLLMService.

Source: https://polvallverdu.github.io/pipedog/docs/services/llm/azure/



## Overview [#overview]

* **`AzureLLMService`**: OpenAI chat models deployed in Azure AI Foundry (Azure OpenAI). It
  extends [`OpenAILLMService`](https://polvallverdu.github.io/pipedog/docs/services/llm/openai.md), so streaming, function calling, token
  usage metrics and `runInference` work the same way; only the client differs.

The shape of `endpoint` picks the API surface:

* An endpoint ending in **`/openai/v1`** uses Azure's v1 API, which tracks new features and
  needs no `api-version`. This is the one to use.
* Any other endpoint (for example `https://my-resource.openai.azure.com`) goes through the dated
  API, with `api-version=2025-04-01-preview`, the last dated version Azure issued.

Both surfaces accept an API key or a Microsoft Entra ID token.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/llm/azure">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/azure/src">
    `packages/azure/src/llm.ts`.
  </Card>

  <Card title="Azure docs" href="https://learn.microsoft.com/en-us/azure/ai-foundry/openai/api-version-lifecycle">
    The v1 API and the dated API versions.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/azure
```

```ts
import { AzureLLMService } from "@pipedog/azure/llm";
```

`@pipedog/azure` depends on `@pipedog/openai` and the `openai` SDK, which it installs for you.

## Prerequisites [#prerequisites]

* An Azure OpenAI (or Azure AI Foundry) resource with a chat model deployed. The deployment
  name is what you pass as `model`.
* The resource's endpoint, and either its API key or a Microsoft Entra ID identity with access
  to it. The examples read the key from `AZURE_OPENAI_API_KEY`.

## AzureLLMService [#azurellmservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `endpoint` | `string` | yes | The Azure endpoint URL. Ending it in `/openai/v1` selects the v1 API surface. |
| `tokenProvider` | `AzureTokenProvider \| null \| undefined` | no | Async callable supplying a Microsoft Entra ID bearer token, used instead of `apiKey`. |
| `settings` | `OpenAILLMSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `runInParallel` | `boolean \| undefined` | no | Whether to run function calls in parallel or sequentially. Defaults to true. |
| `groupParallelTools` | `boolean \| undefined` | no | Whether to group parallel function calls so the LLM is triggered exactly once after all calls in the batch complete. When false, each function call result triggers the LLM independently as it arrives. Defaults to true. |
| `functionCallTimeoutSecs` | `number \| null \| undefined` | no | Optional timeout in seconds for function calls. A call that runs past it is cancelled: its handler is cancelled (it gets a `CancelledError`), the call is settled as cancelled, and inference runs so the LLM can report that it didn't complete. |
| `fileResolver` | `FileResolver \| null \| undefined` | no | Resolves file URLs in the context that this provider can't fetch itself (see `FileResolver`). Pass one configured with a `fileStorage` to resolve uploaded-file URLs (`new FileResolver({ fileStorage: runnerArgs.fileStorage })` when a runner hosts the bot), and share one instance across services that may consume the same files (e.g. services switched between mid-session) so downloads are reused. Defaults to a resolver that handles `data:` and `http(s)` URLs only. |
| `apiKey` | `string \| null \| undefined` | no | OpenAI API key. If null, the SDK reads `OPENAI_API_KEY`. |
| `baseUrl` | `string \| null \| undefined` | no | Custom base URL for the API. |
| `organization` | `string \| null \| undefined` | no | OpenAI organization ID. |
| `project` | `string \| null \| undefined` | no | OpenAI project ID. |
| `defaultHeaders` | `Record<string, string> \| null \| undefined` | no | Additional HTTP headers. |
| `serviceTier` | `string \| null \| undefined` | no | Service tier to use (e.g., "auto", "flex", "priority"). |
| `retryTimeoutSecs` | `number \| null \| undefined` | no | How long an inference may go without producing output before it is abandoned and re-issued, when `retryOnTimeout` is set. Defaults to 5.0 seconds. |
| `retryOnTimeout` | `boolean \| null \| undefined` | no | Whether to re-issue the request once if the first attempt produces no output within `retryTimeoutSecs`. The retry is unbounded. |

The other options (`serviceTier`, `retryOnTimeout`, `retryTimeoutSecs`, and the frame processor
options) are the same as [`OpenAILLMService`](https://polvallverdu.github.io/pipedog/docs/services/llm/openai/#configuration).
`baseUrl` is ignored: the endpoint sets it.

### Settings [#settings]

`AzureLLMService.Settings` (`AzureLLMSettings`) has the same fields and defaults as
[`OpenAILLMSettings`](https://polvallverdu.github.io/pipedog/docs/services/llm/openai/#settings). `model` defaults to `"gpt-4.1"`; on
Azure it must be your deployment's name.

### Usage [#usage]

```ts title="llm.ts"
import { AzureLLMService } from "@pipedog/azure/llm";

const llm = new AzureLLMService({
  apiKey: process.env.AZURE_OPENAI_API_KEY ?? "",
  // Ending the endpoint in /openai/v1 selects Azure's v1 API surface.
  endpoint: "https://my-resource.openai.azure.com/openai/v1",
  settings: new AzureLLMService.Settings({
    // On Azure the model is the deployment name.
    model: "my-gpt-4.1-deployment",
    systemInstruction: "You are a helpful assistant in a voice conversation.",
    temperature: 0.7,
  }),
});
```

With Microsoft Entra ID instead of a key. The token provider is called for every request:

```ts
// Any async function that returns a Microsoft Entra ID bearer token for the
// https://ai.azure.com/.default scope, for example
// getBearerTokenProvider(new DefaultAzureCredential(), "https://ai.azure.com/.default")
// from @azure/identity.
declare const tokenProvider: () => Promise<string>;

const withEntra = new AzureLLMService({
  tokenProvider,
  endpoint: "https://my-resource.openai.azure.com/openai/v1",
  settings: new AzureLLMService.Settings({ model: "my-gpt-4.1-deployment" }),
});
```

Through the dated API:

```ts
// Any other endpoint shape goes through the dated API (api-version 2025-04-01-preview).
const dated = new AzureLLMService({
  apiKey: process.env.AZURE_OPENAI_API_KEY ?? "",
  endpoint: "https://my-resource.openai.azure.com",
  settings: new AzureLLMService.Settings({ model: "my-gpt-4.1-deployment" }),
});
```

### Notes [#notes]

* On the v1 surface the client is the plain `OpenAI` client pointed at the endpoint, with the
  key (or the token provider) as its API key. On the dated surface it is the SDK's `AzureOpenAI`
  client, which sends the key in the `api-key` header, adds `api-version`, and routes to
  `/openai/deployments/<model>/...`.
* `apiKey` or `tokenProvider` is required; the constructor throws without either. With both,
  the token provider is used.
* Event handlers, frames and metrics are those of `OpenAILLMService`.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/azure/package.json`: ✅ means the compat smoke test passes on that runtime.

* The client is the official `openai` JS SDK over `fetch`, so the service works on Bun, Node,
  Deno and Cloudflare Workers.
* `AzureTokenProvider` is any `() => Promise<string>`. `getBearerTokenProvider()` from
  `@azure/identity` returns one; on Workers, where `@azure/identity`'s credentials don't run,
  fetch the token yourself (for example through a client-credentials request to Entra ID).
* The deprecated `model` and `apiVersion` constructor options are not ported. Use `settings`
  for the model, and an `/openai/v1` endpoint instead of a dated version.