# OpenAI Responses

> OpenAI's Responses API with OpenAIResponsesLLMService (WebSocket) and OpenAIResponsesHttpLLMService (HTTP), including reasoning models.

Source: https://polvallverdu.github.io/pipedog/docs/services/llm/openai-responses/



## Overview [#overview]

The Responses API is OpenAI's newer interface for text generation, tools and reasoning models.
pipedog ports both of Pipecat's Responses services:

* **`OpenAIResponsesLLMService`**: keeps a WebSocket open to `wss://api.openai.com/v1/responses`.
  It sends only the new input items with `previous_response_id` when it can, which lowers latency
  in long, tool-heavy conversations. This is the recommended variant for voice.
* **`OpenAIResponsesHttpLLMService`**: the same service over HTTP streaming, through the OpenAI
  SDK. It sends the full context on every turn.

Both store nothing on OpenAI's side (`store: false`). For Chat Completions, see
[OpenAI](https://polvallverdu.github.io/pipedog/docs/services/llm/openai.md).

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/llm/openai-responses">
    Pipecat's documentation for the same services.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/openai/src/responses">
    `packages/openai/src/responses/llm.ts`.
  </Card>

  <Card title="OpenAI docs" href="https://platform.openai.com/docs/api-reference/responses">
    The Responses API reference.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/openai
```

```ts
import {
  OpenAIResponsesHttpLLMService,
  OpenAIResponsesLLMService,
  OpenAIResponsesReasoningConfig,
} from "@pipedog/openai/responses/llm";
```

## Prerequisites [#prerequisites]

* An OpenAI account and an API key from the [OpenAI dashboard](https://platform.openai.com/api-keys).
* Pass the key as `apiKey`. If you leave it out, both services read `OPENAI_API_KEY`. The
  examples read it from `OPENAI_API_KEY`.

## OpenAIResponsesLLMService [#openairesponsesllmservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `settings` | `OpenAIResponsesLLMSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `wsUrl` | `string \| undefined` | no | WebSocket endpoint URL. Defaults to `wss://api.openai.com/v1/responses`. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `reconnectBackoffMinWait` | `number \| undefined` | no | Minimum time, in seconds, to wait between reconnection attempts. |
| `reconnectBackoffMaxWait` | `number \| undefined` | no | Maximum time, in seconds, to wait between reconnection attempts. |
| `reconnectOnError` | `boolean \| undefined` | no | Whether to automatically reconnect on connection errors. |
| `wsCloseTimeout` | `number \| undefined` | no | Maximum time, in seconds, to wait for the peer to acknowledge the websocket closing handshake before dropping the connection. Applied to connections opened through `_websocketConnect()`. Increase it for peers that need longer to complete a graceful close. |
| `wsSendTimeout` | `number \| null \| undefined` | no | Maximum time, in seconds, a single send may wait on a peer that has stopped reading before the connection is dropped and the send throws `ConnectionClosedError`. Applied to connections opened through `_websocketConnect()`. Pass `null` to wait indefinitely. Defaults to `WS_SEND_TIMEOUT`. |
| `runInParallel` | `boolean \| undefined` | no | Whether to run function calls in parallel or sequentially. Defaults to true. |
| `groupParallelTools` | `boolean \| undefined` | no | Whether to group parallel function calls so the LLM is triggered exactly once after all calls in the batch complete. When false, each function call result triggers the LLM independently as it arrives. Defaults to true. |
| `functionCallTimeoutSecs` | `number \| null \| undefined` | no | Optional timeout in seconds for function calls. A call that runs past it is cancelled: its handler is cancelled (it gets a `CancelledError`), the call is settled as cancelled, and inference runs so the LLM can report that it didn't complete. |
| `fileResolver` | `FileResolver \| null \| undefined` | no | Resolves file URLs in the context that this provider can't fetch itself (see `FileResolver`). Pass one configured with a `fileStorage` to resolve uploaded-file URLs (`new FileResolver({ fileStorage: runnerArgs.fileStorage })` when a runner hosts the bot), and share one instance across services that may consume the same files (e.g. services switched between mid-session) so downloads are reused. Defaults to a resolver that handles `data:` and `http(s)` URLs only. |
| `apiKey` | `string \| null \| undefined` | no | OpenAI API key. |
| `baseUrl` | `string \| null \| undefined` | no | Custom base URL for the API. |
| `organization` | `string \| null \| undefined` | no | OpenAI organization ID. |
| `project` | `string \| null \| undefined` | no | OpenAI project ID. |
| `defaultHeaders` | `Record<string, string> \| null \| undefined` | no | Additional HTTP headers. |
| `serviceTier` | `string \| null \| undefined` | no | Service tier to use: "auto", "default", "flex", "scale", "fast", "priority" or "ultrafast". "fast" is OpenAI's low-latency tier, the name that replaced "priority"; both values are accepted. "ultrafast" is the access-controlled Ultrafast Processing tier, which shortens the time between output tokens on the models that offer it. |
| `retryTimeoutSecs` | `number \| null \| undefined` | no | How long an inference may go without producing output before it is abandoned and re-issued, when `retryOnTimeout` is set. Defaults to 5.0 seconds. |
| `retryOnTimeout` | `boolean \| null \| undefined` | no | Whether to re-issue the request once if the first attempt produces no output within `retryTimeoutSecs`. The retry is unbounded. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new OpenAIResponsesLLMService.Settings({...})`
(`OpenAIResponsesLLMSettings`). Both services share them. Any setting you leave out keeps the
default below. Fields marked "not sent" are left out of the request, so OpenAI's own default
applies.

| Setting                                                            | Type                                     | Default     | Description                                                   |
| ------------------------------------------------------------------ | ---------------------------------------- | ----------- | ------------------------------------------------------------- |
| `model`                                                            | `string`                                 | `"gpt-4.1"` | Model.                                                        |
| `systemInstruction`                                                | `string \| null`                         | `null`      | Sent as `instructions`.                                       |
| `temperature`                                                      | `number`                                 | not sent    | Sampling temperature.                                         |
| `topP`                                                             | `number`                                 | not sent    | Nucleus sampling.                                             |
| `maxCompletionTokens`                                              | `number`                                 | not sent    | Sent as `max_output_tokens`.                                  |
| `reasoning`                                                        | `OpenAIResponsesReasoningConfig \| null` | `null`      | Reasoning config, for reasoning models. See below.            |
| `frequencyPenalty`, `presencePenalty`, `seed`, `topK`, `maxTokens` | `number \| null`                         | `null`      | Not part of the Responses request.                            |
| `filterIncompleteUserTurns`                                        | `boolean`                                | `false`     | Filter incomplete user turns.                                 |
| `userTurnCompletionConfig`                                         | `UserTurnCompletionConfig \| null`       | `null`      | Turn completion config used with `filterIncompleteUserTurns`. |
| `extra`                                                            | `Record<string, unknown>`                | `{}`        | Extra request fields, merged in after the fields above.       |

`OpenAIResponsesReasoningConfig` (also `OpenAIResponsesLLMService.ReasoningConfig`) takes:

| Field     | Type                                                                                       | Default | Description                                        |
| --------- | ------------------------------------------------------------------------------------------ | ------- | -------------------------------------------------- |
| `effort`  | `"none" \| "minimal" \| "low" \| "medium" \| "high" \| "xhigh" \| "max" \| string \| null` | `null`  | Reasoning effort. `null` leaves it to the model.   |
| `summary` | `"auto" \| "concise" \| "detailed" \| string \| null`                                      | `null`  | Reasoning summary to return. `null` asks for none. |
| `mode`    | `"standard" \| "pro" \| string \| null`                                                    | `null`  | Reasoning mode, for models that have one.          |

### Usage [#usage]

```ts title="responses.ts"
import {
  OpenAIResponsesHttpLLMService,
  OpenAIResponsesLLMService,
  OpenAIResponsesReasoningConfig,
} from "@pipedog/openai/responses/llm";

const llm = new OpenAIResponsesLLMService({
  apiKey: process.env.OPENAI_API_KEY ?? "",
  settings: new OpenAIResponsesLLMService.Settings({
    model: "gpt-4.1",
    systemInstruction: "You are a helpful assistant in a voice conversation.",
  }),
});
```

A reasoning model, with a reasoning summary:

```ts
const reasoning = new OpenAIResponsesLLMService({
  apiKey: process.env.OPENAI_API_KEY ?? "",
  settings: new OpenAIResponsesLLMService.Settings({
    model: "gpt-5.4-mini",
    reasoning: new OpenAIResponsesReasoningConfig({ effort: "low", summary: "auto" }),
  }),
});
```

### Event handlers [#event-handlers]

| Event                         | Description                                     |
| ----------------------------- | ----------------------------------------------- |
| `on_completion_timeout`       | A completion timed out.                         |
| `on_function_calls_started`   | The model asked for one or more function calls. |
| `on_function_calls_cancelled` | Function calls were cancelled.                  |
| `on_connection_error`         | The WebSocket failed (WebSocket variant only).  |

These come from the `LLMService` and `WebsocketLLMService` base classes.

### Notes [#notes]

* Runs a response for each `LLMContextFrame`, between an `LLMFullResponseStartFrame` and an
  `LLMFullResponseEndFrame`, and streams the reply as `LLMTextFrame`s.
* `wsUrl` sets the WebSocket endpoint. `baseUrl` only affects the HTTP client the service uses
  for `runInference`.
* When the context still starts with what was sent last time, only the new items go out, with
  `previous_response_id`. After a reconnect, or when the server no longer has the previous
  response, the service sends the full context again. It also reconnects and retries when it
  hits OpenAI's 60 minute connection limit.
* With `reasoning` left `null`, the service sends the lowest effort OpenAI documents for a
  known reasoning model (for example `"none"` for `gpt-5.4` and `gpt-6.1-terra`, `"low"` for `o3`), for the fastest
  replies, and logs what it sent. Set `useProviderThinkingDefault` to keep OpenAI's default.
  `gpt-4.1`, the default model, does not reason.
* Reasoning summaries come out as `LLMThoughtStartFrame`, `LLMThoughtTextFrame` and
  `LLMThoughtEndFrame` (the assistant aggregator's `on_assistant_thought` event). The encrypted
  reasoning is kept in the context and sent back on later turns.
* `serviceTier` accepts `"auto"`, `"default"`, `"flex"`, `"scale"`, `"fast"`, `"priority"` and
  `"ultrafast"`.

## OpenAIResponsesHttpLLMService [#openairesponseshttpllmservice]

### Configuration [#configuration-1]

| Option | Type | Required | Description |
|---|---|---|---|
| `serviceTier` | `string \| null \| undefined` | no | Service tier to use: "auto", "default", "flex", "scale", "fast", "priority" or "ultrafast". "fast" is OpenAI's low-latency tier, the name that replaced "priority"; both values are accepted. "ultrafast" is the access-controlled Ultrafast Processing tier, which shortens the time between output tokens on the models that offer it. |
| `settings` | `OpenAIResponsesLLMSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `retryTimeoutSecs` | `number \| null \| undefined` | no | How long an inference may go without producing output before it is abandoned and re-issued, when `retryOnTimeout` is set. Defaults to 5.0 seconds. |
| `retryOnTimeout` | `boolean \| null \| undefined` | no | Whether to re-issue the request once if the first attempt produces no output within `retryTimeoutSecs`. The retry is unbounded. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `runInParallel` | `boolean \| undefined` | no | Whether to run function calls in parallel or sequentially. Defaults to true. |
| `groupParallelTools` | `boolean \| undefined` | no | Whether to group parallel function calls so the LLM is triggered exactly once after all calls in the batch complete. When false, each function call result triggers the LLM independently as it arrives. Defaults to true. |
| `functionCallTimeoutSecs` | `number \| null \| undefined` | no | Optional timeout in seconds for function calls. A call that runs past it is cancelled: its handler is cancelled (it gets a `CancelledError`), the call is settled as cancelled, and inference runs so the LLM can report that it didn't complete. |
| `fileResolver` | `FileResolver \| null \| undefined` | no | Resolves file URLs in the context that this provider can't fetch itself (see `FileResolver`). Pass one configured with a `fileStorage` to resolve uploaded-file URLs (`new FileResolver({ fileStorage: runnerArgs.fileStorage })` when a runner hosts the bot), and share one instance across services that may consume the same files (e.g. services switched between mid-session) so downloads are reused. Defaults to a resolver that handles `data:` and `http(s)` URLs only. |
| `apiKey` | `string \| null \| undefined` | no | OpenAI API key. |
| `baseUrl` | `string \| null \| undefined` | no | Custom base URL for the API. |
| `organization` | `string \| null \| undefined` | no | OpenAI organization ID. |
| `project` | `string \| null \| undefined` | no | OpenAI project ID. |
| `defaultHeaders` | `Record<string, string> \| null \| undefined` | no | Additional HTTP headers. |

### Settings [#settings-1]

Same as `OpenAIResponsesLLMService` above.

### Usage [#usage-1]

```ts title="responses-http.ts"
const http = new OpenAIResponsesHttpLLMService({
  apiKey: process.env.OPENAI_API_KEY ?? "",
  settings: new OpenAIResponsesHttpLLMService.Settings({
    model: "gpt-4.1",
    systemInstruction: "You are a helpful assistant.",
  }),
});
```

### Notes [#notes-1]

* Each turn sends the whole context. Over HTTP, `previous_response_id` needs `store: true`,
  which turns on OpenAI-side conversation storage, so this variant doesn't use it.
* `retryOnTimeout` and `retryTimeoutSecs` work as in [OpenAI](https://polvallverdu.github.io/pipedog/docs/services/llm/openai.md): one
  re-issue when the first attempt produces nothing in time.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/openai/package.json`: ✅ means the compat smoke test passes on that runtime.

* `OpenAIResponsesHttpLLMService` uses the `openai` JS SDK over `fetch`, which works on Bun,
  Node, Deno and Cloudflare Workers.
* `OpenAIResponsesLLMService` sends the key as an `Authorization` header on the WebSocket
  handshake. Bun, Node and Deno send it from the global `WebSocket`; on Cloudflare Workers core
  opens the socket through `fetch`.
* Upstream's mixin base `_BaseOpenAIResponsesLLMService` is a class-factory mixin here
  (`BaseOpenAIResponsesLLMServiceMixin`). `instanceof _BaseOpenAIResponsesLLMService` holds for
  both services.