# Groq

> Speech-to-text with GroqSTTService, Whisper models served by Groq.

Source: https://polvallverdu.github.io/pipedog/docs/services/stt/groq/



## Overview [#overview]

* **`GroqSTTService`**: a `BaseWhisperSTTService` subclass. Each utterance (segmented by VAD) is sent to Groq's OpenAI-compatible transcription endpoint, and the text comes back as a `TranscriptionFrame`.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/stt/groq">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/groq/src">
    `packages/groq/src/stt.ts`.
  </Card>

  <Card title="Groq docs" href="https://console.groq.com/docs/speech-to-text">
    Speech-to-text models and limits.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/groq
```

```ts
import { GroqSTTService } from "@pipedog/groq/stt";
```

## Prerequisites [#prerequisites]

* A Groq account and an API key from the [Groq console](https://console.groq.com/).
* Pass the key as `apiKey`. It is required, and the service never falls back to
  `OPENAI_API_KEY`. The examples read it from `GROQ_API_KEY`.
* A VAD in the pipeline, since this is a segmented service: it transcribes each utterance
  once the user stops speaking.

## GroqSTTService [#groqsttservice]

Base URL `https://api.groq.com/openai/v1`.

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `apiKey` | `string` | yes | Groq API key. |
| `baseUrl` | `string \| null \| undefined` | no | API base URL. Defaults to `"https://api.groq.com/openai/v1"`. |
| `settings` | `GroqSTTSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `audioPassthrough` | `boolean \| undefined` | no | Whether to pass audio frames downstream after processing. Defaults to true. |
| `sampleRate` | `number \| null \| undefined` | no | The sample rate for audio input. If null, will be determined from the start frame. |
| `sttTtfbTimeout` | `number \| undefined` | no | Time in seconds to wait after VAD stop before reporting TTFB. This delay allows the final transcription to arrive. Defaults to 2.0. |
| `ttfsP99Latency` | `number \| null \| undefined` | no | P99 latency from speech end to final transcript in seconds. Broadcast via STTMetadataFrame at pipeline start. |
| `keepaliveTimeout` | `number \| null \| undefined` | no | Seconds of no audio before sending silence to keep the connection alive. null disables keepalive. |
| `keepaliveInterval` | `number \| undefined` | no | Seconds between idle checks when keepalive is enabled. |
| `trailingSilenceSecs` | `number \| undefined` | no | Seconds of silence appended to each segment before it is transcribed, so the model hears the end of speech and can finish the last word. Set to 0 to disable. Defaults to 0.5. |
| `httpClient` | `{ (input: RequestInfo \| URL, init?: RequestInit): Promise<Response>; (input: string \| URL \| Request, init?: RequestInit): Promise<Response>; } \| null \| undefined` | no | Custom `fetch` for API requests, e.g. one with a longer request timeout. Defaults to null, which lets the SDK use the global `fetch`. |
| `includeProbMetrics` | `boolean \| undefined` | no | If true, enables probability metrics in API response. Each service implements this differently (see child classes). Defaults to false. |
| `pushEmptyTranscripts` | `boolean \| undefined` | no | If true, allow empty `TranscriptionFrame` frames to be pushed downstream instead of discarding them. This is intended for situations where VAD fires even though the user did not speak. In these cases, it is useful to know that nothing was transcribed so that the agent can resume speaking, instead of waiting longer for a transcription. Defaults to false. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new GroqSTTService.Settings({...})`
(`GroqSTTSettings`). Any setting you leave out keeps the default below.

| Setting       | Type               | Default                    | Description                                                 |
| ------------- | ------------------ | -------------------------- | ----------------------------------------------------------- |
| `model`       | `string`           | `"whisper-large-v3-turbo"` | Whisper model.                                              |
| `language`    | `Language \| null` | `Language.EN`              | Language of the audio.                                      |
| `prompt`      | `string \| null`   | `null`                     | Text to guide the style, or to continue a previous segment. |
| `temperature` | `number \| null`   | `null`                     | Sampling temperature, 0 to 1.                               |

### Usage [#usage]

```ts title="stt.ts"
import { STTUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { GroqSTTService } from "@pipedog/groq/stt";

const stt = new GroqSTTService({
  apiKey: process.env.GROQ_API_KEY ?? "",
  settings: new GroqSTTService.Settings({ model: "whisper-large-v3-turbo" }),
});
```

Change settings mid-call with an `STTUpdateSettingsFrame`:

```ts
// Queue it on the worker, or push it from a processor upstream of the service.
const update = new STTUpdateSettingsFrame({
  delta: new GroqSTTService.Settings({ prompt: "Pipecat, pipedog" }),
});
```

### Notes [#notes]

* Uploads each utterance as a WAV file, with `response_format: "json"` (`"verbose_json"` when `includeProbMetrics` is on, for per-segment probabilities).
* `httpClient` replaces the SDK's `fetch`, for example to set a longer request timeout.
* `pushEmptyTranscripts` pushes empty transcriptions instead of dropping them.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/groq/package.json`: ✅ means the compat smoke test passes on that runtime.

* The client is the `openai` JS SDK (through `@pipedog/whisper`) over `fetch`. It works on Bun, Node, Deno and Cloudflare Workers.
* The deprecated `model`, `language`, `prompt` and `temperature` constructor options are not ported. Use `settings`.
* Tested against fakes, not against Groq itself.