# Whisper
> Speech-to-text with Whisper. WhisperSTTService and WhisperSTTServiceMLX run Whisper locally on ONNX Runtime, and BaseWhisperSTTService is the base for Whisper-compatible APIs.
Source: https://polvallverdu.github.io/pipedog/docs/services/stt/whisper/
## Overview [#overview]
`@pipedog/whisper` is a port of Pipecat's `pipecat.services.whisper` (`base_stt`, `stt`, `utils`):
* **`WhisperSTTService`** (`@pipedog/whisper/stt`): transcribes each speech segment, cut by the
pipeline's VAD, with a Whisper model on your machine. No account, no API key. Upstream wraps
faster-whisper (CTranslate2); this one runs faster-whisper's algorithm on ONNX Runtime
(`@pipedog/whisper/faster_whisper`), with ONNX exports of the same Whisper checkpoints.
* **`WhisperSTTServiceMLX`** (`@pipedog/whisper/stt`): upstream's MLX Whisper service, for Apple
Silicon. It is **not MLX** here: it keeps upstream's class, settings, model names and filtering,
and runs the same ONNX engine, decoding like mlx_whisper. It works on any platform.
* **`BaseWhisperSTTService`** (`@pipedog/whisper/base_stt`): the base for services that call a
Whisper-style transcription API (OpenAI's `/v1/audio/transcriptions` and compatible servers).
[`OpenAISTTService`](https://polvallverdu.github.io/pipedog/docs/services/stt/openai.md) and [`GroqSTTService`](https://polvallverdu.github.io/pipedog/docs/services/stt/groq.md)
extend it.
* **Probability helpers** (`@pipedog/whisper/utils`): `extractWhisperProbability`,
`extractOpenaiGpt4oProbability` and `extractDeepgramProbability` read a confidence from a
`TranscriptionFrame`'s `result`.
Local Whisper is one of upstream's keyless local defaults for [evals](https://polvallverdu.github.io/pipedog/docs/evals.md): set
`service: whisper` to transcribe the bot's speech with it instead of Moonshine.
Pipecat's documentation for the same services.
`stt.ts`, the engine `faster_whisper.ts`, `base_stt.ts` and `utils.ts`.
The decoding algorithm the local services follow.
## Installation [#installation]
```bash
bun add @pipedog/whisper
# for the local services, also an ONNX runtime:
bun add onnxruntime-node
```
`onnxruntime-node` (native, fastest on Bun, Node and Deno) or `onnxruntime-web@1.30.0`
(WebAssembly) are optional peers of `@pipedog/onnx`. The tokenizer,
[`@huggingface/tokenizers`](https://www.npmjs.com/package/@huggingface/tokenizers) (pure JS), is a
dependency.
```ts
import { Model, WhisperSTTService, WhisperSTTServiceMLX } from "@pipedog/whisper/stt";
import { BaseWhisperSTTService } from "@pipedog/whisper/base_stt";
import { extractWhisperProbability } from "@pipedog/whisper/utils";
```
The package root re-exports `base_stt` and `utils` only. Import the local services from
`@pipedog/whisper/stt`: they pull in the ONNX runtime and the tokenizer, which the API services
don't need.
## What you need [#what-you-need]
**Local services**: the model files, downloaded once from Hugging Face when the pipeline starts,
into `$PIPEDOG_CACHE_DIR/whisper`, else `$XDG_CACHE_HOME/pipedog/whisper`, else
`~/.cache/pipedog/whisper` (one folder per repo). Input must be 16 kHz (the pipeline's input
sample rate), as upstream: segments are not resampled. And a VAD in the pipeline.
Upstream's model names map to the ONNX export of the same checkpoint:
| `Model` / name | ONNX repo | Default download (fp32 encoder, int8 decoder) | `computeType: "int8"` |
| -------------------------------- | --------------------------------------- | --------------------------------------------- | --------------------- |
| `TINY` (`tiny`) | `onnx-community/whisper-tiny` | 64 MB | 41 MB |
| `BASE` (`base`) | `onnx-community/whisper-base` | 136 MB | 77 MB |
| `SMALL` (`small`) | `onnx-community/whisper-small` | 510 MB | 249 MB |
| `MEDIUM` (`medium`) | `onnx-community/whisper-medium-ONNX` | 1.9 GB | 986 MB |
| `LARGE` (`large-v3`) | `onnx-community/whisper-large-v3-ONNX` | 3.7 GB | 1.8 GB |
| `LARGE_V3_TURBO` | `onnx-community/whisper-large-v3-turbo` | 3.0 GB | 1.1 GB |
| `DISTIL_LARGE_V2` | `distil-whisper/distil-large-v2` | 2.7 GB | 767 MB |
| `DISTIL_MEDIUM_EN` (the default) | `distil-whisper/distil-medium.en` | 1.3 GB | 402 MB |
Plus the tokenizer (2.5 MB) and configs. Every other faster-whisper name works too (`tiny.en`,
`base.en`, `small.en`, `medium.en`, `large-v1`, `large-v2`, `distil-small.en`, `distil-large-v3`,
`distil-large-v3.5`, `large-v3-turbo`, `turbo`), as does any Hugging Face repo with the same ONNX
layout (`onnx/encoder_model*.onnx`, `onnx/decoder_model_merged*.onnx`).
To control where the files come from, use `modelOptions`:
* `downloadRoot`: another cache directory.
* `localFilesOnly: true`: never download.
* A local directory with the same layout as a model name, as the `model` setting.
* `files`: every file explicitly, as bytes, paths or URLs (`config.json`,
`generation_config.json`, `preprocessor_config.json`, `tokenizer.json`, the encoder and the
decoder).
```ts
// Never download: read the files from a cache you filled beforehand.
const offline = new WhisperSTTService({
modelOptions: { downloadRoot: "/models/whisper", localFilesOnly: true },
settings: new WhisperSTTService.Settings({ model: Model.LARGE_V3_TURBO, language: Language.ES }),
});
```
`computeType` (faster-whisper's `compute_type`) picks the weights: `default` is an fp32 encoder
and an int8 (`q8`) decoder, `int8*` is `q8` for both, `float32` is fp32 for both, or pass an ONNX
dtype (`fp32`, `q8`, `int8`, `uint8`, `q4`, `bnb4`) or `modelOptions: { dtype: { encoder,
decoder } }`. The `distil-whisper/*` exports have only `fp32` and `q8`, and fp16 isn't supported.
The large fp32 models keep their weights in a separate `.onnx_data` file, which needs
`onnxruntime-node`: use `int8` with `onnxruntime-web`.
**`BaseWhisperSTTService`**: a Whisper-compatible transcription endpoint and its API key, if it
needs one (the examples read it from `WHISPER_API_KEY`), and a VAD in the pipeline.
## Runtimes [#runtimes]
| Runtime | Works | What it needs |
| ------------------ | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Bun | ✅ yes | BaseWhisperSTTService and utils: only fetch. WhisperSTTService / WhisperSTTServiceMLX (@pipedog/whisper/stt): onnxruntime-node or onnxruntime-web, a filesystem for the model cache (or the model files passed as bytes), and 16 kHz input. |
| Node.js | ✅ yes | BaseWhisperSTTService and utils: only fetch. WhisperSTTService / WhisperSTTServiceMLX (@pipedog/whisper/stt): onnxruntime-node or onnxruntime-web, a filesystem for the model cache (or the model files passed as bytes), and 16 kHz input. |
| Deno | ✅ yes | BaseWhisperSTTService and utils: only fetch. WhisperSTTService / WhisperSTTServiceMLX (@pipedog/whisper/stt): onnxruntime-node or onnxruntime-web, a filesystem for the model cache (or the model files passed as bytes), and 16 kHz input. |
| Cloudflare Workers | ✅ yes | BaseWhisperSTTService and utils: only fetch. WhisperSTTService / WhisperSTTServiceMLX (@pipedog/whisper/stt): onnxruntime-node or onnxruntime-web, a filesystem for the model cache (or the model files passed as bytes), and 16 kHz input. BaseWhisperSTTService and utils. The local services' code runs there (smoke-tested with a fake engine), but real Whisper models don't fit a Worker's memory and there is no model cache: not supported. |
Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/whisper/package.json`: ✅ means the compat smoke test passes on that runtime.
`BaseWhisperSTTService` and the probability helpers run on Cloudflare Workers. The local services
don't: Whisper models (41 MB to 3.7 GB) don't fit a Worker's 128 MB of memory or its 10 MB bundle
limit, and there is no model cache. Their code is smoke-tested there with a fake engine.
## WhisperSTTService [#whispersttservice]
### Configuration [#configuration]
| Option | Type | Required | Description |
|---|---|---|---|
| `device` | `string \| undefined` | no | The device to run inference on ('cpu', 'cuda', or 'auto'). Defaults to "auto". |
| `computeType` | `string \| undefined` | no | The compute type for inference ('default', 'int8', 'int8_float16', 'float32', ... or an ONNX dtype: 'fp32', 'q8', 'int8', 'uint8', 'q4', 'bnb4'). Defaults to "default" (fp32 encoder, int8 decoder). |
| `settings` | `WhisperSTTSettings \| null \| undefined` | no | Runtime-updatable settings. When provided, its values take precedence. |
| `modelOptions` | `Omit \| undefined` | no | Options for the ONNX engine (`WhisperModel`). Not in upstream. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `audioPassthrough` | `boolean \| undefined` | no | Whether to pass audio frames downstream after processing. Defaults to true. |
| `sampleRate` | `number \| null \| undefined` | no | The sample rate for audio input. If null, will be determined from the start frame. |
| `sttTtfbTimeout` | `number \| undefined` | no | Time in seconds to wait after VAD stop before reporting TTFB. This delay allows the final transcription to arrive. Defaults to 2.0. |
| `ttfsP99Latency` | `number \| null \| undefined` | no | P99 latency from speech end to final transcript in seconds. Broadcast via STTMetadataFrame at pipeline start. |
| `keepaliveTimeout` | `number \| null \| undefined` | no | Seconds of no audio before sending silence to keep the connection alive. null disables keepalive. |
| `keepaliveInterval` | `number \| undefined` | no | Seconds between idle checks when keepalive is enabled. |
| `trailingSilenceSecs` | `number \| undefined` | no | Seconds of silence appended to each segment before it is transcribed, so the model hears the end of speech and can finish the last word. Set to 0 to disable. Defaults to 0.5. |
### Settings [#settings]
Runtime-updatable settings, passed as `settings: new WhisperSTTService.Settings({...})`
(`WhisperSTTSettings`). Any setting you leave out keeps the default below.
| Setting | Type | Default | Description |
| --------------- | ------------------ | ------------------------ | ----------------------------------------------------------------------------------------- |
| `model` | `string` | `Model.DISTIL_MEDIUM_EN` | A `Model`, a faster-whisper name, a Hugging Face repo or a local directory. |
| `language` | `Language \| null` | `Language.EN` | Transcription language. English-only models reject other languages. |
| `noSpeechProb` | `number` | `0.4` | Segments whose no-speech probability is at or above this are dropped. |
| `hotwords` | `string \| null` | `null` | Words or phrases to bias the transcription towards, space-separated. |
| `initialPrompt` | `string \| null` | `null` | Text given to the decoder as preceding context, to steer style, punctuation and spelling. |
### Usage [#usage]
```ts title="stt.ts"
import { Model, WhisperSTTService } from "@pipedog/whisper/stt";
// Downloads distil-whisper/distil-medium.en (402 MB with int8) on first start.
const local = new WhisperSTTService({
computeType: "int8", // smaller download, faster on CPU
settings: new WhisperSTTService.Settings({
model: Model.DISTIL_MEDIUM_EN,
language: Language.EN,
hotwords: "Pipecat pipedog",
}),
});
```
The engine works on its own too, with faster-whisper's API:
```ts
import { WhisperModel } from "@pipedog/whisper/faster_whisper";
const model = new WhisperModel("tiny", { computeType: "int8" });
const audio = new Float32Array(16000); // 16 kHz mono audio in [-1, 1]
const [segments, info] = await model.transcribe(audio, { language: null });
for await (const s of segments) console.log(info.language, s.start, s.end, s.text, s.noSpeechProb);
```
### Notes [#notes]
* Emits one `TranscriptionFrame` per segment. There are no interim results.
* Decoding follows faster-whisper: the same log-mel features, prompt, `hotwords` and
`initialPrompt`, `no_speech_prob`, token suppression, timestamp rules, beam search (beam 5), the
temperature fallback with best-of-5 sampling, segment splitting and language detection. With
fp32 weights, whisper tiny gives the same text, `avg_logprob` and `no_speech_prob` (to 4
decimals) as faster-whisper on the test clip.
* The model files load in `start()`. A failed load is pushed as an error and retried on the next
segment.
* `device: "cpu"` forces the CPU provider. `cuda` can't be selected.
## WhisperSTTServiceMLX [#whispersttservicemlx]
The MLX service's class, settings, model names and filtering, on the same ONNX engine. Each
`MLXModel` selects the ONNX export of the same checkpoint (`LARGE_V3_TURBO_Q4` selects its 4-bit
`q4` weights). It decodes like mlx_whisper: greedy, at the configured temperature only, and drops
likely hallucinations. The model loads on the first segment.
### Configuration [#configuration-1]
| Option | Type | Required | Description |
|---|---|---|---|
| `settings` | `WhisperMLXSTTSettings \| null \| undefined` | no | Runtime-updatable settings. When provided, its values take precedence. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `audioPassthrough` | `boolean \| undefined` | no | Whether to pass audio frames downstream after processing. Defaults to true. |
| `sampleRate` | `number \| null \| undefined` | no | The sample rate for audio input. If null, will be determined from the start frame. |
| `sttTtfbTimeout` | `number \| undefined` | no | Time in seconds to wait after VAD stop before reporting TTFB. This delay allows the final transcription to arrive. Defaults to 2.0. |
| `ttfsP99Latency` | `number \| null \| undefined` | no | P99 latency from speech end to final transcript in seconds. Broadcast via STTMetadataFrame at pipeline start. |
| `keepaliveTimeout` | `number \| null \| undefined` | no | Seconds of no audio before sending silence to keep the connection alive. null disables keepalive. |
| `keepaliveInterval` | `number \| undefined` | no | Seconds between idle checks when keepalive is enabled. |
| `trailingSilenceSecs` | `number \| undefined` | no | Seconds of silence appended to each segment before it is transcribed, so the model hears the end of speech and can finish the last word. Set to 0 to disable. Defaults to 0.5. |
| `modelOptions` | `Omit \| undefined` | no | Options for the ONNX engine (`WhisperModel`). Not in upstream. |
### Settings [#settings-1]
Runtime-updatable settings, passed as `settings: new WhisperSTTServiceMLX.Settings({...})`
(`WhisperMLXSTTSettings`). Any setting you leave out keeps the default below.
| Setting | Type | Default | Description |
| -------------- | ------------------ | --------------- | ---------------------------------------------------------------------------------------------------------------- |
| `model` | `string` | `MLXModel.TINY` | `TINY`, `MEDIUM`, `LARGE_V3`, `LARGE_V3_TURBO`, `LARGE_V3_TURBO_Q4` or `DISTIL_LARGE_V3`, or another model name. |
| `language` | `Language \| null` | `Language.EN` | Transcription language. |
| `noSpeechProb` | `number` | `0.6` | Segments whose no-speech probability is at or above this are dropped. |
| `temperature` | `number` | `0.0` | Sampling temperature, 0 to 1. |
| `engine` | `string` | `"mlx"` | Kept from upstream; it has no effect here. |
### Usage [#usage-1]
```ts
import { MLXModel, WhisperSTTServiceMLX } from "@pipedog/whisper/stt";
// Not MLX: the same ONNX engine, with MLX Whisper's model names and greedy decoding.
const mlx = new WhisperSTTServiceMLX({
settings: new WhisperSTTServiceMLX.Settings({
model: MLXModel.LARGE_V3_TURBO_Q4,
language: Language.EN,
temperature: 0.0,
}),
});
```
## BaseWhisperSTTService [#basewhispersttservice]
### Configuration [#configuration-2]
| Option | Type | Required | Description |
|---|---|---|---|
| `apiKey` | `string \| null \| undefined` | no | Service API key. Defaults to null. |
| `baseUrl` | `string \| null \| undefined` | no | Service API base URL. Defaults to null. |
| `httpClient` | `((input: RequestInfo \| URL, init?: RequestInit) => Promise) \| null \| undefined` | no | Custom `fetch` for API requests, e.g. one with a longer request timeout. Defaults to null, which lets the SDK use the global `fetch`. |
| `includeProbMetrics` | `boolean \| undefined` | no | If true, enables probability metrics in API response. Each service implements this differently (see child classes). Defaults to false. |
| `pushEmptyTranscripts` | `boolean \| undefined` | no | If true, allow empty `TranscriptionFrame` frames to be pushed downstream instead of discarding them. This is intended for situations where VAD fires even though the user did not speak. In these cases, it is useful to know that nothing was transcribed so that the agent can resume speaking, instead of waiting longer for a transcription. Defaults to false. |
| `settings` | `BaseWhisperSTTSettings \| null \| undefined` | no | Runtime-updatable settings. When provided, its values take precedence. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `audioPassthrough` | `boolean \| undefined` | no | Whether to pass audio frames downstream after processing. Defaults to true. |
| `sampleRate` | `number \| null \| undefined` | no | The sample rate for audio input. If null, will be determined from the start frame. |
| `sttTtfbTimeout` | `number \| undefined` | no | Time in seconds to wait after VAD stop before reporting TTFB. This delay allows the final transcription to arrive. Defaults to 2.0. |
| `ttfsP99Latency` | `number \| null \| undefined` | no | P99 latency from speech end to final transcript in seconds. Broadcast via STTMetadataFrame at pipeline start. |
| `keepaliveTimeout` | `number \| null \| undefined` | no | Seconds of no audio before sending silence to keep the connection alive. null disables keepalive. |
| `keepaliveInterval` | `number \| undefined` | no | Seconds between idle checks when keepalive is enabled. |
| `trailingSilenceSecs` | `number \| undefined` | no | Seconds of silence appended to each segment before it is transcribed, so the model hears the end of speech and can finish the last word. Set to 0 to disable. Defaults to 0.5. |
### Settings [#settings-2]
Runtime-updatable settings, passed as `settings: new BaseWhisperSTTService.Settings({...})`. The
base leaves every field unset; subclasses set their own defaults (`OpenAISTTService` uses
`"gpt-transcribe"` and `Language.EN`).
| Setting | Type | Default | Description |
| ------------- | ------------------ | ------- | ----------------------------------------------------------- |
| `model` | `string \| null` | `null` | Transcription model. |
| `language` | `Language \| null` | `null` | Transcription language. |
| `prompt` | `string \| null` | `null` | Text to guide the style, or to continue a previous segment. |
| `temperature` | `number \| null` | `null` | Sampling temperature, 0 to 1. |
### Usage [#usage-2]
The base class cannot be used on its own: its `_transcribe()` throws. To point at a
Whisper-compatible endpoint, pass `baseUrl` to `OpenAISTTService`:
```ts title="whisper.ts"
import { Language } from "@pipedog/core/transcriptions/language";
import { OpenAISTTService } from "@pipedog/openai/stt";
import { BaseWhisperSTTService, type Transcription } from "@pipedog/whisper/base_stt";
// A server that speaks OpenAI's /v1/audio/transcriptions API.
const stt = new OpenAISTTService({
apiKey: process.env.WHISPER_API_KEY ?? "",
baseUrl: "http://localhost:8000/v1",
settings: new OpenAISTTService.Settings({
model: "whisper-1",
language: Language.EN,
}),
});
```
For an endpoint that needs a different request, subclass the base and implement `_transcribe()`
with the OpenAI client it built from `apiKey`, `baseUrl` and `httpClient`:
```ts
/** A provider whose Whisper-compatible endpoint needs its own request shape. */
class MyWhisperSTTService extends BaseWhisperSTTService {
// Fail at construction without a key, instead of falling back to OPENAI_API_KEY.
protected static override _requiresApiKey = true;
protected override async _transcribe(audio: Uint8Array): Promise {
return await this._client.audio.transcriptions.create({
file: new File([audio.slice()], "audio.wav", { type: "audio/wav" }),
model: String(this._settings.model),
language: String(this._settings.language),
response_format: "json",
});
}
}
const custom = new MyWhisperSTTService({
apiKey: process.env.WHISPER_API_KEY ?? "",
baseUrl: "https://whisper.example.com/v1",
settings: new MyWhisperSTTService.Settings({ model: "whisper-large-v3", language: Language.EN }),
});
```
### Notes [#notes-1]
* Emits one `TranscriptionFrame` per segment, with the API response as `result`. There are no
interim results.
* Empty transcripts are logged and dropped. With `pushEmptyTranscripts: true` they are pushed,
so the bot can move on when the VAD fired but nobody spoke.
* A transcription error is pushed as an `ErrorFrame`.
* Languages map to Whisper's codes. A language not in the map falls back to its base code (`en`
from `en-US`) with a warning.
* Set the static `_requiresApiKey` to `true` in a subclass for a provider other than OpenAI.
Without it, a missing key makes the OpenAI client fall back to `OPENAI_API_KEY` and send it to
that provider.
* The HTTP client is the `openai` JS SDK over `fetch`. `httpClient` replaces its `fetch`, for a
longer timeout for example. Upstream takes an httpx client, so its `Timeout` and connection
limits do not carry over.
## Probability helpers [#probability-helpers]
Each reads a confidence (0 to 1) from a `TranscriptionFrame`'s `result`, or returns `null`:
| Function | Reads | For |
| ------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `extractWhisperProbability` | `segments[0].avg_logprob` (verbose JSON) | `OpenAISTTService` and `GroqSTTService` with Whisper models and `includeProbMetrics: true` |
| `extractOpenaiGpt4oProbability` | the average of `logprobs` | `OpenAISTTService` with GPT transcription models and `includeProbMetrics: true` |
| `extractDeepgramProbability` | the first alternative's `confidence`, or its words' average | `DeepgramSTTService` |
```ts
import { TranscriptionFrame } from "@pipedog/core/frames/frames";
import { extractWhisperProbability } from "@pipedog/whisper/utils";
// In a frame processor after an STT service created with includeProbMetrics: true.
function logConfidence(frame: unknown): void {
if (frame instanceof TranscriptionFrame) {
const prob = extractWhisperProbability(frame);
if (prob !== null) console.log(`Transcription confidence: ${(prob * 100).toFixed(2)}%`);
}
}
```
## Differences from upstream [#differences-from-upstream]
* The local services run on ONNX exports of the same checkpoints, not CTranslate2 conversions.
int8 quantization differs from CTranslate2's, so quantized outputs differ slightly.
* `WhisperSTTServiceMLX` is not MLX (above). Its features are faster-whisper's, not mlx_whisper's
(which pads 30 s of silence before computing them).
* Model files load in `start()`; upstream loads them in the constructor, blocking. The
English-only language check still runs in the constructor for faster-whisper's names, and after
loading for custom models.
* There are no threads: segments are consumed on the event loop, and each decoder step awaits the
runtime (`onnxruntime-node` runs it on its own threads).
* Sampling at temperature above 0 uses `Math.random` (no seed).
* Not implemented (faster-whisper options the services never use): `vad_filter`,
`word_timestamps`, `hallucination_silence_threshold`, `clip_timestamps`,
`repetition_penalty`, `no_repeat_ngram_size`, the batched pipeline.
* `modelOptions` is a pipedog option.
* `extractOpenaiGpt4oProbability` reads each entry's `logprob`, since the API returns
`{ token, logprob, bytes }` objects that upstream's `sum()` can't add.
* The deprecated constructor arguments (`model`, `no_speech_prob`, `language`, `temperature`,
`prompt`) are not ported. Use `settings`.
## Licenses [#licenses]
* `@pipedog/whisper`: BSD 2-Clause. `faster_whisper.ts` follows faster-whisper (MIT, Copyright (c)
2023 SYSTRAN) and OpenAI Whisper (MIT).
* Models: OpenAI's Whisper weights and the distil-whisper models are MIT. The ONNX exports are by
the Hugging Face `onnx-community`, `Xenova` and `distil-whisper` accounts. They are downloaded at
runtime, not shipped.
* `@huggingface/tokenizers`: Apache-2.0.