# Speechmatics

> Realtime speech-to-text with Speechmatics, with diarization and server-side turn detection.

Source: https://polvallverdu.github.io/pipedog/docs/services/stt/speechmatics/



## Overview [#overview]

`SpeechmaticsSTTService` speaks Speechmatics' Agent STT WebSocket protocol (`/v2/agent`). pipedog implements the protocol directly (`src/agent_stt.ts`) instead of depending on the Python-only `speechmatics-rt` SDK.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/stt/speechmatics">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/speechmatics/src">
    `packages/speechmatics/src`.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/speechmatics
```

## Prerequisites [#prerequisites]

* A [Speechmatics](https://www.speechmatics.com) API key, passed as `apiKey` or read from `SPEECHMATICS_API_KEY`. `baseUrl` defaults to `wss://eu2.rt.speechmatics.com/v2/agent` (or `SPEECHMATICS_RT_URL`).

## SpeechmaticsSTTService [#speechmaticssttservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `apiKey` | `string \| null \| undefined` | no | Speechmatics API key for authentication. Uses environment variable `SPEECHMATICS_API_KEY` if not provided. |
| `baseUrl` | `string \| null \| undefined` | no | Base URL for Speechmatics API. Uses environment variable `SPEECHMATICS_RT_URL` or defaults to `wss://eu2.rt.speechmatics.com/v2/agent`. |
| `encoding` | `AudioEncoding \| undefined` | no | Audio encoding format. Defaults to `AudioEncoding.PCM_S16LE`. |
| `shouldInterrupt` | `boolean \| undefined` | no | Determine whether the bot should be interrupted when Speechmatics `turnDetectionMode` is configured to detect user speech. Passed along to the user turn strategies this service recommends, which own the interruption; a user-supplied `userTurnStrategies` overrides the recommendation and this setting with it. Defaults to true. |
| `settings` | `SpeechmaticsSTTSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `audioPassthrough` | `boolean \| undefined` | no | Whether to pass audio frames downstream after processing. Defaults to true. |
| `sampleRate` | `number \| null \| undefined` | no | The sample rate for audio input. If null, will be determined from the start frame. |
| `sttTtfbTimeout` | `number \| undefined` | no | Time in seconds to wait after VAD stop before reporting TTFB. This delay allows the final transcription to arrive. Defaults to 2.0. |
| `ttfsP99Latency` | `number \| null \| undefined` | no | P99 latency from speech end to final transcript in seconds. Broadcast via STTMetadataFrame at pipeline start. |
| `keepaliveTimeout` | `number \| null \| undefined` | no | Seconds of no audio before sending silence to keep the connection alive. null disables keepalive. |
| `keepaliveInterval` | `number \| undefined` | no | Seconds between idle checks when keepalive is enabled. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new SpeechmaticsSTTService.Settings({...})`. Settings you leave out keep their defaults.

| Setting                                                                                                   | Default      | Description                                                             |
| --------------------------------------------------------------------------------------------------------- | ------------ | ----------------------------------------------------------------------- |
| `model`                                                                                                   | `"linden-1"` | Speechmatics model.                                                     |
| `language`, `domain`                                                                                      |              | Language and domain.                                                    |
| `turnDetectionMode`                                                                                       | `EXTERNAL`   | `VAD` hands turn detection to the server.                               |
| `speakerActiveFormat`                                                                                     | `"{text}"`   | Template with `{speaker_id}`, `{text}`, `{ts}`, `{lang}`. Applies live. |
| `enableDiarization`, `speakerSensitivity`, `maxSpeakers`, `preferCurrentSpeaker`, `knownSpeakers`         |              | Diarization.                                                            |
| `additionalVocab`, `enablePartials`, `emitSentences`, `punctuationOverrides`, `transcriptFilteringConfig` |              | Transcript shaping.                                                     |

Changing any setting other than `speakerActiveFormat` reconnects.

### Usage [#usage]

```ts title="stt.ts"
import { Language } from "@pipedog/core/transcriptions/language";
import { SpeechmaticsSTTService, TurnDetectionMode } from "@pipedog/speechmatics/stt";

const stt = new SpeechmaticsSTTService({
  apiKey: process.env.SPEECHMATICS_API_KEY ?? "",
  settings: new SpeechmaticsSTTService.Settings({
    language: Language.EN,
    turnDetectionMode: TurnDetectionMode.EXTERNAL,
    enableDiarization: true,
    speakerActiveFormat: "@{speaker_id}: {text}",
  }),
});
```

### Notes [#notes]

* Event: `on_speakers_result`.
* A refused handshake status is hidden by the standard `WebSocket` on Bun, Node and Deno, so an invalid key is retried 3 times and then reported as a permanent failure.

## Runtimes [#runtimes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/speechmatics/package.json`: ✅ means the compat smoke test passes on that runtime.

The service uses `fetch` and the global `WebSocket` (through core's `websocketConnect`, which sends the auth header on the handshake on Cloudflare Workers too). Pass no SDK; nothing is Node-only.