# AWS Transcribe

> Streaming speech-to-text from Amazon Transcribe with AWSTranscribeSTTService, over Transcribe's WebSocket API.

Source: https://polvallverdu.github.io/pipedog/docs/services/stt/aws/



## Overview [#overview]

`AWSTranscribeSTTService` streams the user's audio to Amazon Transcribe and emits interim and
final transcripts. It connects to Transcribe's WebSocket API with a SigV4-presigned URL and sends
audio as event-stream `AudioEvent` messages, as upstream does.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/stt/aws">
    Pipecat's documentation for the same service.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/aws/src">
    `packages/aws/src/stt.ts` and `utils.ts`.
  </Card>

  <Card title="Transcribe docs" href="https://docs.aws.amazon.com/transcribe/latest/dg/streaming-websocket.html">
    Streaming over WebSocket, languages and partial-result stabilization.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/aws
```

```ts
import { AWSTranscribeSTTService } from "@pipedog/aws/stt";
```

## Prerequisites [#prerequisites]

* An IAM identity allowed `transcribe:StartStreamTranscriptionWebSocket`.
* Keys, in this order of precedence:

  1. `awsAccessKeyId`, `apiKey` (the secret access key) and `awsSessionToken`.
  2. The `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN` environment
     variables.
  3. A `credentialsProvider` function, called for every connection, so each URL is signed with
     current keys.

  The region comes from `region`, else `AWS_REGION`, else `us-east-1`.

## AWSTranscribeSTTService [#awstranscribesttservice]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `apiKey` | `string \| null \| undefined` | no | AWS secret access key. If null, falls back to environment variables and `credentialsProvider`. |
| `awsAccessKeyId` | `string \| null \| undefined` | no | AWS access key ID. Same fallback behaviour as `apiKey`. |
| `awsSessionToken` | `string \| null \| undefined` | no | AWS session token for temporary credentials. |
| `region` | `string \| null \| undefined` | no | AWS region for the service. |
| `settings` | `AWSTranscribeSTTSettings \| null \| undefined` | no | Runtime-updatable settings. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |
| `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. |
| `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. |
| `reconnectBackoffMinWait` | `number \| undefined` | no | Minimum time, in seconds, to wait between reconnection attempts. |
| `reconnectBackoffMaxWait` | `number \| undefined` | no | Maximum time, in seconds, to wait between reconnection attempts. |
| `wsCloseTimeout` | `number \| undefined` | no | Maximum time, in seconds, to wait for the peer to acknowledge the websocket closing handshake before dropping the connection. Applied to connections opened through `_websocketConnect()`. Increase it for peers that need longer to complete a graceful close. |
| `wsSendTimeout` | `number \| null \| undefined` | no | Maximum time, in seconds, a single send may wait on a peer that has stopped reading before the connection is dropped and the send throws `ConnectionClosedError`. Applied to connections opened through `_websocketConnect()`. Pass `null` to wait indefinitely. Defaults to `WS_SEND_TIMEOUT`. |
| `audioPassthrough` | `boolean \| undefined` | no | Whether to pass audio frames downstream after processing. Defaults to true. |
| `sampleRate` | `number \| null \| undefined` | no | The sample rate for audio input. If null, will be determined from the start frame. |
| `sttTtfbTimeout` | `number \| undefined` | no | Time in seconds to wait after VAD stop before reporting TTFB. This delay allows the final transcription to arrive. Defaults to 2.0. |
| `ttfsP99Latency` | `number \| null \| undefined` | no | P99 latency from speech end to final transcript in seconds. Broadcast via STTMetadataFrame at pipeline start. |
| `keepaliveTimeout` | `number \| null \| undefined` | no | Seconds of no audio before sending silence to keep the connection alive. null disables keepalive. |
| `keepaliveInterval` | `number \| undefined` | no | Seconds between idle checks when keepalive is enabled. |
| `reconnectOnError` | `boolean \| undefined` | no | Whether to automatically reconnect on websocket errors. Defaults to true. |
| `credentialsProvider` | `AWSCredentialsProvider \| null \| undefined` | no | Keys to sign with when none are passed in or set in the environment. Called for every request or connection, so it can renew temporary credentials. |
| `fetch` | `FetchLike \| null \| undefined` | no | `fetch` used for HTTP calls (defaults to the global `fetch`). |
| `endpointUrl` | `string \| null \| undefined` | no | Override the service endpoint (e.g. a VPC endpoint or a local fake), `https://host[:port]`. |

### Settings [#settings]

Runtime-updatable settings, passed as `settings: new AWSTranscribeSTTService.Settings({...})`.
Transcribe binds them to the connection, so changing one reconnects.

| Setting                   | Type                          | Default       | Description                                                                |
| ------------------------- | ----------------------------- | ------------- | -------------------------------------------------------------------------- |
| `language`                | `Language`                    | `Language.EN` | Transcription language, mapped to a Transcribe code (`en-US`, `es-ES`, …). |
| `partialResultsStability` | `"high" \| "medium" \| "low"` | `"high"`      | Stability of interim results: `"high"` is fastest, `"low"` most accurate.  |
| `model`                   | `string \| null`              | `null`        | Not used by Transcribe.                                                    |

### Usage [#usage]

```ts title="stt.ts"
import { AWSTranscribeSTTService } from "@pipedog/aws/stt";
import { STTUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { Language } from "@pipedog/core/transcriptions/language";

const stt = new AWSTranscribeSTTService({
  // Or leave these out to read AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY,
  // AWS_SESSION_TOKEN and AWS_REGION from the environment.
  awsAccessKeyId: process.env.AWS_ACCESS_KEY_ID,
  apiKey: process.env.AWS_SECRET_ACCESS_KEY,
  region: "us-east-1",
  sampleRate: 16000,
  settings: new AWSTranscribeSTTService.Settings({
    language: Language.EN_US,
    partialResultsStability: "medium",
  }),
});
```

Change the language mid-call with an `STTUpdateSettingsFrame`:

```ts
// Queue it on the worker, or push it from a processor upstream of the service.
// Transcribe binds the language to the connection, so this reconnects.
const update = new STTUpdateSettingsFrame({
  delta: new AWSTranscribeSTTService.Settings({ language: Language.ES_ES }),
});
```

### Event handlers [#event-handlers]

| Event                 | Description                                    |
| --------------------- | ---------------------------------------------- |
| `on_connected`        | The WebSocket to Transcribe is open.           |
| `on_disconnected`     | The WebSocket to Transcribe closed.            |
| `on_connection_error` | Connecting failed. Receives the error message. |

### Notes [#notes]

* Emits `InterimTranscriptionFrame`s for partial results and a `TranscriptionFrame` for each
  final result.
* Transcribe accepts 8000 or 16000 Hz PCM. Another `sampleRate` (or pipeline rate) is replaced by
  16000 Hz on connect, with a warning.
* Partial-result stabilization is always on; `partialResultsStability` sets its level.
* On disconnect the service sends upstream's end-of-stream message before closing the socket.
* A 401 or 403 from Transcribe is reported as a connectivity error, as upstream does.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/aws/package.json`: ✅ means the compat smoke test passes on that runtime.

* Runs on Bun, Node, Deno and Cloudflare Workers. The URL is presigned with SigV4 on WebCrypto,
  and the socket is the global `WebSocket` (opened through `fetch` on Workers).
* Only the `Origin` handshake header is sent; the runtime's `WebSocket` sets the others.
* There is no default AWS credential chain (profiles, IMDS, ECS roles, SSO). Use keys, the
  environment, or `credentialsProvider`.
* `endpointUrl` replaces `wss://transcribestreaming.<region>.amazonaws.com:8443`, for a proxy or a
  local fake. The URL is still signed for the AWS host.
* Not ported: the deprecated `language` constructor option. Use `settings.language`.