# AWS Nova Sonic > Real-time speech-to-speech with Amazon Nova Sonic on Bedrock through AWSNovaSonicLLMService, with tools and session continuation. Source: https://polvallverdu.github.io/pipedog/docs/services/s2s/aws/ ## Overview [#overview] `AWSNovaSonicLLMService` streams the user's audio to Amazon Nova Sonic on Bedrock (`InvokeModelWithBidirectionalStream`) and returns spoken audio, transcripts and tool calls, so one service replaces the STT, LLM and TTS stages of a pipeline. * **`AWSNovaSonicLLMService`**: the session, with `AWSNovaSonicLLMSettings` and `AudioConfig`. * **`AWSNovaSonicLLMAdapter`**: converts the `LLMContext` history and tools to Nova Sonic's events. * **`SessionContinuationParams`**: hands the conversation to a new session before Nova Sonic's session time limit, buffering audio so nothing is lost. Pipecat's documentation for the same service. `packages/aws/src/nova_sonic/llm.ts`, `session_continuation.ts` and `bidirectional_stream.ts`. Voices, events, tools and limits. ## Installation [#installation] ```bash bun add @pipedog/aws ``` ```ts import { AudioConfig, AWSNovaSonicLLMService } from "@pipedog/aws/nova_sonic/llm"; ``` The default transport needs no AWS SDK. For `transport: "http2"`, also install `@aws-sdk/client-bedrock-runtime` (an optional peer dependency). ## Prerequisites [#prerequisites] * Access to Nova Sonic in the Bedrock console, in a region that offers it. * An IAM identity allowed `bedrock:InvokeModelWithBidirectionalStream`. * Keys, in this order of precedence: 1. `accessKeyId`, `secretAccessKey` and `sessionToken`. 2. The `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN` environment variables. 3. A `credentialsProvider` function, called for every new session. The region comes from `region`, else `AWS_REGION`, else `us-east-1`. ## AWSNovaSonicLLMService [#awsnovasonicllmservice] ### Configuration [#configuration] | Option | Type | Required | Description | |---|---|---|---| | `secretAccessKey` | `string \| null \| undefined` | no | AWS secret access key for authentication. Falls back to `AWS_SECRET_ACCESS_KEY`. | | `accessKeyId` | `string \| null \| undefined` | no | AWS access key ID for authentication. Falls back to `AWS_ACCESS_KEY_ID`. | | `sessionToken` | `string \| null \| undefined` | no | AWS session token for authentication. Falls back to `AWS_SESSION_TOKEN`. | | `region` | `string \| null \| undefined` | no | AWS region where the service is hosted. Supported regions: "us-east-1", "us-west-2", "eu-north-1", "ap-northeast-1". Falls back to `AWS_REGION`. | | `credentialsProvider` | `AWSCredentialsProvider \| null \| undefined` | no | Keys to use when none are passed or set in the environment (pipedog addition). | | `audioConfig` | `AudioConfig \| null \| undefined` | no | Audio configuration (sample rates, sample sizes, channel counts). If not provided, defaults are used. | | `settings` | `AWSNovaSonicLLMSettings \| null \| undefined` | no | AWS Nova Sonic LLM settings. | | `tools` | `(FunctionSchema \| DirectFunction)[] \| ToolsSchema \| null \| undefined` | no | Available tools for the model: a `ToolsSchema` or a plain list of direct functions and/or `FunctionSchema` objects (handlers auto-register). | | `sessionContinuation` | `SessionContinuationParams \| null \| undefined` | no | Configuration for automatic session continuation. When enabled (the default), sessions are seamlessly rotated before the AWS time limit (~8 minutes) with no user-perceptible interruption. | | `transport` | `NovaSonicTransport \| undefined` | no | Transport of the default client (pipedog addition): `"websocket"` (default, every runtime) or `"http2"` (`@aws-sdk/client-bedrock-runtime`, Node and Bun). | | `endpointUrl` | `string \| null \| undefined` | no | Endpoint override, `https://host[:port]` (pipedog addition). | | `clientFactory` | `((service: AWSNovaSonicLLMService) => NovaSonicStreamClient) \| null \| undefined` | no | Creates the Bedrock client (pipedog addition; replaces `createClient()`'s default). | | `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. | | `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. | | `enableDirectMode` | `boolean \| undefined` | no | Process frames immediately instead of using internal queues. | | `metrics` | `FrameProcessorMetrics \| null \| undefined` | no | Optional metrics collector for this processor. | | `runInParallel` | `boolean \| undefined` | no | Whether to run function calls in parallel or sequentially. Defaults to true. | | `groupParallelTools` | `boolean \| undefined` | no | Whether to group parallel function calls so the LLM is triggered exactly once after all calls in the batch complete. When false, each function call result triggers the LLM independently as it arrives. Defaults to true. | | `functionCallTimeoutSecs` | `number \| null \| undefined` | no | Optional timeout in seconds for function calls. A call that runs past it is cancelled: its handler is cancelled (it gets a `CancelledError`), the call is settled as cancelled, and inference runs so the LLM can report that it didn't complete. | | `fileResolver` | `FileResolver \| null \| undefined` | no | Resolves file URLs in the context that this provider can't fetch itself (see `FileResolver`). Pass one configured with a `fileStorage` to resolve uploaded-file URLs (`new FileResolver({ fileStorage: runnerArgs.fileStorage })` when a runner hosts the bot), and share one instance across services that may consume the same files (e.g. services switched between mid-session) so downloads are reused. Defaults to a resolver that handles `data:` and `http(s)` URLs only. | ### Settings [#settings] Runtime-updatable settings, passed as `settings: new AWSNovaSonicLLMService.Settings({...})`. | Setting | Type | Default | Description | | ------------------------ | ------------------------------------- | ------------------------- | --------------------------------------------------------- | | `model` | `string` | `"amazon.nova-2-5-sonic"` | Bedrock model ID. | | `voice` | `string` | `"matthew"` | Voice ID. | | `systemInstruction` | `string \| null` | `null` | System prompt, sent when the session starts. | | `temperature` | `number` | `0.7` | Sampling temperature. | | `maxTokens` | `number` | `1024` | Maximum tokens per response. | | `topP` | `number` | `0.9` | Top-p sampling. | | `endpointingSensitivity` | `"LOW" \| "MEDIUM" \| "HIGH" \| null` | `null` | How quickly Nova Sonic decides the user stopped speaking. | ### AudioConfig [#audioconfig] | Option | Type | Required | Description | |---|---|---|---| | `inputSampleRate` | `number \| undefined` | no | | | `inputSampleSize` | `number \| undefined` | no | | | `inputChannelCount` | `number \| undefined` | no | | | `outputSampleRate` | `number \| undefined` | no | | | `outputSampleSize` | `number \| undefined` | no | | | `outputChannelCount` | `number \| undefined` | no | | Defaults: 16 kHz 16-bit mono in, 24 kHz 16-bit mono out. ### Usage [#usage] ```ts title="s2s.ts" import { AudioConfig, AWSNovaSonicLLMService } from "@pipedog/aws/nova_sonic/llm"; import { SessionContinuationParams } from "@pipedog/aws/nova_sonic/session_continuation"; const llm = new AWSNovaSonicLLMService({ // Or leave these out to read the AWS_* environment variables. accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, region: "us-east-1", settings: new AWSNovaSonicLLMService.Settings({ voice: "tiffany", systemInstruction: "You are a friendly voice assistant. Keep answers short.", endpointingSensitivity: "MEDIUM", }), audioConfig: new AudioConfig({ inputSampleRate: 16000, outputSampleRate: 24000 }), // Hand the conversation to a fresh session before Nova Sonic's time limit. sessionContinuation: new SessionContinuationParams({ enabled: true }), }); ``` To use HTTP/2 through the AWS SDK, as upstream does: ```ts // Upstream's transport: HTTP/2 through the AWS SDK (Node and Bun only). // Needs `@aws-sdk/client-bedrock-runtime`. const http2Llm = new AWSNovaSonicLLMService({ region: "us-east-1", transport: "http2", }); ``` ### Notes [#notes] * Nova Sonic detects turns itself. User and bot transcripts are pushed as transcription and `LLMTextFrame`s, and audio as `TTSAudioRawFrame`s. * Tools from the context (or `tools`) are sent when the session starts. Tool calls run through the usual function-call flow, and results go back as `toolResult` events. * With `sessionContinuation`, a new session is opened ahead of the time limit (`transitionThresholdSeconds`) and takes over at the next turn boundary, with the conversation history and buffered user audio. * The JSON events on the wire are upstream's, character for character. Float settings use JS formatting, so `1.0` is sent as `1`. ## Runtime notes [#runtime-notes] | Runtime | Works | What it needs | | ------------------ | ----- | ------------- | | Bun | ✅ yes | Nothing. | | Node.js | ✅ yes | Nothing. | | Deno | ✅ yes | Nothing. | | Cloudflare Workers | ✅ yes | Nothing. | Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/aws/package.json`: ✅ means the compat smoke test passes on that runtime. * The default transport is Bedrock's WebSocket endpoint for `InvokeModelWithBidirectionalStream`, the one the AWS SDK for JavaScript uses in browsers. pipedog presigns the URL and signs each event with SigV4 on WebCrypto, so it runs on Bun, Node, Deno and Cloudflare Workers. * `transport: "http2"` loads `@aws-sdk/client-bedrock-runtime` on connect and uses Node's `http2`, as upstream does: Node and Bun only. * `clientFactory` replaces the stream client entirely (for tests or another transport), and `endpointUrl` changes the host. * Upstream requires the keys; here they are optional, with the environment and `credentialsProvider` as fallbacks. * Not ported: the deprecated `Params` class and the `model`, `voiceId`, `params` and `systemInstruction` constructor options. Use `settings` and `audioConfig`.