AWS Bedrock

OverviewCopy Anchor Link

AWSBedrockLLMService runs any Bedrock model that supports the Converse API. It streams text with ConverseStream, calls functions, supports prompt caching and reasoning effort, and reports token usage, including cache tokens.

• AWSBedrockLLMService: the LLM service, with AWSBedrockLLMSettings for runtime-updatable settings.
• AWSBedrockLLMAdapter: converts the universal LLMContext and tools to Converse messages, system blocks and toolConfig.

InstallationCopy Anchor Link

bun add @pipedog/aws

import { AWSBedrockLLMService } from "@pipedog/aws/llm";

No AWS SDK is needed: the service calls the Bedrock REST API with fetch, signed with SigV4.

PrerequisitesCopy Anchor Link

• An AWS account with access to the model in the Bedrock console, in the region you use.

• An IAM identity allowed bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream.

• Keys, in this order of precedence:

  1. awsAccessKey, awsSecretKey and awsSessionToken.
  2. The AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN environment variables.
  3. A credentialsProvider function, called for every request.

  The region comes from awsRegion, else AWS_REGION, else us-east-1.

AWSBedrockLLMServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new AWSBedrockLLMService.Settings({...}). Any setting you leave out keeps the default below. null means the field is not sent.

UsageCopy Anchor Link

import { AWSBedrockLLMService } from "@pipedog/aws/llm";
import { LLMUpdateSettingsFrame } from "@pipedog/core/frames/frames";

const llm = new AWSBedrockLLMService({
  // Or leave these out: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY,
  // AWS_SESSION_TOKEN and AWS_REGION are read from the environment.
  awsAccessKey: process.env.AWS_ACCESS_KEY_ID,
  awsSecretKey: process.env.AWS_SECRET_ACCESS_KEY,
  awsRegion: "us-east-1",
  settings: new AWSBedrockLLMService.Settings({
    model: "us.anthropic.claude-haiku-4-5-20251001-v1:0",
    systemInstruction: "You are a helpful voice assistant. Keep answers short.",
    maxTokens: 1024,
    enablePromptCaching: true,
  }),
});

With temporary credentials from a role, SSO or the instance metadata service, pass a provider. fromNodeProviderChain() from @aws-sdk/credential-providers (Node only) caches and renews them:

// Temporary credentials that renew themselves, e.g. from the AWS SDK's
// provider chain: the function is called for every request.
declare function fromNodeProviderChain(): () => Promise<{
  accessKeyId: string;
  secretAccessKey: string;
  sessionToken?: string;
}>;

const roleLlm = new AWSBedrockLLMService({
  awsRegion: "eu-west-1",
  credentialsProvider: fromNodeProviderChain(),
});

Change settings mid-call with an LLMUpdateSettingsFrame:

// Queue it on the worker, or push it from a processor upstream of the service.
const update = new LLMUpdateSettingsFrame({
  delta: new AWSBedrockLLMService.Settings({ temperature: 0.3, maxTokens: 512 }),
});

Event handlersCopy Anchor Link

Inherited from LLMService:

NotesCopy Anchor Link

• Consumes LLMContextFrame and emits LLMFullResponseStartFrame, LLMTextFrames and LLMFullResponseEndFrame.
• Reasoning the model returns in reasoningContent (Claude with extended thinking, Nova 2 with reasoning enabled) comes out as LLMThoughtStartFrame, LLMThoughtTextFrame and LLMThoughtEndFrame. Signed reasoning, such as Claude's, is kept in the context and sent back on later turns, so Claude keeps its reasoning across tool calls.
• Amazon Nova (v1) models write their reasoning into the answer, in <thinking>...</thinking>, before a tool call. A <thinking> block at the start of the response, or of any text between tool calls, is emitted as thought frames instead of LLMTextFrames, so it isn't spoken.
• When the context has tool calls in its history but no tools are configured, the service sends a no-op tool, because Bedrock rejects tool messages without a toolConfig.
• For Claude models that reject assistant prefill, a trailing user message is added when the context ends with an assistant message.
• runInference uses Converse (not streaming). Converse has no reply schema, so a responseSchema is accepted but not enforced.
• With retryOnTimeout: true, the first attempt is limited to retryTimeoutSecs (5 by default). If it times out, the request is retried once with no timeout.
• Errors are AWSServiceErrors with the AWS error code (ThrottlingException, ValidationException, …) and the HTTP status. An exception sent inside the event stream has status 0.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/aws/package.json: ✅ means the compat smoke test passes on that runtime.

• Runs on Bun, Node, Deno and Cloudflare Workers: requests are signed with SigV4 on WebCrypto, and the application/vnd.amazon.eventstream response is decoded in the package.
• There is no default AWS credential chain (profiles, IMDS, ECS roles, SSO). Use the keys, the environment, or credentialsProvider.
• botocore's client_config is not ported, so there are no client-side retries of throttled or 5xx requests and no 300 s timeouts. Pass your own fetch to add them.
• endpointUrl sends requests to another host (a VPC endpoint or a local fake); fetch replaces the global fetch.
• Not ported: the deprecated AWSBedrockLLMService.InputParams and the model, params and stopSequences constructor arguments. Use settings.