Azure OpenAI

OverviewCopy Anchor Link

• AzureImageGenServiceREST: turns each TextFrame it receives into a prompt for Azure OpenAI's asynchronous image generation API. It submits the job, polls it once a second until it succeeds, downloads the image and pushes it as a URLImageRawFrame of RGB pixels.

InstallationCopy Anchor Link

bun add @pipedog/azure

import { AzureImageGenServiceREST } from "@pipedog/azure/image";

PrerequisitesCopy Anchor Link

• An Azure OpenAI resource whose region offers image generation, and its API key. The examples read it from AZURE_OPENAI_API_KEY.
• The resource endpoint, ending in / (the service appends openai/images/... to it).

AzureImageGenServiceRESTCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new AzureImageGenServiceREST.Settings({...}) (AzureImageGenSettings). Any setting you leave out keeps the default below.

UsageCopy Anchor Link

import { AzureImageGenServiceREST } from "@pipedog/azure/image";

const imageGen = new AzureImageGenServiceREST({
  apiKey: process.env.AZURE_OPENAI_API_KEY ?? "",
  // The resource endpoint, with its trailing slash.
  endpoint: "https://my-resource.openai.azure.com/",
  settings: new AzureImageGenServiceREST.Settings({ imageSize: "1024x1024" }),
});

NotesCopy Anchor Link

• The request is POST {endpoint}openai/images/generations:submit?api-version=... with the key in the api-key header and { prompt, n: 1, size } as the body. The job's status is read from the operation-location URL the submission returns.
• Polling gives up after 119 attempts, about two minutes, with an ErrorFrame ("Image generation timed out"). A finished job without an image URL gives "Image generation failed".
• A submission without an operation-location header (a 429, for example) throws, as upstream.
• apiVersion defaults to "2023-06-01-preview", the version this asynchronous API was published under.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/azure/package.json: ✅ means the compat smoke test passes on that runtime.

• All requests go through fetch (the global one, or the fetch option), which replaces upstream's required aiohttp_session. The service works on Bun, Node, Deno and Cloudflare Workers.
• The image is decoded to RGB with decodePngToRgb from @pipedog/openai/image, a pure JS PNG decoder, and the frame's format is always "RGB". Upstream decodes with Pillow and reports Pillow's mode. Pass decodeImage to decode other formats.
• The deprecated imageSize and model constructor options are not ported. Use settings.