Rime

OverviewCopy Anchor Link

Rime's Coda and Mist models turn text into speech. pipedog ports these Pipecat services:

• RimeTTSService: Rime's JSON WebSocket API (/ws3), with word timestamps and one audio context per turn.
• RimeHttpTTSService: Rime's HTTP API, streaming raw PCM, for when a WebSocket is not needed.
• RimeV1TTSService: Rime's WebSocket v1 API, which carries protobuf envelopes (binary or proto3 JSON). Interruptions cancel a context without reconnecting, and Rime can accumulate streamed LLM tokens into sentences.

InstallationCopy Anchor Link

bun add @pipedog/rime

import { RimeHttpTTSService, RimeTTSService } from "@pipedog/rime/tts";
import { RimeV1TTSService } from "@pipedog/rime/tts_v1";

PrerequisitesCopy Anchor Link

• A Rime account and an API key from the Rime dashboard.
• A voice (speaker) for the model you use. There is no default voice.
• Pass the key as apiKey. The examples read it from RIME_API_KEY.

RimeTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Runtime-updatable settings, passed as settings: new RimeTTSService.Settings({...}). RimeHttpTTSService uses the same settings class. Each setting is only sent when set; the sampling settings go to coda and the mistv2 ones to the other models.

UsageCopy Anchor Link

import { TTSUpdateSettingsFrame } from "@pipedog/core/frames/frames";
import { RimeTTSService } from "@pipedog/rime/tts";

const tts = new RimeTTSService({
  apiKey: process.env.RIME_API_KEY ?? "",
  settings: new RimeTTSService.Settings({
    model: "coda",
    voice: "luna",
    timeScaleFactor: 1.0,
  }),
});

Text helpers for Rime's markup:

// Text helpers for Rime's markup, for use in text transforms.
const spelled = RimeTTSService.SPELL("A1B2");
const paused = `One moment.${RimeTTSService.PAUSE_TAG(0.5)} Thanks.`;

// Custom pronunciation and inline speed apply to the next message sent.
const pronounced = tts.PRONOUNCE("Say hello", "hello", "h'El0");
const slow = tts.INLINE_SPEED("slowly", 0.7);

Change settings mid-call with a TTSUpdateSettingsFrame:

// Queue it on the worker, or push it from a processor upstream of the service.
// Every setting of the WebSocket service is a URL parameter, so a change reconnects.
const update = new TTSUpdateSettingsFrame({
  delta: new RimeTTSService.Settings({ voice: "another-voice" }),
});

Event handlersCopy Anchor Link

Inherited from TTSService:

NotesCopy Anchor Link

• Every setting is a query parameter of the WebSocket URL, so any change reconnects.
• The service does not push the LLM's text frames. It builds TTSTextFrames from Rime's word timestamps.
• Text inside spell(...) is never split across aggregates.
• PRONOUNCE() and INLINE_SPEED() add fields to the next message only.

RimeHttpTTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

UsageCopy Anchor Link

import { RimeHttpTTSService } from "@pipedog/rime/tts";

const httpTts = new RimeHttpTTSService({
  apiKey: process.env.RIME_API_KEY ?? "",
  settings: new RimeHttpTTSService.Settings({ model: "mistv2", voice: "luna", speedAlpha: 1.1 }),
});

NotesCopy Anchor Link

• It posts to https://users.rime.ai/v1/rime-tts and streams the PCM body. There are no word timestamps.
• The default language is "eng".

RimeV1TTSServiceCopy Anchor Link

ConfigurationCopy Anchor Link

SettingsCopy Anchor Link

Passed as settings: new RimeV1TTSService.Settings({...}). model is required at construction. Each context uses the settings in effect when it starts. A setting the model does not accept is left out with a warning.

UsageCopy Anchor Link

import { TextAggregationMode } from "@pipedog/core/services/tts_service";
import { RimeV1TTSService } from "@pipedog/rime/tts_v1";

const v1 = new RimeV1TTSService({
  apiKey: process.env.RIME_API_KEY ?? "",
  // "binary" (protobuf) or "json" (proto3 JSON). Binary is the default.
  websocketProtocol: "binary",
  // Stream LLM tokens and let Rime accumulate them into sentences.
  textAggregationMode: TextAggregationMode.TOKEN,
  // The model is required: "coda" and "mistv3" select Rime's public endpoints.
  settings: new RimeV1TTSService.Settings({ model: "coda", voice: "luna" }),
});

NotesCopy Anchor Link

• Pass websocketUrl for a custom deployment. The model is never inferred from the URL.
• Rime v1 sends a ready event before the first context. The service waits up to 10 seconds for it and reports a connection error otherwise.
• A failed context is reported with the error's ErrorCategory, and the service stays usable for the next turn.
• Rime v1 does not report word timestamps.

Runtime notesCopy Anchor Link

• The WebSocket services use the global WebSocket and send the key as an Authorization header on the handshake. That works on Bun, Node and Deno, and on Cloudflare Workers, where the socket is opened through fetch.
• RimeV1TTSService reads the negotiated subprotocol (rime.v1.binary or rime.v1.json) from the socket and refuses a connection that did not accept it.
• The protobuf messages come from Rime's own package, @rimelabs/api, with @bufbuild/protobuf. Both are plain JavaScript.
• RimeHttpTTSService uses fetch. Upstream's required aiohttp_session is replaced by an optional fetch option.
• Not ported: the deprecated RimeNonJsonTTSService and the InputParams classes, and the voiceId, model, params and aggregateSentences constructor options.
• Whole-number floats, such as PAUSE_TAG(0.5)'s 500, print without Python's .0.

Generated from pipecat.runtimes and pipecat.compat in packages/rime/package.json: ✅ means the compat smoke test passes on that runtime.