Protobuf

OverviewCopy Anchor Link

A serializer turns frames into the messages a transport sends, and incoming messages back into frames. ProtobufFrameSerializer encodes frames as Protocol Buffers with Pipecat's frames.proto schema, so a pipedog bot talks to Pipecat's WebSocket clients and to Pipecat bots over the same wire format.

It is the default serializer of WebsocketClientTransport, and the usual choice for FastAPIWebsocketTransport and SingleClientWebsocketServerTransport when the other end is a Pipecat client.

InstallationCopy Anchor Link

bun add @pipedog/serializer-protobuf

import { ProtobufFrameSerializer } from "@pipedog/serializer-protobuf/protobuf";

@pipedog/transport-websocket already depends on it.

PrerequisitesCopy Anchor Link

None. No account or key is needed. The other end must speak the same schema: a Pipecat client SDK's WebSocket transport with its protobuf serializer, or another Pipecat or pipedog bot.

ProtobufFrameSerializerCopy Anchor Link

ConfigurationCopy Anchor Link

new ProtobufFrameSerializer(options?). The options are the base serializer's:

UsageCopy Anchor Link

import { TextFrame } from "@pipedog/core/frames/frames";
import { ProtobufFrameSerializer } from "@pipedog/serializer-protobuf/protobuf";
import {
  FastAPIWebsocketParams,
  FastAPIWebsocketTransport,
} from "@pipedog/transport-websocket/fastapi";
import type { WebsocketConnection } from "@pipedog/transport-websocket/socket";

function createTransport(connection: WebsocketConnection): FastAPIWebsocketTransport {
  return new FastAPIWebsocketTransport(
    connection,
    new FastAPIWebsocketParams({
      audioInEnabled: true,
      audioOutEnabled: true,
      serializer: new ProtobufFrameSerializer(),
    }),
  );
}

The transports call serialize() and deserialize() for you. Called directly:

// The transports call these for you; this shows the wire form of one frame.
const serializer = new ProtobufFrameSerializer();
const bytes = await serializer.serialize(new TextFrame({ text: "hello" }));
const frame = bytes ? await serializer.deserialize(bytes) : null;

NotesCopy Anchor Link

The schema has five frame types:

• Transport messages go on the wire as a message frame whose data is the JSON-encoded message. On the way in, data is parsed back to an object; invalid JSON logs an error and returns null.
• Any other frame type logs a warning and serializes to null, which the transport skips.
• RTVI messages are always sent. The base serializer drops them by default to keep them off telephony streams, but this constructor sets ignoreRtviMessages to false, even if you pass params: WebSocket transports are how RTVI messages reach the client.
• deserialize() takes bytes. A text message throws a TypeError, as protobuf's FromString does upstream.
• The frame's id, name and pts (for audio) are carried across when set.

Runtime notesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/serializer-protobuf/package.json: ✅ means the compat smoke test passes on that runtime.

Upstream's frames_pb2 is generated by protoc and needs the protobuf runtime. pipedog's core has no third-party dependencies, so frames_pb2.ts is a hand-written proto3 codec for the same schema, with the same wire format. uint64 fields are JS numbers, exact up to 2^53.

The serializer runs on Bun, Node, Deno and Cloudflare Workers.

The other serializers are ported too: RTVI client, and the telephony serializers for Twilio, Telnyx, Plivo, Exotel, Vonage and Genesys.