# Protobuf

> Binary frame serialization for WebSocket transports with ProtobufFrameSerializer, wire-compatible with Pipecat's frames.proto.

Source: https://polvallverdu.github.io/pipedog/docs/serializers/protobuf/



## Overview [#overview]

A serializer turns frames into the messages a transport sends, and incoming messages back into
frames. **`ProtobufFrameSerializer`** encodes frames as Protocol Buffers with Pipecat's
`frames.proto` schema, so a pipedog bot talks to Pipecat's WebSocket clients and to Pipecat bots
over the same wire format.

It is the default serializer of `WebsocketClientTransport`, and the usual choice for
`FastAPIWebsocketTransport` and `SingleClientWebsocketServerTransport` when the other end is a
Pipecat client.

<Cards>
  <Card title="Upstream: serializers" href="https://docs.pipecat.ai/api-reference/server/services/serializers/introduction">
    Pipecat's serializer overview. Upstream has no page for this class.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/serializer-protobuf/src">
    `packages/serializer-protobuf/src/protobuf.ts`, with the schema codec in
    `packages/core/src/frames/protobufs/frames_pb2.ts`.
  </Card>

  <Card title="WebSocket transports" href="/docs/transports/websocket">
    The transports this serializer is used with.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/serializer-protobuf
```

```ts
import { ProtobufFrameSerializer } from "@pipedog/serializer-protobuf/protobuf";
```

`@pipedog/transport-websocket` already depends on it.

## Prerequisites [#prerequisites]

None. No account or key is needed. The other end must speak the same schema: a Pipecat client
SDK's WebSocket transport with its protobuf serializer, or another Pipecat or pipedog bot.

## ProtobufFrameSerializer [#protobufframeserializer]

### Configuration [#configuration]

`new ProtobufFrameSerializer(options?)`. The options are the base serializer's:

| Option | Type | Required | Description |
|---|---|---|---|
| `params` | `FrameSerializerInputParams \| null \| undefined` | no | Configuration parameters. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |

### Usage [#usage]

```ts title="transport.ts"
import { TextFrame } from "@pipedog/core/frames/frames";
import { ProtobufFrameSerializer } from "@pipedog/serializer-protobuf/protobuf";
import {
  FastAPIWebsocketParams,
  FastAPIWebsocketTransport,
} from "@pipedog/transport-websocket/fastapi";
import type { WebsocketConnection } from "@pipedog/transport-websocket/socket";

function createTransport(connection: WebsocketConnection): FastAPIWebsocketTransport {
  return new FastAPIWebsocketTransport(
    connection,
    new FastAPIWebsocketParams({
      audioInEnabled: true,
      audioOutEnabled: true,
      serializer: new ProtobufFrameSerializer(),
    }),
  );
}
```

The transports call `serialize()` and `deserialize()` for you. Called directly:

```ts
// The transports call these for you; this shows the wire form of one frame.
const serializer = new ProtobufFrameSerializer();
const bytes = await serializer.serialize(new TextFrame({ text: "hello" }));
const frame = bytes ? await serializer.deserialize(bytes) : null;
```

### Notes [#notes]

The schema has five frame types:

| Proto field     | Serialized from                                                    | Deserialized to              |
| --------------- | ------------------------------------------------------------------ | ---------------------------- |
| `text`          | `TextFrame`                                                        | `TextFrame`                  |
| `audio`         | `OutputAudioRawFrame`                                              | `InputAudioRawFrame`         |
| `transcription` | `TranscriptionFrame`                                               | `TranscriptionFrame`         |
| `message`       | `OutputTransportMessageFrame`, `OutputTransportMessageUrgentFrame` | `InputTransportMessageFrame` |
| `interruption`  | `InterruptionFrame`                                                | `InterruptionFrame`          |

* Transport messages go on the wire as a `message` frame whose `data` is the JSON-encoded
  message. On the way in, `data` is parsed back to an object; invalid JSON logs an error and
  returns `null`.
* Any other frame type logs a warning and serializes to `null`, which the transport skips.
* RTVI messages are always sent. The base serializer drops them by default to keep them off
  telephony streams, but this constructor sets `ignoreRtviMessages` to `false`, even if you pass
  params: WebSocket transports are how RTVI messages reach the client.
* `deserialize()` takes bytes. A text message throws a `TypeError`, as protobuf's `FromString`
  does upstream.
* The frame's `id`, `name` and `pts` (for audio) are carried across when set.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/serializer-protobuf/package.json`: ✅ means the compat smoke test passes on that runtime.

Upstream's `frames_pb2` is generated by `protoc` and needs the `protobuf` runtime. pipedog's core
has no third-party dependencies, so `frames_pb2.ts` is a hand-written proto3 codec for the same
schema, with the same wire format. `uint64` fields are JS numbers, exact up to 2^53.

The serializer runs on Bun, Node, Deno and Cloudflare Workers.

The other serializers are ported too: [RTVI client](https://polvallverdu.github.io/pipedog/docs/serializers/rtvi-client.md), and the
telephony serializers for [Twilio](https://polvallverdu.github.io/pipedog/docs/serializers/twilio.md), [Telnyx](https://polvallverdu.github.io/pipedog/docs/serializers/telnyx.md),
[Plivo](https://polvallverdu.github.io/pipedog/docs/serializers/plivo.md), [Exotel](https://polvallverdu.github.io/pipedog/docs/serializers/exotel.md),
[Vonage](https://polvallverdu.github.io/pipedog/docs/serializers/vonage.md) and [Genesys](https://polvallverdu.github.io/pipedog/docs/serializers/genesys.md).