# Vonage

> Vonage Audio Connector streams with VonageFrameSerializer, used with FastAPIWebsocketTransport.

Source: https://polvallverdu.github.io/pipedog/docs/serializers/vonage/



## Overview [#overview]

**`VonageFrameSerializer`** speaks the WebSocket protocol of Vonage's Audio Connector, which
streams a Vonage Video session's audio (or a Voice API call's) to your server:

* Binary messages carry raw 16-bit PCM (16 kHz by default), resampled to the pipeline's input rate
  as `InputAudioRawFrame`s.
* Bot audio is resampled to the Vonage rate and sent as binary messages.
* JSON text events: `websocket:connected`, `websocket:cleared` and `websocket:notify` are logged,
  and `websocket:dtmf` becomes an `InputDTMFFrame`.
* An `InterruptionFrame` sends `{"action": "clear"}`.

<Cards>
  <Card title="Upstream page" href="https://docs.pipecat.ai/api-reference/server/services/serializers/vonage">
    Pipecat's documentation for the same serializer.
  </Card>

  <Card title="Source" href="https://github.com/polvallverdu/pipedog/tree/main/packages/serializer-vonage/src">
    `packages/serializer-vonage/src/vonage.ts`.
  </Card>

  <Card title="Vonage docs" href="https://developer.vonage.com/en/video/guides/audio-connector">
    The Audio Connector.
  </Card>
</Cards>

## Installation [#installation]

```bash
bun add @pipedog/serializer-vonage @pipedog/transport-websocket
```

```ts
import { VonageFrameSerializer } from "@pipedog/serializer-vonage/vonage";
```

## Prerequisites [#prerequisites]

* A Vonage application, and an Audio Connector (or a Voice API `websocket` endpoint) pointed at
  your WebSocket URL, with `audio/l16` at the rate you configure.

## VonageFrameSerializer [#vonageframeserializer]

### Configuration [#configuration]

| Option | Type | Required | Description |
|---|---|---|---|
| `params` | `VonageFrameSerializerInputParams \| null \| undefined` | no | Configuration parameters. |
| `name` | `string \| null \| undefined` | no | Custom name; defaults to `ClassName#count`. |
| `taskManager` | `BaseTaskManager \| null \| undefined` | no | Task manager for handling tasks. |

### InputParams [#inputparams]

`params` is a `VonageFrameSerializer.InputParams` (`VonageFrameSerializerInputParams`):

| Param                     | Type             | Default | Description                                                                                    |
| ------------------------- | ---------------- | ------- | ---------------------------------------------------------------------------------------------- |
| `vonageSampleRate`        | `number`         | `16000` | Sample rate of the Vonage stream, in Hz (8000, 16000 or 24000).                                |
| `sampleRate`              | `number \| null` | `null`  | Override for the pipeline input rate. `null` uses the pipeline's.                              |
| `resamplerClearAfterSecs` | `number \| null` | `0.2`   | Seconds of silence after which the stream resamplers clear their history. `null` never clears. |
| `ignoreRtviMessages`      | `boolean`        | `true`  | Drop RTVI messages instead of sending them to Vonage.                                          |

### Usage [#usage]

```ts title="vonage.ts"
import { VonageFrameSerializer } from "@pipedog/serializer-vonage/vonage";
import {
  FastAPIWebsocketParams,
  FastAPIWebsocketTransport,
} from "@pipedog/transport-websocket/fastapi";
import { websocketFetchHandler } from "@pipedog/transport-websocket/fetch";
import type { WebsocketConnection } from "@pipedog/transport-websocket/socket";

async function runSession(connection: WebsocketConnection): Promise<void> {
  const serializer = new VonageFrameSerializer({
    // The rate you gave the Audio Connector (audio/l16;rate=16000).
    params: new VonageFrameSerializer.InputParams({ vonageSampleRate: 16000 }),
  });

  const transport = new FastAPIWebsocketTransport(
    connection,
    new FastAPIWebsocketParams({
      audioInEnabled: true,
      audioOutEnabled: true,
      addWavHeader: false,
      serializer,
    }),
  );
  // ... build and run the pipeline with transport.input() and transport.output()
}

// The Audio Connector's websocket.uri points here.
const handler = websocketFetchHandler(runSession);
```

### Notes [#notes]

* DTMF digits may arrive as strings or numbers, under `digit` or `dtmf.digit`; both work.
* `OutputTransportMessageFrame`s are sent as JSON text (e.g. a `notify` action), except RTVI
  messages while `ignoreRtviMessages` is on.
* There is no hang-up call: end the session from Vonage's side.

## Runtime notes [#runtime-notes]

| Runtime            | Works | What it needs |
| ------------------ | ----- | ------------- |
| Bun                | ✅ yes | Nothing.      |
| Node.js            | ✅ yes | Nothing.      |
| Deno               | ✅ yes | Nothing.      |
| Cloudflare Workers | ✅ yes | Nothing.      |

Generated from `pipecat.runtimes` and `pipecat.compat` in `packages/serializer-vonage/package.json`: ✅ means the compat smoke test passes on that runtime.

Runs on Bun, Node, Deno and Cloudflare Workers. The package's golden tests replay vectors recorded
from upstream: at 16 kHz the audio is byte-identical.