Sentry metrics

OverviewCopy Anchor Link

SentryMetrics (@pipedog/sentry, a port of pipecat.processors.metrics.sentry) extends FrameProcessorMetrics. Wherever a service measures time to first byte or processing time, SentryMetrics also starts a Sentry transaction and ends it when the measurement stops:

The usual MetricsFrames are produced as before, so observers and RTVI clients see no difference. At cleanup it ends any pending transaction and flushes Sentry for up to 5 seconds.

InstallationCopy Anchor Link

bun add @pipedog/sentry

@pipedog/sentry depends on no Sentry package. You install and initialise the Sentry SDK for your runtime, and pass its namespace in. See Choosing the SDK.

RuntimesCopy Anchor Link

Generated from pipecat.runtimes and pipecat.compat in packages/sentry/package.json: ✅ means the compat smoke test passes on that runtime.

The smoke test on every runtime drives SentryMetrics with @sentry/core's tracing functions and a recording transport, and checks the transactions that come out.

UsageCopy Anchor Link

import { OpenAILLMService } from "@pipedog/openai/llm";
import { SentryMetrics } from "@pipedog/sentry";
import * as Sentry from "@sentry/core";

// Sentry.init({ dsn, tracesSampleRate: 1.0 }) has already run, with the SDK for your runtime.

const llm = new OpenAILLMService({
  apiKey: process.env.OPENAI_API_KEY ?? null,
  // One SentryMetrics per processor: each tracks its own transactions.
  metrics: new SentryMetrics({ sentry: Sentry }),
});

// Sentry transactions are only started when metrics are enabled.
const worker = new PipelineWorker(new Pipeline([llm]), {
  params: new PipelineParams({ enableMetrics: true }),
});

Metrics have to be on (enableMetrics) for a service to measure anything, and so for Sentry to see anything. Pass SentryMetrics as the metrics option of a service (or any FrameProcessor), one instance per processor.

OptionsCopy Anchor Link

If the SDK has not been initialised (isInitialized() is false when the SentryMetrics is constructed), a warning is logged and Sentry reporting is switched off. The ordinary metrics keep working.

Choosing the SDKCopy Anchor Link

Sentry's JavaScript SDKs are one per runtime, and all re-export the same tracing functions from @sentry/core. SentryMetrics calls only three of them, described by the SentryClient interface:

interface SentryClient {
  isInitialized(): boolean;
  startInactiveSpan(options: { op: string; name: string; forceTransaction: boolean }): {
    end(): void;
  };
  flush(timeout?: number): PromiseLike<boolean>; // milliseconds
}

So import * as Sentry from "<sdk>" satisfies it, and so does any object of your own with those three methods.

Cloudflare WorkersCopy Anchor Link

@sentry/cloudflare initialises per request, so it needs the Worker wrapped: either the Vite plugin (sentryCloudflareVitePlugin() from @sentry/cloudflare/vite, with options in an instrument.server.ts that default-exports defineCloudflareOptions((env) => ({ dsn, tracesSampleRate }))) or withSentry. The Worker needs the nodejs_compat flag and a compatibility_date of 2024-09-23 or later. Then build the SentryMetrics inside the request handler (the WebSocket upgrade that starts the bot), so isInitialized() is true when it is created:

import * as Sentry from "@sentry/cloudflare";
import { SentryMetrics } from "@pipedog/sentry";

// inside the handler that starts the pipeline:
const metrics = new SentryMetrics({ sentry: Sentry });

Two Workers caveats from Sentry's documentation: span durations show 0 ms where only performance.now() and Date.now() are available, because the Workers clock advances only after I/O, and the SDK needs the request's lifetime to send events (the cleanup flush happens in the bot's own request). Treat the durations in Sentry on Workers as indicative; the MetricsFrame values come from the same clock and have the same limit.

Differences from upstreamCopy Anchor Link

• The SDK is passed in (sentry option) instead of importing sentry_sdk, so one package serves every runtime. Upstream has one optional extra for the Python SDK.
• Transactions are root spans. The JavaScript SDKs since v8 have no startTransaction; the equivalent is startInactiveSpan({ forceTransaction: true }). Sentry's docs say forceTransaction is not available in stream mode, which is the default trace lifecycle of @sentry/core 11. Set traceLifecycle: "static" in Sentry.init to get transactions as upstream sends them. With the default (stream) lifecycle the spans are sent as streamed segment spans, which we have not verified with a real Sentry backend.
• Ending a span runs inline in the background task. Upstream hands tx.finish to an executor thread; ending a JavaScript span only records its end time.
• flush takes milliseconds in Sentry's JS SDKs; SentryMetrics passes 5000.

One upstream behaviour is kept as is: with reportOnlyInitialTtfb: true the base class turns reporting off before SentryMetrics checks it, so that mode starts no TTFB transaction at all.

TestingCopy Anchor Link

Upstream has no test for this module. The package's own tests use a fake SentryClient (spans, flush, the not-initialised case, a span whose end throws) and the real @sentry/core with a transport that records envelopes. Nothing has been sent to Sentry itself, and @sentry/cloudflare, @sentry/node, @sentry/bun and @sentry/deno were not run; they share the functions tested here.