Skip to main content

Overview

PerformanceEval measures runtime performance metrics against configurable limits.

Quick Start

import { PerformanceEval } from "@radaros/eval";
import { Agent, openai } from "@radaros/core";

const agent = new Agent({ name: "fast-bot", model: openai("gpt-4o-mini") });

const eval = new PerformanceEval({
  name: "latency-test",
  agent,
  maxDurationMs: 5000,
  maxTimeToFirstTokenMs: 1000,
  maxTokens: 500,
  cases: [
    { name: "greeting", input: "Hello!" },
    { name: "complex", input: "Explain quantum computing" },
  ],
});

const result = await eval.run();

Metrics Tracked

MetricConfigDescription
DurationmaxDurationMsTotal wall-clock time
TTFTmaxTimeToFirstTokenMsTime to first token
TokensmaxTokensTotal token consumption
MemoryHeap memory delta (always tracked)