Skip to main content
A benchmark takes the calls of one of your call names, replays them on other models, and has evaluators compare each answer with your current model’s. It then recommends the models that keep your quality, ready to become a routing strategy.
Model benchmarking page listing benchmarks with their status, recommendation and routing strategies
A benchmark replays your prompts, so your calls must keep their content (the default, see store_message_content on the gateway). It is paid from your credits.

Import past calls

Benchmark calls you logged elsewhere, from a CSV.

Create a benchmark

Pick a call, the models to try and how to judge them.

Benchmark report

Read the results and turn them into a routing strategy.

​
Benchmark list

Each benchmark shows its status (Draft, Running, Stopped, Completed, Archived), its recommendation and the routing strategies made from it. From a row’s menu, continue a draft, stop or resume a run, export a finished benchmark as a CSV, or archive it.