> ## Documentation Index
> Fetch the complete documentation index at: https://docs.weflayr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Create a benchmark

> Pick a call, the examples to replay, the models to try and how to judge their answers.

From [Model benchmarks](/product/model-benchmarks), click **New benchmark** and pick the calls to benchmark. The setup then takes four steps, with the estimated cost on the side. **Save draft** keeps it for later.

<Frame>
  <video src="https://mintcdn.com/weflayr/jf99vD0W4dsGuj5S/images/product/create-a-benchmark.mp4?fit=max&auto=format&n=jf99vD0W4dsGuj5S&q=85&s=5ec162749e21bab3bdc7f89f59c343e7" poster="/images/product/create-a-benchmark-poster.jpg" aria-label="New benchmark: pick a call, then the Data, Models, Evaluators and Review steps" controls loop muted playsInline preload="metadata" className="w-full rounded-xl" data-path="images/product/create-a-benchmark.mp4" />
</Frame>

<Steps>
  <Step title="Data">
    The period to sample calls from, the number of **benchmarked examples** (500 is usually enough) and of **validation examples**: later calls kept aside to check the recommended mix on calls it never saw. **Sampling** picks the examples (Smart is recommended), and **Replays per example** repeats each one to steady the scores.
  </Step>

  <Step title="Models">
    **Benchmarking criteria**: what a model must do to be recommended. **Same quality, lowest cost** looks for cheaper models that match your current one, **Same cost, best quality** for better models at about the same price, **Best quality** for the best answers whatever the cost.

    **Baseline model**: the answers every model is compared with. Compare against production for free, with the answers your calls already got, or pick another model.

    **Models for exploration** start with the ones Weflayr recommends for your criteria. Each runs on a few examples first, and only those that meet the criteria run the full benchmark. Put a model in **Always benchmark** to skip that first round.
  </Step>

  <Step title="Evaluators">
    The [evaluators](/product/evaluators) that judge each answer against the baseline's. **Overall quality** is there by default. Add more from your library, create one, or let Weflayr generate evaluators for this call. Mark each one as **Impacts recommendation** or **For reference**.
  </Step>

  <Step title="Review">
    Check the setup, then **Reserve and start benchmark**. The estimate plus a 20% buffer is reserved from your [credits](/product/credits); you pay only the exact cost, and you get an email when the benchmark is done.
  </Step>
</Steps>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.