> ## Documentation Index
> Fetch the complete documentation index at: https://docs.weflayr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Routing

> For each new call, the router looks up the most similar calls of your benchmark and picks a model based on how well each model answered them.

> [!TIP]
>
> Create your [routing strategy](https://docs.weflayr.com/product/routing-strategies.md).

For every incoming call, the router finds the **benchmark calls most similar to it** and looks at how each candidate model did on those calls. By combining their [verdicts](https://docs.weflayr.com/concepts/benchmarking.md#side-by-side-relative-evaluation), weighted by similarity, it estimates which models are most likely to **match or beat your current model's quality on that specific call**.

![The new request is encoded as one vector, its nearest benchmark requests are found, and each model at each reasoning level gets a quality score from those requests' verdicts, weighted by distance, which ranks the models.](https://mintcdn.com/weflayr/gQ4NnqAYQo0h7eyp/images/concepts/router-similar-requests.png?fit=max\&auto=format\&n=gQ4NnqAYQo0h7eyp\&q=85\&s=119e08926b213e6a39047163bba5b0a4)

## Finding similar calls

The router encodes each call from two kinds of signals:

* **Semantic**: embeddings of what the call is about, such as its latest message or its tool definitions.
* **Structural**: the shape of the call, such as its length and its number of turns and tool calls.

It then finds the benchmark calls closest to it.

## Predicting each model's quality against the baseline

For each candidate model, at each reasoning level, the router reads its verdicts on those similar calls, giving more weight to the calls closest to the incoming one. The result is a quality score per model: how likely its answer is to be as good as or better than your current model's.

## Choosing which model to call

The router ranks the models by the objective of your [routing strategy](https://docs.weflayr.com/product/routing-strategies.md):

* **Best quality**: the model with the highest quality score, the cheapest one on a tie.
* **Same quality, lowest cost**: the cheapest model whose quality score reaches your current model's.
* **Same cost, best quality**: the best model at about your current model's cost.

The gateway calls the first model at the reasoning level it ranked best at. If that call fails, it moves on to the next model of the ranking (see [Fallback and model health](https://docs.weflayr.com/concepts/fallback-and-model-health.md)). The ranking also accounts for each model's [prompt cache](https://docs.weflayr.com/concepts/prompt-caching.md): a model that already holds the start of the prompt costs less to call.

<video src="https://mintcdn.com/weflayr/gQ4NnqAYQo0h7eyp/images/concepts/router-picks-a-model.mp4?fit=max&auto=format&n=gQ4NnqAYQo0h7eyp&q=85&s=8d6137684c9fb104e3ccffd87d28f65c" poster="/images/concepts/router-picks-a-model-poster.jpg" aria-label="A call from your app is matched to the most similar benchmarked calls, then the router picks a model and reasoning level from the recommended model mix" autoPlay loop muted playsInline preload="metadata" />

## Next

> [card] [Routing strategies](https://docs.weflayr.com/product/routing-strategies.md)
>
> Pick the objective and the models in the dashboard.

> [card] [Benchmarking](https://docs.weflayr.com/concepts/benchmarking.md)
>
> How the verdicts the router reads are produced.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.