> ## Documentation Index
> Fetch the complete documentation index at: https://docs.weflayr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Create a routing strategy from a benchmark

> From a completed benchmark to a live routing strategy with an API key or an agent: optionally walk through the report, pick an objective, predict its cost, then save the strategy.

> [!NOTE]
>
> An agent, or you, can follow this page by calling the API routes directly with an API key that has the **Management** scope. The [MCP server](https://docs.weflayr.com/api/management/mcp-server.md) is optional: it exposes each route below as a tool with the same name.

Give your agent the sentence "route my `summarize_profile` calls with the results of my last benchmark". It starts from a completed benchmark: if there is none yet, follow [Start a benchmark](https://docs.weflayr.com/agents/start-a-benchmark.md) first.

## Before you start: the API key

Every step needs an API key with the **Management** scope, sent in the `Authorization` header, or set in the MCP server's configuration if the agent uses it. If the agent has none, it asks you to create one at `https://app.weflayr.com/api-keys/`. It then calls [Get API key info](https://docs.weflayr.com/api/management/get-api-key-info.md) (`get_api_key_info`, which works with a key of any scope) and, if `scopes` lacks `management` or the key has expired, asks you for one that has it before going on.

## What changes live routing

[Set routing strategy](https://docs.weflayr.com/api/management/set-routing-strategy.md), [Archive routing strategy](https://docs.weflayr.com/api/management/archive-routing-strategy.md) and [Restore routing strategy](https://docs.weflayr.com/api/management/restore-routing-strategy.md) change how live calls are routed at once, so the agent asks you before each of them. Every other step only reads.

## Who it is for

An API key belongs to a project, not to a person. What an agent creates is recorded for the person who created the API key, unless the request names another member of your organization with `user_email`. The response says which email was recorded (`recorded_for_email`): if it is not yours, the agent asks you for the email of a member of your organization and never guesses it.

## Technical questions

When the person asks how something works (how models are judged, how the router picks a model, the baseline, prompt caching, fallbacks), answer from the **How it works** pages rather than guessing: [Adaptive routing](https://docs.weflayr.com/concepts/adaptive-routing.md), [Benchmarking](https://docs.weflayr.com/concepts/benchmarking.md), [Routing](https://docs.weflayr.com/concepts/routing.md), [Prompt caching](https://docs.weflayr.com/concepts/prompt-caching.md), [Self-heal](https://docs.weflayr.com/concepts/self-heal.md) and [Fallback and model health](https://docs.weflayr.com/concepts/fallback-and-model-health.md). Give the link of the page you answered from.

The agent asks the person twice only: once to confirm the benchmark and the objective (step 1), once to accept the strategy before it changes live routing (step 3). Going through the report is optional.

## 1. Confirm the benchmark and the objective

Use the benchmark the person names. Otherwise [Get benchmarks](https://docs.weflayr.com/api/management/get-benchmarks.md) with `status` `completed` lists them, newest first, without their calls: read each one's `call_names` with [Get benchmark](https://docs.weflayr.com/api/management/get-benchmark.md) and propose the latest for their call. Only a `completed` or `archived` benchmark has a report. A `draft`, `running` or `stopped` one has none yet: tell the person, point to [Start a benchmark](https://docs.weflayr.com/agents/start-a-benchmark.md) to follow it, and stop.

Read its report and its `suggested_routing_strategies` with [Get benchmark](https://docs.weflayr.com/api/management/get-benchmark.md). When its `blockers` say that no optimization objective recommends a model, or that the benchmark offers no model a strategy can route to, tell the person: a strategy made from it would keep their calls on the baseline and save nothing. Then stop. Otherwise ask the person, in one message:

* **Which benchmark:** the one proposed, with its report's `url`.
* **Which objective to route for:** the objectives the routing strategy page offers, in one line each with its recommended mix, its quality against the baseline and its savings per month. An objective of `suggested_routing_strategies` whose `offered_objective` is another one is not offered (Best quality, when it would route as Same cost, best quality). An objective whose `document` is null cannot be routed for either: none of the benchmark's deployments can be routed to for it (no longer served, too expensive at today's prices, or lacking a capability the calls use). Propose the benchmark's first objective, as the page does, or the one it is offered as.
* **Whether to go through the results first**, or straight to the strategy.

## 2. Walk through the report (optional)

Only when the person asked for it. The report gives, for each optimization objective, the recommended mix of models with each one's share, the quality against the baseline, the cost and the savings per month. [Get benchmark examples](https://docs.weflayr.com/api/management/get-benchmark-examples.md) shows the calls behind the figures.

Present the report as it measures it: the baseline is production (the model or models that served the calls) or the model picked as baseline, and the judges are the evaluators picked. Do not guess why a result looks good or bad, and show the examples if the person wants to read the answers. A model that matches or beats the baseline at a lower cost is what a benchmark is meant to find.

## 3. Show the strategy and ask the person to accept it

[Get benchmark](https://docs.weflayr.com/api/management/get-benchmark.md) with `suggested_routing_strategies` returns, for the objective picked, a strategy document made from the benchmark and ready to push: keep it as it is, named after the call. Its `slug` is empty, so it is the name made into a slug (lowercase, spaces turned into `-`) once saved. A slug stays taken by an archived strategy too, and saving a new strategy with a taken slug is refused (409 `slug_taken`), so first list the strategies with [Get routing strategies](https://docs.weflayr.com/api/management/get-routing-strategies.md) and `status` `all`. When one already has that slug (usually an earlier strategy for the same call), ask the person whether to save the next version of it (its slug in the document, and its live `version` as `expected_version`) or to give the new one another name. Saving a version of an archived strategy makes it live again: say so when asking.

In one message, give the person:

* its name and the slug the calls will send as their model (`strategy/<slug>`),
* the models it routes to,
* what it costs per month and how good its answers are against the baseline, worked out as below. The suggested document leaves out the mix's models that the gateway no longer serves, that cost too much at today's prices or that lack a capability the calls use, so work it out from its actual picks rather than assuming it is the objective's recommendation,
* that saving it changes live routing at once for the calls that send `strategy/<slug>`.
* what to do with the cache breakpoints their code sends (`cache_control` or `prompt_cache_breakpoint`), when the document applies the benchmark's caching (`apply_auto_caching_from_benchmark`, on in the suggested document). Read `user_cache_breakpoints` in `suggested_routing_strategies.routable_models` for each model the strategy routes to, overrides included:
  * any `kept`: tell them to **keep** their breakpoints, naming those models. The cheapest caching the benchmark found on them uses those breakpoints as they are, so removing them makes those models' calls cost more.
  * every non-null one `replaced`: tell them the gateway replaces their breakpoints with its own on every model, so they **can remove** them from their code. They must add them back if they later turn the caching off or route to a model whose value is `kept`.
  * all null (none of the benchmark's calls carried breakpoints): say nothing about them.
* when the project's customers carry tags ([Get tags](https://docs.weflayr.com/api/management/get-tags.md), `customer_tags`), a proposal of override rules for the segments they define, for example the customers tagged `plan:enterprise` routed for Best quality while the others route for the cheapest. Such a rule is a `customer_selection` override rule whose `included_customers` are the customers [Get customers](https://docs.weflayr.com/api/management/get-customers.md) returns for that `tag`. Say that the list is the one of today: a customer tagged later is not in it until a new version is saved, and the predicted cost covers the global rule only. Route each override rule for an objective that is offered (its `offered_objective` is itself, so often not Best quality), give it the global rule's `switch_from_cached_model_or_cheaper_better_model_condition`, and take its allowed deployments from that objective's `document`.

Then ask them to accept, or to say what to change (other models, another objective, a name). After a change, predict again and ask again. For each change:

* **Another objective:** start from that objective's own `document`, since each objective can route to different deployments.
* **Other models:** a rule routing for an objective cannot allow a deployment listed in that objective's `not_selectable`, which gives the `reason` (it costs too much against the baseline, or lacks a capability the calls use). Tell the person why instead of adding it.
* **Fixed order:** a `fixed_order` rule takes `null` as its `switch_from_cached_model_or_cheaper_better_model_condition`, and a rule routing for an objective needs one (`same_or_better_quality_is_unlikely` in the suggested document).

When a save is refused with a 422 naming deployments (a price changed since the benchmark can make a deployment `not_selectable`), remove them or switch that rule to `fixed_order`, and tell the person why with the `reason`. Never save it without their acceptance. Once accepted, [Set routing strategy](https://docs.weflayr.com/api/management/set-routing-strategy.md) saves it: give the person its `url`.

### Predict what a strategy costs

The agent works the prediction out from the `report` of [Get benchmark](https://docs.weflayr.com/api/management/get-benchmark.md), with no other call. It reads the picks from the strategy's `global_rule` and its allowed deployments, counting the deployments of one model once. The benchmark's figures are per model: `objectives[].models[]` gives each model's `cost_per_run` (mean USD per call) and `quality` (against the baseline, null for the baseline, which counts as 0% better, 100% same, 0% worse and 100% same or better). A month is `calls_per_month` calls. The report names a production baseline after the model its calls logged (`gpt-4.1-2025-04-14`), while the strategy names it after the gateway's catalog (`gpt-4.1`): match the strategy's model whose `suggested_routing_strategies` entry has `baseline` true to the report's model with `baseline` true. A model's own figures are read from an objective whose entry for it has `full_run` true (or `baseline` true), since the other objectives explored it on a few calls only.

* **Baseline:** its monthly cost is the `cost_per_run` of the model with `baseline` true, times `calls_per_month`.
* **`fixed_order`:** every call goes to the first model, so the prediction is that model's own `cost_per_run` times `calls_per_month`, and its own `quality`. The calls that fall back to the next models are left out. A first model the report does not list cannot be predicted.
* **An optimization objective:** read the objective of `objectives[]` with that `optimization_objective`. Every picked model must be listed in its `models`, since a strategy routing dynamically only routes to the benchmark's models. Nothing can be predicted when the objective is not in the report, when its `recommendation` is null, or when a picked model has a null `mix` (it is outside the recommended mix, and the gateway never sends it a call under that rule). A null `recommendation` has two causes: no model held the baseline's quality (no model has a `mix`), where the gateway keeps every call on the production model, so the strategy saves nothing and the agent suggests not creating it; or the mix was not validated (models have a `mix` but no call was held back to check it), where the gateway still routes with the mix.
  * **The picks are exactly the models with a `mix`:** the prediction is the `recommendation`, its `monthly_cost` and its `quality`.
  * **The picks are part of the mix:** keep the picked models whose `mix.num_calls` is above 0 (none left: nothing can be predicted). The cost per call is the mean of their `mix.cost_per_run` weighted by their `mix.num_calls`, and the quality is the mean of each model's own `quality` (not `mix.quality`) weighted the same way, each percentage rounded to one decimal. The monthly cost is that cost per call times `calls_per_month`. Use `mix.cost_per_run`, not the model's `cost_per_run`: the baseline's holds its model at other reasoning efforts at their own cost. The prediction assumes the router sends each kept model the calls it sent it in the benchmark, which is an approximation.

When nothing can be predicted, the agent tells you why instead of giving a figure.

## 4. Send the calls to the strategy

A strategy routes only the calls sent through the Weflayr gateway with `strategy/<slug>` as their model. Offer to make that change in the code making the call, keeping or removing its cache breakpoints as step 3 told the person. When the call sends a reasoning effort (`reasoning_effort` or `reasoning.effort`), check it is in the `routed_reasoning_efforts` of a model the rule routes to, in the objective's `models` of `suggested_routing_strategies`: otherwise the gateway refuses every such call (400 `reasoning_effort_not_benchmarked`). Offer to remove the effort from the call then, so the router picks the effort with the model. If the call does not go through the gateway yet (it is logged with the telemetry SDK, or imported), say so and point to [Integrate Weflayr](https://docs.weflayr.com/agents/integration-playbook.md) to move it there first.

End with the strategy's `url`, which [Set routing strategy](https://docs.weflayr.com/api/management/set-routing-strategy.md) returns: it opens the strategy on the routing strategies page, so the person can check it there.

## 5. Read it back and save a new version

[Get routing strategy](https://docs.weflayr.com/api/management/get-routing-strategy.md) returns the live version's document and its `version`, and [Get routing strategy results](https://docs.weflayr.com/api/management/get-routing-strategy-results.md) the calls it routed by rule and by route. To change it, save the next version with [Set routing strategy](https://docs.weflayr.com/api/management/set-routing-strategy.md), passing the version you read as `expected_version`. Saving a version of an archived strategy makes it live again, so ask the person first.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.