> ## Documentation Index
> Fetch the complete documentation index at: https://docs.weflayr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt caching optimization

> The benchmark finds the best caching configuration for each model, your routing strategy applies it, and you decide when a call leaves the model holding its prompt in cache to keep your quality standard.

Providers bill the start of a prompt they already hold in cache at a fraction of the input price. On agents and long conversations, where each call repeats the previous ones, caching is often the largest cost saving after the model choice. But each provider caches differently, and routing between models can lose the cache at every switch.

Weflayr ensures your use case runs with optimal caching in three steps.

## 1. The benchmark finds the best caching for each model

Each provider has its own rules: whether it caches implicitly or only where a breakpoint is marked, how long a cached prompt lives, the minimum length it caches, where it looks for a cached prefix, how many cache breakpoints a request takes, what a cache write costs, etc. The best setup depends on those rules and on your traffic: how long your prompts are, how they grow from one call to the next, and how much time passes between calls.

So during the [benchmark](https://docs.weflayr.com/product/model-benchmarks), Weflayr replays your calls of the same period in time order against a simulation of each candidate model's cache, and tries different breakpoint placements and cache lifetimes.

It keeps, for each model, the setup that costs the least on your calls, cache writes included. The [benchmark report](https://docs.weflayr.com/product/benchmark-report.md) prices each model both with the caching your calls use today and with this setup.

## 2. Your routing strategy applies it on every call

In a [routing strategy](https://docs.weflayr.com/product/routing-strategies.md) made from a benchmark, you can choose to **apply recommended caching from benchmark** to make the gateway place the recommended breakpoints for the model each call goes to, with no code to change. Your app does not need to know which model answers: a call routed to Claude gets Claude's best breakpoints, the same call routed to GPT gets GPT's.

When it is on, the gateway keeps your app's breakpoints on the models where they cache better than its own setup and replaces them on the others. Mark them with Anthropic's `cache_control` or OpenAI's `prompt_cache_breakpoint`, whatever the model.

## 3. You decide when quality is worth losing the cache

The gateway keeps track of which prompt prefixes each provider is likely to hold in cache for your organization. When the [router](https://docs.weflayr.com/concepts/routing.md) ranks the models for a call, it prices each one with the tokens it would read from its cache, and by default it keeps the call on the model whose cache is warm: switching would lose the cheaper cache reads.

Staying on the cached model is cheaper, but it is not always the right model for the next call: a conversation can start with easy questions a cheap model answers well, then ask a hard one it answers badly. How much quality is worth giving up for the cheaper cache reads depends on your use case, so you choose it. Weflayr scores every call separately, and **When to switch from a model with warm cache** sets, for each rule of your routing strategy, when a call leaves the cached model even if that costs more:

| Setting | The call leaves the model with a warm cache |
| - | - |
| **Never** | Only for a model that is better and still cheaper |
| **When same quality is unlikely** | Also when the cached model is unlikely to match your current model's quality on this call |
| **When best quality is unlikely** | Also when the cached model is unlikely to give the best answer on this call |
| **Anytime** | Whenever the router ranks another model first, the cache ignored |

Whatever the setting, a model with a higher quality score that stays cheaper than the cached model, its cache discount included, is always called first.

## Next

> [card] [Routing strategies](https://docs.weflayr.com/product/routing-strategies.md)
>
> Turn on the benchmark's caching and choose when to switch.

> [card] [Adaptive routing](https://docs.weflayr.com/concepts/adaptive-routing.md)
>
> How the benchmark and the router work together.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.