Skip to main content

0.10.0 (2026-11-19)

Changes

  • OpenAI’s ultrafast service tier: a call with service_tier: "ultrafast" is sent to OpenAI’s Responses API or Bedrock’s OpenAI Responses API on a model priced in that tier, and billed at its ultrafast prices. With self-healing, a /v1/chat/completions call asking for it is sent to the Responses API, which is the only one serving it. A call to a routing strategy asking for flex, priority or ultrafast now goes only to the strategy’s models priced in that tier. If none is, it is refused with 400 service_tier_not_offered, or the tier is removed when the request lists service_tier in drop_inputs_if_unsupported
  • each call records the endpoint it was sent to (/v1/chat/completions, /v1/responses or /v1/embeddings)

0.9.0 (2026-11-19)

Bug Fixes

  • a flex or priority call is sent only to a region with a price in that service tier, and GET /v1/models no longer offers those tiers in a region that has none (Azure’s US and EU data zone deployments, which have no flex price)

Changes

  • a Chat Completions call to an OpenAI or Azure model is sent to the Responses API when the model catalog records that the provider’s Chat Completions API does not serve that model, or refuses its function tools at the call’s reasoning_effort, for every model of the catalog, newly added ones included

0.8.0 (2026-11-18)

Bug Fixes

  • a streamed Responses answer the gateway changed works with the OpenAI SDKs’ responses.stream(), its warnings now in the Response
  • streamed Responses answers keep phase and every provider field

Changes

  • the AI gateway keeps the provider’s prompt cache when only the reasoning effort changes

0.8.0 (2026-11-18)

Bug Fixes

  • a streamed Responses answer the gateway changed works with the OpenAI SDKs’ responses.stream(), its warnings now in the Response

Changes

  • the AI gateway keeps the provider’s prompt cache when only the reasoning effort changes

0.7.0 (2026-11-18)

Bug Fixes

  • model benchmark calls go to the gateway’s Responses API, avoiding OpenAI’s 500s on streamed flex Chat Completions calls

Changes

  • A/B tests split the calls that send their slug between two routes, from the dashboard, the public API and the MCP server

0.6.0 (2026-11-18)

Changes

  • Chat Completions callers reach OpenAI and Azure Chat Completions

0.5.0 (2026-11-17)

Bug Fixes

  • the gateway checks providers’ TLS certificates with patched versions of its certificate verification libraries, which fix known vulnerabilities

0.4.0 (2026-11-17)

Changes

  • a promotional discount lowers the cost of a call, and the credits it uses, only once Weflayr has checked it, so a discount read wrongly from a provider’s pricing page is never applied

0.3.0 (2026-11-16)

Bug Fixes

  • the gateway passes on DeepSeek’s error for a call too long for the model
  • a Claude strict tool’s optional nullable arguments stay optional, only a structured output’s are made required
  • a failed provider call’s error says which model and region failed
  • a routing strategy routes dynamically among any benchmarked models it allows, for any optimization objective
  • a structured output schema leaving additionalProperties unset is no longer refused by Claude models
  • a structured output schema using keywords Claude models do not take (minimum, maxLength, pattern…) is no longer refused by them
  • a structured output schema with a property named like a schema keyword (e.g. enum) is no longer refused by Claude models
  • a structured output schema with many optional nullable properties is no longer refused by Claude models as too complex
  • an invalid request is a 400, not a 500
  • gateway streams cut short by the provider now end as errors
  • gateway streams no longer cut at 10 minutes
  • Claude models get their full output limit on every provider
  • streamed calls with thinking are recorded like non-streamed ones
  • the AI gateway answers a provider answer it cannot read with a 502 provider error, which clients retry, instead of a 400 blaming the caller
  • the AI gateway ends a provider stream that stops before its finish reason or usage with an error saying the call was not billed
  • the AI gateway no longer asks GPT-5.4 and GPT-5.5 on Bedrock for a reasoning summary
  • the gateway no longer sends Claude the empty text of an assistant message calling tools, which Anthropic, Bedrock and Vertex refused with a 400
  • the gateway raises a max_tokens below the fewest output tokens a model takes to that minimum, instead of refusing the call
  • the gateway removes the stop sequences a model refuses
  • a burst of requests no longer waits for the gateway to load its model catalog
  • a Claude request with many nullable properties is no longer slow
  • the gateway finishes a provider call after the client disconnects
  • Gemini answers requests with function tools and web search
  • Responses calls answer when a provider returns several choices
  • the AI gateway’s 402 insufficient_credits error says that calls made with your own provider keys need credit too, for Weflayr’s fee

Changes

  • a gateway request naming a model retries its first deployment on retry_on_error and tries unhealthy deployments after the others, as a routing strategy does
  • a gateway request naming no provider or no region falls back to the model’s other providers and regions when its first call fails
  • a gateway request to turn thinking off, on a model that always thinks, now runs it at the lowest thinking level it takes instead of its default level
  • OpenAI’s 12-hour cache breakpoint works through the gateway and is billed at its own price
  • a fixed order routing strategy can send each model’s calls at a chosen reasoning effort
  • GET /v1/models lists the discount in force today, with its end date
  • routing strategies route for the benchmark’s optimization objectives
  • the AI gateway accepts a cache marker written as Anthropic’s cache_control or as OpenAI’s prompt_cache_breakpoint, whatever the provider
  • the gateway removes a top_p, a top_k, log probabilities, several choices, stop sequences and penalties the model does not take
  • a request to turn thinking off on a Claude model no longer gets an answer the model thought before
  • the AI Router picks the cheapest model likely as good as or better than the baseline, and a routing rule can keep a request on the model holding its prompt cache unless a cheaper or better model is likely
  • GET /v1/models lists the routing strategies and their versions when it receives an API key
  • GET /v1/models lists modalities in one list, and each deployment’s data terms and output speed
  • organizations can store their own provider keys (bring your own key), which the gateway calls providers with
  • a gateway request can list several models, providers and regions, and the gateway calls the next combination when one cannot be called or fails
  • the gateway enforces API key rate and spend limits
  • a gateway call can set its routing strategy’s retry on error
  • the gateway lists its models on GET /v1/models
  • a Bedrock model served through an EU or US inference profile has one deployment per geography, named EU or US, and is not shown as zero data retention
  • the gateway writes the prompt cache of Claude, GPT-5.6 and Kimi K3 automatically
  • each organization has request and token rate limits the gateway enforces

0.2.0 (2026-10-06)

Bug Fixes

  • Mistral embeddings accept a bare text input
  • the AI gateway handles image and file inputs each provider can read, and refuses the others with a 400 naming the field
  • stop the gateway from rejecting simultaneous requests that share an API key

Changes

  • add message content retention control
  • support embeddings through /v1/embeddings
  • gateway calls are billed from the organization’s credit balance
  • the gateway no longer serves Google Gemini embeddings
  • support the Anthropic provider
  • support the Google AI provider
  • refuse gateway calls from a key without the gateway scope
  • support the Bedrock provider
  • support the Mistral provider
  • support the Azure OpenAI provider in the gateway
  • refuse provider calls when the credit balance is exhausted
  • take extended thinking through OpenRouter’s reasoning field
  • support the xAI provider
  • POST /v1/responses for every provider, with reasoning summaries and continuity
  • the gateway routes to Google Gemini 3 and Gemma 4
  • name the provider with the body’s provider field, or leave it out to let the gateway resolve the model through its catalog
  • the AI gateway rejects a chat or embeddings request whose region is not EU, US or GLOBAL
  • the gateway calls a model in the region a request names, and refuses a region the model is not available or priced in
  • the AI gateway checks what a model accepts (images, audio, PDFs, tools, structured output) from the model catalog
  • the AI gateway supports structured output on Bedrock and Vertex Anthropic
  • the AI gateway answers a request holding audio or video with an error pointing to support
  • the AI gateway removes the request inputs a model does not take, as the new self_heal field asks
  • web search for OpenAI, Anthropic, Gemini and xAI
  • the AI gateway routes a flex or priority request only to a provider with a price for that service tier
  • price each call by the region it ran in
  • support Vertex AI models
  • name each call with set_name_call and send a call’s metadata to the AI gateway in the x-weflayr-metadata header
  • the AI gateway calls Azure OpenAI on its v1 Responses API
  • the AI gateway calls xAI on its Responses API, and refuses with a 400 naming the field each request field xAI would ignore or refuse

0.1.0 (2026-09-03)

Changes

  • first release of the AI Gateway