This page is written for an AI coding agent working inside a user’s codebase. If you are a human, read the Quickstart instead.
What Weflayr needs
Weflayr benchmarks other models on the application’s real LLM calls, recommends the ones that answer as well for less, and routes the calls to them. It also joins the cost of every call to the revenue it produced, per customer and per feature. To do that it needs:- The LLM calls. Preferably sent through the Weflayr gateway: an OpenAI-compatible endpoint that calls any model of any provider, logs the call, and can later follow a routing strategy. When the calls must keep going to the providers directly, the telemetry SDK records them instead.
- What each call is for: a
feature_name(the product job), acall_name(the LLM call inside it) and acustomer_name(who it is for). - Revenue, or key metrics for an internal tool, booked against the same
customer_name. This part is configured outside the code, in the dashboard or through the API.
Rules
- Assume the user has never used Weflayr. Prefer leaving Weflayr-specific concepts out of your questions entirely: ask the plain question (“Is this a product you sell, or an internal tool?”) and map the answer to the Weflayr concept yourself. Only name and explain a concept when the user needs it to answer or to act, and then in one plain sentence.
- Never change what an LLM call does. Keep the same model, prompt and parameters. Moving a call to the gateway changes where it is sent, not what it asks.
- Do not move calls to the gateway without the user’s go-ahead (step 2): gateway calls are paid from the user’s Weflayr credits instead of their provider accounts.
- Name the features and calls yourself. They come from the code, so do not make the user do it: read what the application does, propose the names, wire them up, and list them in your hand-off for review. If you are in doubt about where a feature starts and stops, or about what to call it, ask rather than guess.
- Never invent a
customer_name. It has to match the identifier on the user’s revenue side, which is outside the code. If the codebase does not tell you, ask (step 9). - Keep the diff small. One client change or one startup call per process, one metadata scope per feature entry point, no refactor of the surrounding code.
- Do not commit the API key. It goes in the environment, like the user’s other secrets.
- Ask before adding a new process, service or background job. Prefer hooking into something that already exists.
Step 1 - Survey the codebase
Do this before writing anything, and report what you found.Step 2 - Choose how calls reach Weflayr
Propose the gateway, and ask the user before switching (step 9, question 1). Calls made with a provider’s own SDK (Anthropic, Bedrock, Gemini…) reach the gateway only once rewritten for the OpenAI client: propose that as a separate change, or keep those calls on the SDK path. A codebase can mix both.
Step 3 - Send the calls
Gateway
The user creates an API key with the Gateway scope athttps://app.weflayr.com/api-keys/ and adds credits at https://app.weflayr.com/credits/. Expose the key as WEFLAYR_API_KEY with the mechanism the codebase already uses for secrets, and add it to .env.example (or the equivalent).
Change only the client’s base URL and key. Keep every call as it is:
- Model names: the gateway takes the catalog names (e.g.
gpt-5.6,claude-opus-5-5) and picks the provider; passproviderin the body to pin one. A name the catalog does not know returns400 unknown_model: list it in your hand-off. - Fields a model does not take (e.g.
temperatureon a reasoning model) are removed and listed inweflayr_warningsby default (self_heal). See Responses for every field. - Metadata (step 5): turn the header on once at startup, then wrap calls in
propagate_metadataas for the SDK:
x-weflayr-metadata: {"feature_name": "support-chat"} as a header, and "weflayr": {"customer_name": "...", "call_name": "..."} in the body (extra_body in Python).
Telemetry SDK
ExposeWEFLAYR_API_KEY from a key with the Management scope, then install the packages and call auto_instrument once per process.
Install with the tool the repository already uses (
npm/pnpm/yarn, pip/uv/poetry). With an AI framework, instrument the provider SDK underneath it: LangChain, LlamaIndex, Haystack, CrewAI and LangGraph call the provider SDK in the end, so every call of the chain is caught.
One auto_instrument call per process, at startup, before the provider SDK is loaded. The instrumentation patches the provider module as it is imported, so a client created earlier is not traced.
Put the call in the entry point that already exists. Give it a file of its own only when you have to: several processes that would otherwise each carry a copy of the configuration, or a Node.js ESM entry point, which has to be preloaded (see below).
-
Node.js, CommonJS:
await autoInstrumentinside anasyncentry function, thenrequirethe provider SDKs. -
Node.js, ESM: put the call in a preload module that also registers OpenTelemetry’s ESM loader hook, and start the app with
node --import ./instrumentation.mjs:Installinstrumentation.mjsimport-in-the-middlealongside the instrumentation packages. -
TypeScript: follow the CommonJS or the ESM rule depending on what
moduleintsconfig.jsoncompiles to. - Python: at the top of the entry point, before the module that builds the provider client is imported.
-
Several processes (server + workers + cron): this is when a file of its own earns its place. Put the call in one shared module (
observability.py,instrumentation.ts) and import it first from each entry point, rather than copying the configuration into each. -
Serverless / Lambda: same call at module scope, plus
export_mode="immediate"andflush()(step 7).
provider_name_override_if_telemetry_instrumentation=MISTRAL on the scope (step 7). Any other OpenAI-compatible endpoint is priced as OpenAI: flag it in your hand-off.
OpenAI streaming
If the codebase streams responses from the OpenAI SDK directly, token usage is only returned when it is asked for. Without it, those calls arrive with no token counts and no cost. Add it:Step 4 - Name the features and calls
A feature is the product-side job that caused the LLM call: something the user would recognize, something you would put on a pricing page or a roadmap line. It is the row you will read on the AI Cost page. Naming features this way keeps the business view separate from the technical implementation. One run of a feature may make 1 LLM call or 10, and that can change next month, but the dashboard keeps answering the same question: one run of this feature costs this much. Every call it makes is summed under it, so refactoring the code does not move the number. A feature is not a function name, not a single LLM call, and not a model name.Pick the level
Wrap the outermost function that corresponds to one user-visible job. Every LLM call nested inside it, at any depth, inherits the metadata, so you only need one scope per feature and not one per call.
If a feature has internal steps worth separating (retrieval, drafting, judging), keep one feature and give each step its own call name (see below). Do not create a feature per step.
Most applications end up with somewhere between a handful and a few dozen features. If your list is much longer, you picked too low a level.
Naming rules
- kebab-case, lowercase, stable over time.
- Describes the job, not the implementation:
document-summary, notopenai-summarizer-v2. - No ids, dates, versions, model names or environment names in the name. Those belong in tags.
- Short enough to read in a dashboard row, self-explanatory to someone who has not read the code.
- Once a name is live it becomes a dashboard row and a benchmark unit. Renaming it splits its history, so choose a name that will still be right in six months.
Name the features yourself from what the code does and wire them up. You only need to ask the user when the code genuinely does not reveal the product boundary: a generic
run_agent() entry point that serves several unrelated jobs, or a name that hinges on how they sell the product. Otherwise, list the names in your hand-off (step 10) so they can rename any of them before traffic accumulates.
Name the calls too
A call name is the LLM call inside a feature:classify-ticket, draft-reply, summarize-thread. A benchmark is set up per call name, so give each distinct prompt its own name, in kebab-case, with the same rules as features. A feature with a single LLM call still gets a call name.
Step 5 - Place the metadata scope
propagate_metadata / propagateMetadata opens a scope. Every LLM call made inside it (sent through the gateway with the header turned on, or recorded by the SDK), including nested and awaited calls, is stamped with the same metadata and the same correlation id.
Decorator or context manager
Pick with one question: does any value change per request?customer_namecomes from the request in almost every multi-tenant application, so use the context manager / callback form. The decorator fixes its metadata when the function is defined, which cannot express a per-request customer.- Use the decorator only when every value is a constant: single-tenant applications, internal tools, batch jobs that always run for the same entity.
set_call_name / setCallName (same forms: context manager or decorator in Python, callable or decorator in Node.js). It keeps the feature’s scope and does not start a new run.
In TypeScript, the @propagateMetadata(...) decorator syntax only works on class methods with experimentalDecorators enabled. Everywhere else use propagateMetadata(options, fn) or propagateMetadata(options)(fn).
Where to open the scope
- Web handler: inside the route handler or the service function it calls, once the authenticated user or tenant is resolved. Middleware works too if the tenant is known there and the feature name can be derived from the route.
- Background job / queue consumer: inside the job function, around the whole job body.
- Agent or chain: around the top-level
invoke/runcall. Every model call the loop makes inherits the metadata. - Script or one-off task: around
main().
Async and concurrency
The scope is carried bycontextvars (Python) and OpenTelemetry context (Node.js), so it survives await and isolates correctly across concurrent tasks and threads. One thing it does not cover: a call fired into a detached background task that outlives the block. If the codebase does that, open the scope inside the task instead.
Step 6 - Fill customer_name
Always set customer_name, on the metadata scope or, for a gateway call, in the weflayr block of the request body. It is the string Weflayr uses to join AI cost to revenue, so it has to match the identifier on the revenue side exactly.
Find it in the codebase, in this order
- The billing entity: organization, workspace, account, tenant. Prefer this, because it is what gets invoiced.
- The authenticated user, if the product bills per user.
- The owner of the API key, for a machine-facing API.
- An existing Stripe customer id or name, if there is billing code.
If you cannot find it
Stop and ask the user (step 9, question 4). Present the candidates you found and where they live in the code. Do not invent a value, do not fall back to"unknown", and do not use email addresses or raw database ids unless the user confirms that is what their revenue rows will carry.
If the application genuinely serves one customer only (an internal tool, a single-tenant deployment), a constant is fine, but ask first: an internal tool is usually better tracked as an internal project, where the equivalent field is your own user rather than a paying customer.
It also selects the customer overrides of a routing strategy (by margin, spend or a list of customers), which only work when the call carries its customer.
Step 7 - The other parameters
extra_tags / extraTags
Per-scope dimensions that become filters on the dashboard. Values must be strings, numbers or booleans.
Use them for anything you want to slice cost by that is neither the feature nor the customer: the step inside a feature (step: "retrieval"), the plan tier, the request source (web / api / mobile), a prompt version, an experiment arm.
Do not put high-cardinality values in tags: user ids, request ids, timestamps, raw prompt text. A tag should have a handful of distinct values, not thousands. customer_name already covers the per-customer breakdown.
SDK path only
provider_name_override_if_telemetry_instrumentation/providerNameOverrideIfTelemetryInstrumentation: the provider to bill when the client on the wire is not the provider being paid (OPENAI,AZURE_OPENAI,ANTHROPIC,BEDROCK,COHERE,ELEVENLABS,MISTRAL,GOOGLE,VERTEX,ALIBABA,SPACEXAI). Ignored for gateway calls.default_tags/defaultTagsinauto_instrument: stamped on every call of the process, e.g. environment and service name.export_mode="immediate"andflush()in short-lived runtimes (Lambda, scripts, one-off jobs), so the last calls are sent before the process ends.
Step 8 - Verify
Run the app and make one real LLM call through each feature you wired, then check the data arrived (with a key that has the Management scope):Step 9 - Questions to ask the user
Five questions, asked in one message, with your proposed answers filled in where the codebase already told you. Phrase them for someone who has never opened Weflayr. Do not guess your way past questions 1, 3 and 4. 1. Can the LLM calls go through Weflayr? Explain in one sentence: Weflayr then calls the models itself, billed from prepaid credits instead of their provider accounts, and can later switch models without a deploy. If they say no, use the telemetry SDK. 2. Is this a product you sell, or an internal tool? A normal project tracks AI cost against revenue. An internal project tracks it against the key metrics the tool drives (meetings booked, tickets resolved) and its accounts are your own users instead of customers. 3. Where does your revenue come from? This decides how revenue reaches Weflayr, and it is the half of the picture the code cannot provide.- Stripe: nothing to code. They connect Stripe from Costs & revenue in the dashboard sidebar and issued invoices sync automatically, credit notes deducted. The one thing to check is the join key: the
customer_nameyou stamp on calls must equal the Stripe customer name, or they set aweflayr_customer_namemetadata field on each Stripe customer. See Sync Revenue. - Another billing system: push revenue with the Revenue API. Find the cleanest hook in what already exists rather than building something new, in this order of preference: an existing invoice or subscription webhook handler, an existing nightly billing job, an existing data export or reporting job, and only as a last resort a small standalone script. Also propose a one-off backfill of past months so the AI Margin page is not empty. Show them the code before writing it.
- No revenue yet, or revenue tracked in a spreadsheet: CSV upload from Costs & revenue, no code.
- Internal project: key metrics instead of revenue, by CSV or through the Key metrics API.
Acme Corp or org_8123).
5. Review the feature and call names.
Not a blocking question: you named them in step 4 and wired them up. Show the list, one line each with the code path it wraps, and say that renaming later splits the history of that dashboard row, so now is the cheap moment to object.