This page is written for an AI coding agent working inside a user’s codebase. If you are a human, read the Quickstart instead.
What Weflayr needs
Weflayr joins the cost of every LLM call to the revenue it produced, per customer and per feature. To do that it needs three things:- The SDK started once per process, with your AI provider SDKs auto-instrumented.
- Every LLM call running inside a scope that carries a
feature_name(why the call happens) and acustomer_name(who it is for). - Revenue, or key metrics for an internal tool, booked against the same
customer_name. This part is configured outside the code, in the dashboard or through the API.
Rules
- Assume the user has never used Weflayr. Prefer leaving Weflayr-specific concepts out of your questions entirely: ask the plain question (“Is this a product you sell, or an internal tool?”) and map the answer to the Weflayr concept yourself. Only name and explain a concept when the user needs it to answer or to act, and then in one plain sentence.
- Never change what an LLM call does. No model swap, no prompt edit, no parameter change. You are adding telemetry around existing calls.
- Name the features yourself. They come from the code, so do not make the user do it: read what the application does, propose the names, wire them up, and list them in your hand-off for review. If you are in doubt about where a feature starts and stops, or about what to call it, ask rather than guess.
- Never invent a
customer_name. It has to match the identifier on the user’s revenue side, which is outside the code. If the codebase does not tell you, ask (step 9). - Keep the diff small. One
auto_instrumentcall at startup, one metadata scope per feature entry point, no refactor of the surrounding code. - Do not commit the API key. It goes in the environment, like the user’s other secrets.
- Ask before adding a new process, service or background job. Prefer hooking into something that already exists.
Step 1 - Survey the codebase
Do this before writing anything, and report what you found.Step 2 - API key
The user creates a project API key athttps://app.weflayr.com/api-keys/.
Expose it as WEFLAYR_API_KEY using whatever mechanism the codebase already uses for secrets. Add it to .env.example (or the equivalent) so the next developer knows it exists. The SDK reads it from the environment; only pass api_key explicitly if the codebase never uses environment variables.
.env
WEFLAYR_API_KEY per environment. One API key means one Weflayr project, so a staging deployment sharing the production key mixes its costs into the same dashboard and skews what the optimisation engine sees. If the codebase gives you no way to tell environments apart, set the key for production only.
Step 3 - Install and auto-instrument
Both SDKs run server-side only, on Node.js or Python.Pick the packages
Install with the tool the repository already uses (
npm/pnpm/yarn, pip/uv/poetry).
If the codebase uses an AI framework
Instrument the provider SDK underneath the framework, not the framework. LangChain, LlamaIndex, Haystack, CrewAI, LangGraph and Mastra all call the provider SDK in the end, so the provider instrumentation catches every call the chain or agent makes, including tool-calling loops. There is nothing framework-specific to install. The one exception is the Vercel AI SDK in Node.js, which Weflayr instruments natively:aiv7:autoInstrument({ aiSdks: 'vercel_ai_gateway' })and install@ai-sdk/otel.aiv5 or v6:autoInstrument({ aiSdks: [] })and passexperimental_telemetry: { isEnabled: true }on every call.- Calls made through a provider package with the user’s own key (for example
@ai-sdk/openai) are attributed to that provider and priced at its rates. Calls through the gateway are priced at the gateway’s listed price plus its ~3% markup.
Where to put the call
Oneauto_instrument call per process, at startup, before the provider SDK is loaded. The instrumentation patches the provider module as it is imported, so a client created earlier is not traced.
Put the call in the entry point that already exists. Give it a file of its own only when you have to: several processes that would otherwise each carry a copy of the configuration, or a Node.js ESM entry point, which has to be preloaded (see below).
-
Node.js, CommonJS:
await autoInstrumentinside anasyncentry function, thenrequirethe provider SDKs. -
Node.js, ESM: put the call in a preload module that also registers OpenTelemetry’s ESM loader hook, and start the app with
node --import ./instrumentation.mjs:Installinstrumentation.mjsimport-in-the-middlealongside the instrumentation packages. -
TypeScript: follow the CommonJS or the ESM rule depending on what
moduleintsconfig.jsoncompiles to. - Python: at the top of the entry point, before the module that builds the provider client is imported.
-
Several processes (server + workers + cron): this is when a file of its own earns its place. Put the call in one shared module (
observability.py,instrumentation.ts) and import it first from each entry point, rather than copying the configuration into each. -
Serverless / Lambda: same call at module scope, plus
export_mode="immediate"andflush()(step 7).
If calls go through an OpenAI-compatible endpoint
Some providers are reached through the OpenAI SDK with a custombase_url. Instrument openai, then set provider_name_override on the metadata scope so the call is priced with the right provider’s rates (step 7):
- Mistral in Node.js: OpenAI SDK against
https://api.mistral.ai/v1, withproviderNameOverride: weflayr.AIProviderName.MISTRAL. - Vercel AI Gateway in Python: OpenAI SDK against
https://ai-gateway.vercel.sh/v1, withprovider_name_override=weflayr.AIProviderName.VERCEL_AI_GATEWAY. - Azure OpenAI: use the OpenAI SDK’s
AzureOpenAIclient. Weflayr detects the Azure endpoint on its own, so no override is needed.
OpenAI streaming
If the codebase streams responses from the OpenAI SDK directly, token usage is only returned when it is asked for. Without it, those calls arrive with no token counts and no cost. Add it:Step 4 - Name the features
A feature is the product-side job that caused the LLM call: something the user would recognise, something you would put on a pricing page or a roadmap line. It is the row you will read on the Cost page. Naming features this way keeps the business view separate from the technical implementation. One run of a feature may make 1 LLM call or 10, and that can change next month, but the dashboard keeps answering the same question: one run of this feature costs this much. Every call it makes is summed under it, so refactoring the code does not move the number. A feature is not a function name, not a single LLM call, and not a model name.Pick the level
Wrap the outermost function that corresponds to one user-visible job. Every LLM call nested inside it, at any depth, inherits the metadata, so you only need one scope per feature and not one per call.
If a feature has internal steps worth separating (retrieval, drafting, judging), keep one feature and split the steps with
extra_tags (step 7). Do not create a feature per step.
Most applications end up with somewhere between a handful and a few dozen features. If your list is much longer, you picked too low a level.
Naming rules
- kebab-case, lowercase, stable over time.
- Describes the job, not the implementation:
document-summary, notopenai-summariser-v2. - No ids, dates, versions, model names or environment names in the name. Those belong in tags.
- Short enough to read in a dashboard row, self-explanatory to someone who has not read the code.
- Once a name is live it becomes a dashboard row and a benchmark unit. Renaming it splits its history, so choose a name that will still be right in six months.
Name the features yourself from what the code does and wire them up. You only need to ask the user when the code genuinely does not reveal the product boundary: a generic
run_agent() entry point that serves several unrelated jobs, or a name that hinges on how they sell the product. Otherwise, list the names in your hand-off (step 10) so they can rename any of them before traffic accumulates.
Step 5 - Place the metadata scope
propagate_metadata / propagateMetadata opens a scope. Every instrumented LLM call made inside it, including nested and awaited calls, is stamped with the same metadata and the same correlation id.
Decorator or context manager
Pick with one question: does any value change per request?customer_namecomes from the request in almost every multi-tenant application, so use the context manager / callback form. The decorator fixes its metadata when the function is defined, which cannot express a per-request customer.- Use the decorator only when every value is a constant: single-tenant applications, internal tools, batch jobs that always run for the same entity.
@propagateMetadata(...) decorator syntax only works on class methods with experimentalDecorators enabled. Everywhere else use propagateMetadata(options, fn) or propagateMetadata(options)(fn).
Where to open the scope
- Web handler: inside the route handler or the service function it calls, once the authenticated user or tenant is resolved. Middleware works too if the tenant is known there and the feature name can be derived from the route.
- Background job / queue consumer: inside the job function, around the whole job body.
- Agent or chain: around the top-level
invoke/runcall. Every model call the loop makes inherits the metadata. - Script or one-off task: around
main().
Async and concurrency
The scope is carried bycontextvars (Python) and OpenTelemetry context (Node.js), so it survives await and isolates correctly across concurrent tasks and threads. One thing it does not cover: a call fired into a detached background task that outlives the block. If the codebase does that, open the scope inside the task instead.
Step 6 - Fill customer_name
customer_name is required. It is the string Weflayr uses to join AI cost to revenue, so it has to match the identifier on the revenue side exactly.
Find it in the codebase, in this order
- The billing entity: organisation, workspace, account, tenant. Prefer this, because it is what gets invoiced.
- The authenticated user, if the product bills per user.
- The owner of the API key, for a machine-facing API.
- An existing Stripe customer id or name, if there is billing code.
If you cannot find it
Stop and ask the user (step 9, question 3). Present the candidates you found and where they live in the code. Do not invent a value, do not fall back to"unknown", and do not use email addresses or raw database ids unless the user confirms that is what their revenue rows will carry.
If the application genuinely serves one customer only (an internal tool, a single-tenant deployment), a constant is fine, but ask first: an internal tool is usually better tracked as an internal project, where the equivalent field is your own user rather than a paying customer.
Step 7 - The other parameters, and when they matter
provider_name_override / providerNameOverride
Tells Weflayr which provider to bill the call to when the SDK on the wire is not the provider being paid. Get this wrong and the call is priced at the wrong rates, so cost and margin are wrong.
Accepted values: OPENAI, AZURE_OPENAI, ANTHROPIC, BEDROCK, COHERE, ELEVENLABS, MISTRAL, VERCEL_AI_GATEWAY, GEMINI, VERTEX.
extra_tags / extraTags
Per-scope dimensions that become filters on the dashboard. Values must be strings, numbers or booleans.
Use them for anything you want to slice cost by that is neither the feature nor the customer: the step inside a feature (step: "retrieval"), the plan tier, the request source (web / api / mobile), a prompt version, an experiment arm.
Do not put high-cardinality values in tags: user ids, request ids, timestamps, raw prompt text. A tag should have a handful of distinct values, not thousands. customer_name already covers the per-customer breakdown.
default_tags / defaultTags
Set once in auto_instrument, stamped on every call from that process. Use them for what is constant for the whole process: environment, service name, region, release version. The environment tag is the one that pays off first, because it lets you keep staging traffic out of your production numbers.
export_mode / exportMode and flush()
"batch" (the default) buffers spans and is right for long-running servers. In short-lived runtimes (Lambda, scripts, one-off jobs) buffered spans can be lost on exit, so use "immediate" and call flush() before the process ends.
Step 8 - Verify
Run the app and make one real LLM call through each feature you instrumented, then check the data arrived:Step 9 - Questions to ask the user
Four questions, asked in one message, with your proposed answers filled in where the codebase already told you. Phrase them for someone who has never opened Weflayr: ask in plain words, without the Weflayr terms, and use the explanations below only where the user needs one to answer. Do not guess your way past questions 2 and 3. Once they are answered the setup is complete and the user is live. 1. Is this a product you sell, or an internal tool? A normal project tracks AI cost against revenue. An internal project tracks it against the key metrics the tool drives (meetings booked, tickets resolved) and its accounts are your own users instead of customers. The project type is chosen in the dashboard, under Switch project. 2. Where does your revenue come from? This decides how revenue reaches Weflayr, and it is the half of the picture the code cannot provide.- Stripe: nothing to code. They connect Stripe from Costs & revenue in the dashboard sidebar and issued invoices sync automatically, credit notes deducted. The one thing to check is the join key: the
customer_nameyou stamp on calls must equal the Stripe customer name, or they set aweflayr_customer_namemetadata field on each Stripe customer. See Sync Revenue. - Another billing system: push revenue with the Revenue API. Find the cleanest hook in what already exists rather than building something new, in this order of preference: an existing invoice or subscription webhook handler, an existing nightly billing job, an existing data export or reporting job, and only as a last resort a small standalone script. Also propose a one-off backfill of past months so the Margin page is not empty. Show them the code before writing it.
- No revenue yet, or revenue tracked in a spreadsheet: CSV upload from Costs & revenue, no code.
- Internal project: key metrics instead of revenue, by CSV or through the Key metrics API.
Acme Corp or org_8123).
4. Review the feature names.
Not a blocking question: you named them in step 4 and wired them up. Show the list, one line each with the code path it wraps, and say that renaming later splits the history of that dashboard row, so now is the cheap moment to object.