Skip to main content
This page is written for an AI coding agent working inside a user’s codebase. If you are a human, read the Quickstart instead.
You are integrating Weflayr into this codebase. Follow the steps below in order. Steps 1 to 8 are code work, step 9 is what you have to ask the user, step 10 is the hand-off you must produce. Decide everything you can decide from the code yourself; only ask about what the code cannot tell you.

What Weflayr needs

Weflayr benchmarks other models on the application’s real LLM calls, recommends the ones that answer as well for less, and routes the calls to them. It also joins the cost of every call to the revenue it produced, per customer and per feature. To do that it needs:
  1. The LLM calls. Preferably sent through the Weflayr gateway: an OpenAI-compatible endpoint that calls any model of any provider, logs the call, and can later follow a routing strategy. When the calls must keep going to the providers directly, the telemetry SDK records them instead.
  2. What each call is for: a feature_name (the product job), a call_name (the LLM call inside it) and a customer_name (who it is for).
  3. Revenue, or key metrics for an internal tool, booked against the same customer_name. This part is configured outside the code, in the dashboard or through the API.
Get the naming right and the rest follows: benchmarks are set up per call, routing strategies switch models per customer, and the dashboard shows cost per feature and margin per customer.

Rules

  • Assume the user has never used Weflayr. Prefer leaving Weflayr-specific concepts out of your questions entirely: ask the plain question (“Is this a product you sell, or an internal tool?”) and map the answer to the Weflayr concept yourself. Only name and explain a concept when the user needs it to answer or to act, and then in one plain sentence.
  • Never change what an LLM call does. Keep the same model, prompt and parameters. Moving a call to the gateway changes where it is sent, not what it asks.
  • Do not move calls to the gateway without the user’s go-ahead (step 2): gateway calls are paid from the user’s Weflayr credits instead of their provider accounts.
  • Name the features and calls yourself. They come from the code, so do not make the user do it: read what the application does, propose the names, wire them up, and list them in your hand-off for review. If you are in doubt about where a feature starts and stops, or about what to call it, ask rather than guess.
  • Never invent a customer_name. It has to match the identifier on the user’s revenue side, which is outside the code. If the codebase does not tell you, ask (step 9).
  • Keep the diff small. One client change or one startup call per process, one metadata scope per feature entry point, no refactor of the surrounding code.
  • Do not commit the API key. It goes in the environment, like the user’s other secrets.
  • Ask before adding a new process, service or background job. Prefer hooking into something that already exists.

Step 1 - Survey the codebase

Do this before writing anything, and report what you found.

Step 2 - Choose how calls reach Weflayr

Propose the gateway, and ask the user before switching (step 9, question 1). Calls made with a provider’s own SDK (Anthropic, Bedrock, Gemini…) reach the gateway only once rewritten for the OpenAI client: propose that as a separate change, or keep those calls on the SDK path. A codebase can mix both.

Step 3 - Send the calls

Gateway

The user creates an API key with the Gateway scope at https://app.weflayr.com/api-keys/ and adds credits at https://app.weflayr.com/credits/. Expose the key as WEFLAYR_API_KEY with the mechanism the codebase already uses for secrets, and add it to .env.example (or the equivalent). Change only the client’s base URL and key. Keep every call as it is:
  • Model names: the gateway takes the catalog names (e.g. gpt-5.6, claude-opus-5-5) and picks the provider; pass provider in the body to pin one. A name the catalog does not know returns 400 unknown_model: list it in your hand-off.
  • Fields a model does not take (e.g. temperature on a reasoning model) are removed and listed in weflayr_warnings by default (self_heal). See Responses for every field.
  • Metadata (step 5): turn the header on once at startup, then wrap calls in propagate_metadata as for the SDK:
Without the SDK, send the metadata on the request: x-weflayr-metadata: {"feature_name": "support-chat"} as a header, and "weflayr": {"customer_name": "...", "call_name": "..."} in the body (extra_body in Python).

Telemetry SDK

Expose WEFLAYR_API_KEY from a key with the Management scope, then install the packages and call auto_instrument once per process. Install with the tool the repository already uses (npm/pnpm/yarn, pip/uv/poetry). With an AI framework, instrument the provider SDK underneath it: LangChain, LlamaIndex, Haystack, CrewAI and LangGraph call the provider SDK in the end, so every call of the chain is caught. One auto_instrument call per process, at startup, before the provider SDK is loaded. The instrumentation patches the provider module as it is imported, so a client created earlier is not traced. Put the call in the entry point that already exists. Give it a file of its own only when you have to: several processes that would otherwise each carry a copy of the configuration, or a Node.js ESM entry point, which has to be preloaded (see below).
Practical placement:
  • Node.js, CommonJS: await autoInstrument inside an async entry function, then require the provider SDKs.
  • Node.js, ESM: put the call in a preload module that also registers OpenTelemetry’s ESM loader hook, and start the app with node --import ./instrumentation.mjs:
    instrumentation.mjs
    Install import-in-the-middle alongside the instrumentation packages.
  • TypeScript: follow the CommonJS or the ESM rule depending on what module in tsconfig.json compiles to.
  • Python: at the top of the entry point, before the module that builds the provider client is imported.
  • Several processes (server + workers + cron): this is when a file of its own earns its place. Put the call in one shared module (observability.py, instrumentation.ts) and import it first from each entry point, rather than copying the configuration into each.
  • Serverless / Lambda: same call at module scope, plus export_mode="immediate" and flush() (step 7).
Providers reached through the OpenAI client with another base URL: Azure OpenAI, Alibaba (DashScope) and xAI are detected on their own. For Mistral through the OpenAI client, set provider_name_override_if_telemetry_instrumentation=MISTRAL on the scope (step 7). Any other OpenAI-compatible endpoint is priced as OpenAI: flag it in your hand-off.

OpenAI streaming

If the codebase streams responses from the OpenAI SDK directly, token usage is only returned when it is asked for. Without it, those calls arrive with no token counts and no cost. Add it:

Step 4 - Name the features and calls

A feature is the product-side job that caused the LLM call: something the user would recognize, something you would put on a pricing page or a roadmap line. It is the row you will read on the AI Cost page. Naming features this way keeps the business view separate from the technical implementation. One run of a feature may make 1 LLM call or 10, and that can change next month, but the dashboard keeps answering the same question: one run of this feature costs this much. Every call it makes is summed under it, so refactoring the code does not move the number. A feature is not a function name, not a single LLM call, and not a model name.

Pick the level

Wrap the outermost function that corresponds to one user-visible job. Every LLM call nested inside it, at any depth, inherits the metadata, so you only need one scope per feature and not one per call. If a feature has internal steps worth separating (retrieval, drafting, judging), keep one feature and give each step its own call name (see below). Do not create a feature per step. Most applications end up with somewhere between a handful and a few dozen features. If your list is much longer, you picked too low a level.

Naming rules

  • kebab-case, lowercase, stable over time.
  • Describes the job, not the implementation: document-summary, not openai-summarizer-v2.
  • No ids, dates, versions, model names or environment names in the name. Those belong in tags.
  • Short enough to read in a dashboard row, self-explanatory to someone who has not read the code.
  • Once a name is live it becomes a dashboard row and a benchmark unit. Renaming it splits its history, so choose a name that will still be right in six months.
Name the features yourself from what the code does and wire them up. You only need to ask the user when the code genuinely does not reveal the product boundary: a generic run_agent() entry point that serves several unrelated jobs, or a name that hinges on how they sell the product. Otherwise, list the names in your hand-off (step 10) so they can rename any of them before traffic accumulates.

Name the calls too

A call name is the LLM call inside a feature: classify-ticket, draft-reply, summarize-thread. A benchmark is set up per call name, so give each distinct prompt its own name, in kebab-case, with the same rules as features. A feature with a single LLM call still gets a call name.

Step 5 - Place the metadata scope

propagate_metadata / propagateMetadata opens a scope. Every LLM call made inside it (sent through the gateway with the header turned on, or recorded by the SDK), including nested and awaited calls, is stamped with the same metadata and the same correlation id.

Decorator or context manager

Pick with one question: does any value change per request?
  • customer_name comes from the request in almost every multi-tenant application, so use the context manager / callback form. The decorator fixes its metadata when the function is defined, which cannot express a per-request customer.
  • Use the decorator only when every value is a constant: single-tenant applications, internal tools, batch jobs that always run for the same entity.
Name each call inside the scope with set_call_name / setCallName (same forms: context manager or decorator in Python, callable or decorator in Node.js). It keeps the feature’s scope and does not start a new run. In TypeScript, the @propagateMetadata(...) decorator syntax only works on class methods with experimentalDecorators enabled. Everywhere else use propagateMetadata(options, fn) or propagateMetadata(options)(fn).

Where to open the scope

  • Web handler: inside the route handler or the service function it calls, once the authenticated user or tenant is resolved. Middleware works too if the tenant is known there and the feature name can be derived from the route.
  • Background job / queue consumer: inside the job function, around the whole job body.
  • Agent or chain: around the top-level invoke / run call. Every model call the loop makes inherits the metadata.
  • Script or one-off task: around main().

Async and concurrency

The scope is carried by contextvars (Python) and OpenTelemetry context (Node.js), so it survives await and isolates correctly across concurrent tasks and threads. One thing it does not cover: a call fired into a detached background task that outlives the block. If the codebase does that, open the scope inside the task instead.

Step 6 - Fill customer_name

Always set customer_name, on the metadata scope or, for a gateway call, in the weflayr block of the request body. It is the string Weflayr uses to join AI cost to revenue, so it has to match the identifier on the revenue side exactly.

Find it in the codebase, in this order

  1. The billing entity: organization, workspace, account, tenant. Prefer this, because it is what gets invoiced.
  2. The authenticated user, if the product bills per user.
  3. The owner of the API key, for a machine-facing API.
  4. An existing Stripe customer id or name, if there is billing code.
Prefer the identifier that already appears in the billing system. If the product bills organizations, do not use the individual user: the AI Margin page would then show one row per seat and no revenue against most of them.

If you cannot find it

Stop and ask the user (step 9, question 4). Present the candidates you found and where they live in the code. Do not invent a value, do not fall back to "unknown", and do not use email addresses or raw database ids unless the user confirms that is what their revenue rows will carry. If the application genuinely serves one customer only (an internal tool, a single-tenant deployment), a constant is fine, but ask first: an internal tool is usually better tracked as an internal project, where the equivalent field is your own user rather than a paying customer. It also selects the customer overrides of a routing strategy (by margin, spend or a list of customers), which only work when the call carries its customer.

Step 7 - The other parameters

extra_tags / extraTags

Per-scope dimensions that become filters on the dashboard. Values must be strings, numbers or booleans. Use them for anything you want to slice cost by that is neither the feature nor the customer: the step inside a feature (step: "retrieval"), the plan tier, the request source (web / api / mobile), a prompt version, an experiment arm. Do not put high-cardinality values in tags: user ids, request ids, timestamps, raw prompt text. A tag should have a handful of distinct values, not thousands. customer_name already covers the per-customer breakdown.

SDK path only

  • provider_name_override_if_telemetry_instrumentation / providerNameOverrideIfTelemetryInstrumentation: the provider to bill when the client on the wire is not the provider being paid (OPENAI, AZURE_OPENAI, ANTHROPIC, BEDROCK, COHERE, ELEVENLABS, MISTRAL, GOOGLE, VERTEX, ALIBABA, SPACEXAI). Ignored for gateway calls.
  • default_tags / defaultTags in auto_instrument: stamped on every call of the process, e.g. environment and service name.
  • export_mode="immediate" and flush() in short-lived runtimes (Lambda, scripts, one-off jobs), so the last calls are sent before the process ends.

Step 8 - Verify

Run the app and make one real LLM call through each feature you wired, then check the data arrived (with a key that has the Management scope):
Your feature names and customer names should be listed after a minute or two. The calls also show up live on the Overview.

Step 9 - Questions to ask the user

Five questions, asked in one message, with your proposed answers filled in where the codebase already told you. Phrase them for someone who has never opened Weflayr. Do not guess your way past questions 1, 3 and 4. 1. Can the LLM calls go through Weflayr? Explain in one sentence: Weflayr then calls the models itself, billed from prepaid credits instead of their provider accounts, and can later switch models without a deploy. If they say no, use the telemetry SDK. 2. Is this a product you sell, or an internal tool? A normal project tracks AI cost against revenue. An internal project tracks it against the key metrics the tool drives (meetings booked, tickets resolved) and its accounts are your own users instead of customers. 3. Where does your revenue come from? This decides how revenue reaches Weflayr, and it is the half of the picture the code cannot provide.
  • Stripe: nothing to code. They connect Stripe from Costs & revenue in the dashboard sidebar and issued invoices sync automatically, credit notes deducted. The one thing to check is the join key: the customer_name you stamp on calls must equal the Stripe customer name, or they set a weflayr_customer_name metadata field on each Stripe customer. See Sync Revenue.
  • Another billing system: push revenue with the Revenue API. Find the cleanest hook in what already exists rather than building something new, in this order of preference: an existing invoice or subscription webhook handler, an existing nightly billing job, an existing data export or reporting job, and only as a last resort a small standalone script. Also propose a one-off backfill of past months so the AI Margin page is not empty. Show them the code before writing it.
  • No revenue yet, or revenue tracked in a spreadsheet: CSV upload from Costs & revenue, no code.
  • Internal project: key metrics instead of revenue, by CSV or through the Key metrics API.
Revenue sources add up, so tell them to use one source only, otherwise a customer-month counted in both Stripe and the API is double counted. 4. Confirm the customer identifier. Show what you found in the codebase and which one you propose, and state explicitly that it must match the identifier on their revenue rows or in Stripe. Ask them to confirm the exact string format (for example Acme Corp or org_8123). 5. Review the feature and call names. Not a blocking question: you named them in step 4 and wired them up. Show the list, one line each with the code path it wraps, and say that renaming later splits the history of that dashboard row, so now is the cheap moment to object.

Then mention these two, and offer to help

Not needed to get value out of Weflayr, so raise them after the four questions above are settled rather than in the same breath. Offer to do the work if the user wants either. Free AI credits. If a provider granted them credits, their invoiced cost is lower than list prices until the credits run out. Each grant (provider, amount, effective from date) is declared under Free AI credits in Costs & revenue, and every dashboard page then offers a Show real costs with free tokens toggle: off shows list prices, which is what you want to judge unit economics, on shows the real net cost. There is no API for this yet, so it is a dashboard action. See Free AI credits. Customer tags. Their own dimensions on customers: plan tier, segment, region, industry, account manager. They become filters on the AI Cost and AI Margin pages, so they can ask what a segment costs them. Uploaded as a CSV or pushed with the Customer tags API. If the codebase already holds these fields, offer to push them from the same place the revenue is pushed. See Customer tags.

Step 10 - Hand off a TODO list

End with a list in this shape. Be specific: name files, features and customers, not categories.

Pages to read for more detail