Track your LLM calls

Get your API key

Create an API key in the Weflayr Dashboard. Either expose it as the WEFLAYR_API_KEY environment variable, or pass it directly to autoInstrument() in the next step.

.env

WEFLAYR_API_KEY=your-api-key

Install and instrument

Choose your AI provider SDK you are using & follow the install commands. Then run the auto instrumentation command as shown. PS: This works regardless of the Framework you are using (Langchain, Mastra, etc…)

Anthropic
OpenAI
AWS Bedrock
Google GenAI
Other

Install:

npm install weflayr @anthropic-ai/sdk @traceloop/instrumentation-anthropic

pip install weflayr anthropic opentelemetry-instrumentation-anthropic

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'anthropic' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.ANTHROPIC)

Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'openai' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

Streaming with the OpenAI SDK directly? OpenAI only returns token usage on streamed responses when you ask for it. Pass stream_options with include_usage on streaming calls, otherwise the prompt/completion token counts (and cost) will be missing from those spans.

await client.chat.completions.create({
  model: 'gpt-5.5',
  messages: [{ role: 'user', content: 'Hello!' }],
  stream: true,
  stream_options: { include_usage: true },
});

client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)

Install:

npm install weflayr @aws-sdk/client-bedrock-runtime @traceloop/instrumentation-bedrock

pip install weflayr boto3 opentelemetry-instrumentation-bedrock

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'bedrock' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.BEDROCK)

Install:

npm install weflayr @google/genai @traceloop/instrumentation-google-generativeai

pip install weflayr google-genai opentelemetry-instrumentation-google-genai

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'google-genai' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.GOOGLE_GENAI)

Azure OpenAI

Use Azure OpenAI through the OpenAI SDK. Weflayr auto-detects the Azure endpoint.Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'openai' });

const { AzureOpenAI } = require('openai');
const client = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT,
  apiKey: process.env.AZURE_OPENAI_API_KEY,
  apiVersion: '2024-10-21',
});

await client.chat.completions.create({
  model: 'your-deployment',
  messages: [{ role: 'user', content: 'Hello!' }],
});

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-10-21",
)

client.chat.completions.create(
    model="your-deployment",
    messages=[{"role": "user", "content": "Hello!"}],
)

Cohere

Install:

npm install weflayr cohere-ai @traceloop/instrumentation-cohere

pip install weflayr cohere opentelemetry-instrumentation-cohere

Instrument:

await weflayr.autoInstrument({ aiSdks: 'cohere' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.COHERE)

ElevenLabs

Install:

npm install weflayr @elevenlabs/elevenlabs-js

pip install weflayr elevenlabs

Instrument:

await weflayr.autoInstrument({ aiSdks: 'elevenlabs' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.ELEVENLABS)

Mistral AI

Node.js
Python

Mistral models are called through the OpenAI SDK, so instrument OpenAI and pass providerNameOverride: weflayr.AIProviderName.MISTRAL to propagateMetadata().

Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr mistralai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'openai' });

const OpenAI = require('openai');
const client = new OpenAI({
  baseURL: 'https://api.mistral.ai/v1',
  apiKey: process.env.MISTRAL_API_KEY,
});

await weflayr.propagateMetadata(
  { featureName: 'support-chat', customerId: 'c_123', providerNameOverride: weflayr.AIProviderName.MISTRAL },
  async () => {
    await client.chat.completions.create({
      model: 'mistral-small-latest',
      messages: [{ role: 'user', content: 'Hello!' }],
    });
  }
);

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.MISTRAL)

Vercel AI SDK

Node.js
Python

Weflayr instruments the Vercel AI SDK natively. Calls through a provider package with your own key (e.g. @ai-sdk/openai) are attributed to that provider and priced at its rates.

Call Vercel through the OpenAI SDK and label the calls with provider_name_override=weflayr.AIProviderName.VERCEL_AI_GATEWAY in propagate_metadata.

Vercel adds a markup of ~3% when buying credits (small variations depending how much tokens you buy). Weflayr therefore prices gateway calls at the model price listed by the gateway plus 3%.

ai v7
ai v5 / v6

Install:

npm install weflayr ai @ai-sdk/otel

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'vercel_ai_gateway' });

const { generateText } = require('ai');

// AI Gateway (default): plain "creator/model" string, key in AI_GATEWAY_API_KEY.
await weflayr.propagateMetadata(
  { featureName: 'support-chat', customerName: 'Acme Corp' },
  async () => {
    await generateText({
      model: 'anthropic/claude-sonnet-4.5',
      prompt: 'Hello!',
    });
  }
);

// Provider package with your own key: the call is automatically attributed
// to that provider (openai here) and priced at its rates.
const { createOpenAI } = require('@ai-sdk/openai');
const openai = createOpenAI({ apiKey: process.env.OPENAI_API_KEY });

await weflayr.propagateMetadata(
  { featureName: 'support-chat', customerName: 'Acme Corp' },
  async () => {
    await generateText({ model: openai('gpt-5.4'), prompt: 'Hello!' });
  }
);

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import OpenAI

client = OpenAI(
    base_url="https://ai-gateway.vercel.sh/v1",
    api_key=os.environ["AI_GATEWAY_API_KEY"],
)

with weflayr.propagate_metadata(
    feature_name="support-chat",
    customer_name="Acme Corp",
    provider_name_override=weflayr.AIProviderName.VERCEL_AI_GATEWAY,
):
    client.chat.completions.create(
        model="anthropic/claude-sonnet-4.5",
        messages=[{"role": "user", "content": "Hello!"}],
    )

Node.js
Python

Autoinstrument without specifying any SDK, and make sure experimental_telemetry: { isEnabled: true } is passed on each Vercel call to allow the spans to be picked up by Weflayr’s tracer.

Nothing changes between ai versions: Vercel is reached through the OpenAI SDK either way.

Install:

npm install weflayr ai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: [] });

const { generateText } = require('ai');

// AI Gateway (default): plain "creator/model" string, key in AI_GATEWAY_API_KEY.
await weflayr.propagateMetadata(
  { featureName: 'support-chat', customerName: 'Acme Corp' },
  async () => {
    await generateText({
      model: 'anthropic/claude-sonnet-4.5',
      prompt: 'Hello!',
      experimental_telemetry: { isEnabled: true },
    });
  }
);

// Provider package with your own key: the call is automatically attributed
// to that provider (openai here) and priced at its rates.
const { createOpenAI } = require('@ai-sdk/openai');
const openai = createOpenAI({ apiKey: process.env.OPENAI_API_KEY });

await weflayr.propagateMetadata(
  { featureName: 'support-chat', customerName: 'Acme Corp' },
  async () => {
    await generateText({
      model: openai('gpt-5.4'),
      prompt: 'Hello!',
      experimental_telemetry: { isEnabled: true },
    });
  }
);

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import OpenAI

client = OpenAI(
    base_url="https://ai-gateway.vercel.sh/v1",
    api_key=os.environ["AI_GATEWAY_API_KEY"],
)

with weflayr.propagate_metadata(
    feature_name="support-chat",
    customer_name="Acme Corp",
    provider_name_override=weflayr.AIProviderName.VERCEL_AI_GATEWAY,
):
    client.chat.completions.create(
        model="anthropic/claude-sonnet-4.5",
        messages=[{"role": "user", "content": "Hello!"}],
    )

All parameters for autoInstrument() / auto_instrument()

string, enum member, or a list of them

required

Node.js
Python

'openai', 'anthropic', 'bedrock', 'cohere', 'google-genai', 'elevenlabs', or an array of them.

A weflayr.AiSdk enum member (OPENAI, ANTHROPIC, BEDROCK, COHERE, GOOGLE_GENAI, ELEVENLABS, MISTRAL), or an iterable of them.

Each value picks the matching OpenTelemetry instrumentation package to load. Install the package separately.

string

Defaults to the WEFLAYR_API_KEY environment variable. Pass explicitly to override.

object

Tags stamped on every emitted span.

"batch" | "immediate"

default:"\"batch\""

Use "immediate" to export telemetry after every call.

boolean

default:"true"

Set false to strip gen_ai.prompt / gen_ai.completion from spans before export.

function

Redact or rewrite fields before they leave your process. See the mask example below.

boolean

default:"true"

Set false to silence the one-shot stderr warning emitted on the first export failure.

Mask example

Use mask to redact PII (emails, credit cards, customer names) or to drop entire prompts before they’re sent to Weflayr. Your callback runs on a copy of the span attributes right before export; whatever you return is what we store. Return null to drop the span entirely.

await weflayr.autoInstrument({
  aiSdks: 'openai',
  mask: (attrs) => {
    // Strip emails out of the prompt
    if (typeof attrs['gen_ai.prompt'] === 'string') {
      attrs['gen_ai.prompt'] = attrs['gen_ai.prompt'].replace(
        /[\w.+-]+@[\w-]+\.[\w.-]+/g,
        '[email]'
      );
    }
    return attrs;
  },
});

import re

def mask(attrs):
    prompt = attrs.get("gen_ai.prompt")
    if isinstance(prompt, str):
        attrs["gen_ai.prompt"] = re.sub(r"[\w.+-]+@[\w-]+\.[\w.-]+", "[email]", prompt)
    return attrs

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI, mask=mask)

Add metadata to your LLM calls for finer analysis

Wrap your LLM calls in propagateMetadata / propagate_metadata to stamp each emitted span with who the call is for (customer_name) and why it’s happening (feature_name). Anything inside the scope, including nested calls, picks up the same metadata.

Tagging feature_name and customer_name unlock per-customer and per-feature breakdowns across the dashboard, for analysis (e.g. margin) and for benchmarking (e.g. model benchmark).

As a callback / context managerWrap a single block of code. Every LLM call made inside picks up the metadata.

await weflayr.propagateMetadata(
  { featureName: 'support-chat', customerName: 'Acme Corp' },
  async () => {
    // Any LLM call made here gets feature_name + customer_name stamped on its span.
    await client.chat.completions.create({ ... });
  }
);

with weflayr.propagate_metadata(feature_name="support-chat", customer_name="Acme Corp"):
    # Any LLM call made here gets feature_name + customer_name stamped on its span.
    client.chat.completions.create(...)

As a decoratorApply once to a function or method, and every call automatically picks up the metadata.

class ChatService {
  @weflayr.propagateMetadata({ featureName: 'support-chat', customerName: 'Acme Corp' })
  async reply() {
    await client.chat.completions.create({ ... });
  }
}

@weflayr.propagate_metadata(feature_name="support-chat", customer_name="Acme Corp")
def reply():
    client.chat.completions.create(...)

All parameters for propagateMetadata() / propagate_metadata()

string

required

The product-side feature triggering the LLM call (e.g. "support-chat", "onboarding-summary").

string

required

Name of the end-user or tenant customer the call is made for.

enum member

Override the auto-detected provider. Use this when the SDK on the wire differs from the provider to bill (e.g. Mistral or the Vercel AI Gateway reached through the OpenAI SDK).

object

Any scalar key/value pairs (string / number / boolean) that you may need to analyse your costs in more detail.

Flush before exit (short-lived runtimes only)

In long-running servers, spans are flushed automatically. In short-lived runtimes (Lambda, scripts, edge functions), call flush() before exit so the final spans reach Weflayr.

await weflayr.flush();

weflayr.flush()

Observe your margin and optimise your AI costs

Your calls now stream to the dashboard and can be optimised: see the platform documentation

Get started

Technical documentation

Platform documentation

Get your API key

Install and instrument

Mask example

Add metadata to your LLM calls for finer analysis

Flush before exit (short-lived runtimes only)

Observe your margin and optimise your AI costs

Implementation examples

Multiple AI providers

AI Framework

Get your API key

Install and instrument

​Mask example

Add metadata to your LLM calls for finer analysis

Flush before exit (short-lived runtimes only)

Observe your margin and optimise your AI costs

​Implementation examples

Multiple AI providers

AI Framework

Mask example

Implementation examples