Telemetry

The gateway is the recommended way to send your calls: one change of base URL, no instrumentation, and your calls can then follow a routing strategy. Use the SDK below when your calls must keep going to your providers directly.

Get your API key

Create an API key with the Management scope in API keys. Either expose it as the WEFLAYR_API_KEY environment variable, or pass it directly to autoInstrument() in the next step.

.env

WEFLAYR_API_KEY=your-api-key

Install and instrument

Pick the AI provider SDK you use, install it with its instrumentation package, then call autoInstrument() / auto_instrument() once at startup. Frameworks that call these SDKs underneath (LangChain, LlamaIndex…) are covered too.

Anthropic
OpenAI
AWS Bedrock
Google GenAI
Other

Install:

npm install weflayr @anthropic-ai/sdk @traceloop/instrumentation-anthropic

pip install weflayr anthropic opentelemetry-instrumentation-anthropic

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'anthropic' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.ANTHROPIC)

Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'openai' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

Streaming with the OpenAI SDK directly? OpenAI only returns token usage on streamed responses when you ask for it. Pass stream_options with include_usage on streaming calls, otherwise the prompt/completion token counts (and cost) will be missing from those spans.

await client.chat.completions.create({
  model: 'gpt-5.6',
  messages: [{ role: 'user', content: 'Hello!' }],
  stream: true,
  stream_options: { include_usage: true },
});

client.chat.completions.create(
    model="gpt-5.6",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)

Install:

npm install weflayr @aws-sdk/client-bedrock-runtime @traceloop/instrumentation-bedrock

pip install weflayr boto3 opentelemetry-instrumentation-bedrock

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'bedrock' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.BEDROCK)

Install:

npm install weflayr @google/genai @traceloop/instrumentation-google-generativeai

pip install weflayr google-genai opentelemetry-instrumentation-google-genai

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'google-genai' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.GOOGLE_GENAI)

Azure OpenAI

Use Azure OpenAI through the OpenAI SDK. Weflayr auto-detects the Azure endpoint.Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'openai' });

const { AzureOpenAI } = require('openai');
const client = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT,
  apiKey: process.env.AZURE_OPENAI_API_KEY,
  apiVersion: '2024-10-21',
});

await client.chat.completions.create({
  model: 'your-deployment',
  messages: [{ role: 'user', content: 'Hello!' }],
});

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-10-21",
)

client.chat.completions.create(
    model="your-deployment",
    messages=[{"role": "user", "content": "Hello!"}],
)

Cohere

Install:

npm install weflayr cohere-ai @traceloop/instrumentation-cohere

pip install weflayr cohere opentelemetry-instrumentation-cohere

Instrument:

await weflayr.autoInstrument({ aiSdks: 'cohere' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.COHERE)

ElevenLabs

Install:

npm install weflayr @elevenlabs/elevenlabs-js

pip install weflayr elevenlabs

Instrument:

await weflayr.autoInstrument({ aiSdks: 'elevenlabs' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.ELEVENLABS)

Mistral AI

Instrument the Mistral SDK directly. Mistral’s models on Vertex AI, called through MistralGCP, are traced too: in Node.js, install @mistralai/mistralai-gcp next to @mistralai/mistralai.Install:

npm install weflayr @mistralai/mistralai @opentelemetry/api

pip install weflayr mistralai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'mistral' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.MISTRAL)

All parameters for autoInstrument() / auto_instrument()

string, enum member, or a list of them

required

Node.js
Python

'openai', 'anthropic', 'bedrock', 'cohere', 'google-genai', 'elevenlabs', 'mistral', or an array of them.

A weflayr.AiSdk enum member (OPENAI, ANTHROPIC, BEDROCK, COHERE, GOOGLE_GENAI, ELEVENLABS, MISTRAL), or an iterable of them.

Each value picks the matching OpenTelemetry instrumentation package to load. Install the package separately.

string

Defaults to the WEFLAYR_API_KEY environment variable. Pass explicitly to override.

string

default:"\"https://api.weflayr.com\""

Where spans are sent. Defaults to the WEFLAYR_BASE_URL environment variable when set.

object

Tags stamped on every emitted span.

"batch" | "immediate"

default:"\"batch\""

Use "immediate" to export telemetry after every call.

boolean

default:"true"

Set false to keep prompts, answers and tool definitions in your process: only tokens, cost and latency are sent.

function

Redact or rewrite fields before they leave your process. See the mask example below.

boolean

default:"true"

Set false to silence the warning written to stderr on each failed export.

Mask example

Use mask to redact personal data (emails, card numbers, names) before it leaves your process. Your callback receives the span attributes right before export, and what it returns is what Weflayr stores. Returning null keeps the attributes unchanged; throwing an error drops all of them.

const EMAIL = /[\w.+-]+@[\w-]+\.[\w.-]+/g;

await weflayr.autoInstrument({
  aiSdks: 'openai',
  mask: (attrs) => {
    // Replace every email in the prompts and answers
    for (const [key, value] of Object.entries(attrs)) {
      if (typeof value === 'string') attrs[key] = value.replace(EMAIL, '[email]');
    }
    return attrs;
  },
});

import re

EMAIL = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")

def mask(attrs):
    # Replace every email in the prompts and answers
    return {
        key: EMAIL.sub("[email]", value) if isinstance(value, str) else value
        for key, value in attrs.items()
    }

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI, mask=mask)

Add metadata to your LLM calls for finer analysis

Wrap your LLM calls in propagateMetadata / propagate_metadata to stamp each emitted span with who the call is for (customer_name) and why it’s happening (feature_name). Anything inside the scope, including nested calls, picks up the same metadata.

Tagging feature_name and customer_name unlock per-customer and per-feature breakdowns across the dashboard, for analysis (e.g. AI Margin) and for benchmarking (e.g. model benchmarks).

As a callback / context managerWrap a single block of code. Every LLM call made inside picks up the metadata.

await weflayr.propagateMetadata(
  { featureName: 'support-chat', customerName: 'Acme Corp' },
  async () => {
    // Any LLM call made here gets feature_name + customer_name stamped on its span.
    await client.chat.completions.create({ ... });
  }
);

with weflayr.propagate_metadata(feature_name="support-chat", customer_name="Acme Corp"):
    # Any LLM call made here gets feature_name + customer_name stamped on its span.
    client.chat.completions.create(...)

As a decoratorApply once to a function or method, and every call automatically picks up the metadata.

class ChatService {
  @weflayr.propagateMetadata({ featureName: 'support-chat', customerName: 'Acme Corp' })
  async reply() {
    await client.chat.completions.create({ ... });
  }
}

@weflayr.propagate_metadata(feature_name="support-chat", customer_name="Acme Corp")
def reply():
    client.chat.completions.create(...)

All parameters for propagateMetadata() / propagate_metadata()

string

required

The product-side feature triggering the LLM call (e.g. "support-chat", "onboarding-summary").

string

Name of the end-user or tenant customer the call is made for. When omitted, the calls are recorded with no customer. A call sent through the Weflayr AI gateway can name its customer in the request body instead (weflayr: { customer_name: "..." }).

object

Any scalar key/value pairs (string / number / boolean) that you may need to analyze your costs in more detail.

string

The run id to record the calls under. By default each propagateMetadata() call starts a new run. Pass the same value to group the calls of one run made in several scopes or processes, e.g. a job whose workers each open their own scope: create it once with createFeatureRunUuid() / create_feature_run_uuid() where the run starts. Any other value must have the same shape, 32 lowercase hexadecimal characters.

enum member

Override the auto-detected provider of a call recorded by the SDK’s telemetry instrumentation. Use this when the SDK on the wire differs from the provider to bill (e.g. Mistral reached through the OpenAI SDK). Ignored for a call sent through the Weflayr AI gateway, which reports the provider it called itself.

Flush before exit (short-lived runtimes only)

In long-running servers, spans are flushed automatically. In short-lived runtimes (Lambda, scripts, edge functions), call flush() before exit so the final spans reach Weflayr.

await weflayr.flush();

weflayr.flush()

See your calls

Your calls now show on the Overview, and can be benchmarked.

Examples

Multiple AI providers

autoInstrument() accepts an array of providers, so one call wires both up; subsequent propagateMetadata() blocks tag every LLM call inside them regardless of which provider runs.

Install

npm install weflayr openai @anthropic-ai/sdk \
  @traceloop/instrumentation-openai @traceloop/instrumentation-anthropic

pip install weflayr openai anthropic \
  opentelemetry-instrumentation-openai opentelemetry-instrumentation-anthropic

Setup

const weflayr = require('weflayr');
const OpenAI = require('openai');
const Anthropic = require('@anthropic-ai/sdk');

await weflayr.autoInstrument({
  aiSdks: ['openai', 'anthropic'],
  defaultTags: { app: 'support-bot', env: 'production', region: 'eu-west-1' },
});

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

import os
import weflayr
from openai import OpenAI
import anthropic

weflayr.auto_instrument(
    ai_sdks=[weflayr.AiSdk.OPENAI, weflayr.AiSdk.ANTHROPIC],
    default_tags={"app": "support-bot", "release": "v.1.3"},
)

openai_client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
anthropic_client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

Use

propagateMetadata wraps an entire request handler. Both LLM calls inside, regardless of provider, are stamped with the same customer_name and feature_name.

async function handleSupportTicket(customerName, ticket) {
  return weflayr.propagateMetadata(
    {
      featureName: 'support-ticket',
      customerName,
      extraTags: { plan_tier: 'enterprise', priority: 'high' },
    },
    async () => {
      const classification = await openai.chat.completions.create({
        model: 'gpt-5.6',
        messages: [{ role: 'user', content: `Classify: ${ticket}` }],
      });

      const answer = await anthropic.messages.create({
        model: 'claude-opus-5-5',
        max_tokens: 512,
        messages: [{ role: 'user', content: `Answer this ticket: ${ticket}` }],
      });

      return { classification, answer };
    }
  );
}

def handle_support_ticket(customer_name: str, ticket: str):
    with weflayr.propagate_metadata(
        feature_name="support-ticket",
        customer_name=customer_name,
        extra_tags={"plan_tier": "enterprise", "priority": "high"},
    ):
        classification = openai_client.chat.completions.create(
            model="gpt-5.6",
            messages=[{"role": "user", "content": f"Classify: {ticket}"}],
        )
        answer = anthropic_client.messages.create(
            model="claude-opus-5-5",
            max_tokens=512,
            messages=[{"role": "user", "content": f"Answer this ticket: {ticket}"}],
        )
        return classification, answer

In your dashboard, both spans land under the same customer_name + feature_name, but the provider field (auto-detected) is openai for the first and anthropic for the second.

AI framework

AI frameworks like LangChain, LlamaIndex or Haystack call the underlying provider SDK (OpenAI, Anthropic, …) so the OpenTelemetry instrumentation for that provider catches every call your chain makes, transparently. You don’t need a framework-specific instrumentation. The example below uses LangChain, but the same approach works with any framework that calls an instrumented provider. This example also turns on exportMode: 'immediate', suitable for short-lived runtimes (Lambda, edge functions, scripts) where you don’t want to risk losing buffered spans on exit.

Install

npm install weflayr langchain @langchain/openai @traceloop/instrumentation-openai

pip install weflayr langchain langchain-openai opentelemetry-instrumentation-openai

Setup

const weflayr = require('weflayr');
const { ChatOpenAI } = require('@langchain/openai');

await weflayr.autoInstrument({
  aiSdks: 'openai',
  defaultTags: { app: 'rag-pipeline', env: 'production', version: '2.4.1' },
  exportMode: 'immediate',
});

const llm = new ChatOpenAI({
  model: 'gpt-5.6',
  apiKey: process.env.OPENAI_API_KEY,
});

import os
import weflayr
from langchain_openai import ChatOpenAI

weflayr.auto_instrument(
    ai_sdks=weflayr.AiSdk.OPENAI,
    default_tags={"app": "rag-pipeline"},
    export_mode="immediate",
)

llm = ChatOpenAI(model="gpt-5.6", api_key=os.environ["OPENAI_API_KEY"])

Use

A simple RAG-style chain: retrieve context, then ask the LLM. Decorate the feature function with propagateMetadata / propagate_metadata: every instrumented call it makes (here the OpenAI call inside .invoke(...)) is stamped with the metadata, and each invocation gets a fresh correlation id.

const { ChatPromptTemplate } = require('@langchain/core/prompts');

const prompt = ChatPromptTemplate.fromMessages([
  ['system', 'Answer using only the provided context.'],
  ['user', 'Context:\n{context}\n\nQuestion: {question}'],
]);
const chain = prompt.pipe(llm);

const answer = weflayr.propagateMetadata({
  featureName: 'rag-answer',
  customerName: 'Acme Corp',
  extraTags: { retrieval_strategy: 'hybrid', top_k: 5 },
})((question, context) => chain.invoke({ question, context }));

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer using only the provided context."),
    ("user", "Context:\n{context}\n\nQuestion: {question}"),
])
chain = prompt | llm


@weflayr.propagate_metadata(
    feature_name="rag-answer",
    customer_name="Acme Corp",
    extra_tags={"retrieval_strategy": "hybrid", "top_k": 5},
)
def answer(question: str, context: str):
    return chain.invoke({"question": question, "context": context})

The same pattern works for any LangChain construct that ultimately calls an instrumented provider: chains, agents, runnables, LCEL pipelines. If the chain makes several LLM calls (e.g. a tool-calling agent loop), every call inside the decorated function carries the same customer_name and feature_name, so the dashboard can correlate them as one logical request. The decorator fixes its metadata at decoration time; when a value varies per request (a per-call customer_name, say), use the inline form instead (weflayr.propagateMetadata(options, fn) in JS, or with weflayr.propagate_metadata(...): in Python).

SDKs

Guides

Changelog

Get your API key

Install and instrument

Mask example

Add metadata to your LLM calls for finer analysis

Flush before exit (short-lived runtimes only)

See your calls

Examples

Multiple AI providers

Install

Setup

Use

AI framework

Install

Setup

Use

Get your API key

Install and instrument

​Mask example

Add metadata to your LLM calls for finer analysis

Flush before exit (short-lived runtimes only)

See your calls

​Examples

​Multiple AI providers

​Install

​Setup

​Use

​AI framework

​Install

​Setup

​Use

Mask example

Examples

Multiple AI providers

Install

Setup

Use

AI framework

Install

Setup

Use