Quickstart - Weflayr Docs

1. Set up

Create your organization

Sign up at app.weflayr.com and name your organization, or accept a teammate’s invitation to join theirs. See Organizations & projects.

Create an API key

In API keys, create a key with the Gateway scope to call the gateway, or the Management scope to send telemetry with the SDK or import past calls with the API. Importing a CSV needs no key.

.env

WEFLAYR_API_KEY=your-api-key

Top up your credits

Add credits from the balance at the bottom of the sidebar: your gateway calls and your benchmarks are paid from them.

2. Use the gateway, or just log your calls with telemetry, a CSV or via API

The gateway is the recommended way: it logs your calls, and lets you route them later.

	What you change	Best for
Gateway (recommended)	The base URL of your OpenAI client	Logging, benchmarking and routing your calls
Import past calls, from a CSV or via API	Nothing in your code	Benchmarking calls you already logged elsewhere
Telemetry	A setup call and your provider’s instrumentation	Calls that must keep going to your providers directly

Gateway
Import past calls
Telemetry

Point your OpenAI client at the gateway

The gateway speaks the OpenAI format for every provider: keep your OpenAI client, change its base URL and key, and name any model.

from openai import OpenAI

client = OpenAI(base_url="https://gateway.weflayr.com/v1", api_key="<WEFLAYR_API_KEY>")

client.responses.create(
    model="claude-opus-5-5",
    input="Hello",
    extra_body={"weflayr": {"call_name": "answer_question"}},  # Benchmarks are set up per call name
)

const OpenAI = require('openai');

const client = new OpenAI({ baseURL: 'https://gateway.weflayr.com/v1', apiKey: '<WEFLAYR_API_KEY>' });

await client.responses.create({
  model: 'claude-opus-5-5',
  input: 'Hello',
  weflayr: { call_name: 'answer_question' }, // Benchmarks are set up per call name
});

# Benchmarks are set up per call name ("call_name" below)
curl https://gateway.weflayr.com/v1/responses \
  -H "Authorization: Bearer $WEFLAYR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-opus-5-5",
    "input": "Hello",
    "weflayr": {"call_name": "answer_question"}
  }'

See Responses and Chat completions for every field.

For calls that must keep going to your providers directly: the SDK records each one for Weflayr.

Install and instrument

Pick the AI provider SDK you use, install it with its instrumentation package, then call autoInstrument() / auto_instrument() once at startup. Frameworks that call these SDKs underneath (LangChain, LlamaIndex…) are covered too.

Anthropic
OpenAI
AWS Bedrock
Google Vertex AI
Other

Install:

npm install weflayr @anthropic-ai/sdk @traceloop/instrumentation-anthropic

pip install weflayr anthropic opentelemetry-instrumentation-anthropic

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'anthropic' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.ANTHROPIC)

Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'openai' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

Streaming with the OpenAI SDK directly? OpenAI only returns token usage on streamed responses when you ask for it. Pass stream_options with include_usage on streaming calls, otherwise the prompt/completion token counts (and cost) will be missing from those spans.

await client.chat.completions.create({
  model: 'gpt-6-luna',
  messages: [{ role: 'user', content: 'Hello!' }],
  stream: true,
  stream_options: { include_usage: true },
});

client.chat.completions.create(
    model="gpt-6-luna",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)

Install:

npm install weflayr @aws-sdk/client-bedrock-runtime @traceloop/instrumentation-bedrock

pip install weflayr boto3 opentelemetry-instrumentation-bedrock

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'bedrock' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.BEDROCK)

Bedrock through the OpenAI SDK: calls made with the OpenAI client pointed at Bedrock’s OpenAI-compatible endpoint (https://bedrock-mantle.<region>.api.aws/v1) are recorded with the openai instrumentation instead, and billed as Bedrock calls in the region the endpoint names.

await weflayr.autoInstrument({ aiSdks: 'openai' });

const OpenAI = require('openai');
const client = new OpenAI({
  apiKey: process.env.AWS_BEARER_TOKEN_BEDROCK,
  baseURL: 'https://bedrock-mantle.us-east-1.api.aws/v1',
});

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AWS_BEARER_TOKEN_BEDROCK"],
    base_url="https://bedrock-mantle.us-east-1.api.aws/v1",
)

Each model family on Vertex AI is called with its own provider’s SDK, and Weflayr bills every call at Vertex’s prices:

Gemini: the Google GenAI SDK with vertexai set, instrumented with google-genai.
Claude: Anthropic’s AnthropicVertex client, instrumented with anthropic.
Partner models (Grok, DeepSeek, Llama…): the OpenAI SDK pointed at Vertex’s OpenAI-compatible endpoint, with a Google access token as its API key, instrumented with openai.
Mistral: MistralGCP, see Mistral AI under Other.

Install and instrument only the ones you use.Install:

npm install weflayr @google/genai @traceloop/instrumentation-google-generativeai \
  @anthropic-ai/sdk @anthropic-ai/vertex-sdk @traceloop/instrumentation-anthropic \
  openai @traceloop/instrumentation-openai

pip install weflayr google-genai opentelemetry-instrumentation-google-genai \
  "anthropic[vertex]" opentelemetry-instrumentation-anthropic \
  openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: ['google-genai', 'anthropic', 'openai'] });

const { GoogleGenAI } = require('@google/genai');
const { AnthropicVertex } = require('@anthropic-ai/vertex-sdk');
const OpenAI = require('openai');

const gemini = new GoogleGenAI({ vertexai: true, project: 'your-project', location: 'global' });
const claude = new AnthropicVertex({ projectId: 'your-project', region: 'global' });
const partnerModels = new OpenAI({
  baseURL: 'https://aiplatform.googleapis.com/v1/projects/your-project/locations/global/endpoints/openapi',
  apiKey: googleAccessToken,
});

weflayr.auto_instrument(
    ai_sdks=[weflayr.AiSdk.GOOGLE_GENAI, weflayr.AiSdk.ANTHROPIC, weflayr.AiSdk.OPENAI]
)

from anthropic import AnthropicVertex
from google import genai
from openai import OpenAI

gemini = genai.Client(vertexai=True, project="your-project", location="global")
claude = AnthropicVertex(project_id="your-project", region="global")
partner_models = OpenAI(
    base_url="https://aiplatform.googleapis.com/v1/projects/your-project/locations/global/endpoints/openapi",
    api_key=google_access_token,
)

Alibaba Cloud (Qwen)

Use Alibaba Cloud Model Studio through the OpenAI SDK. Weflayr detects the DashScope endpoint (dashscope.aliyuncs.com, dashscope-intl.aliyuncs.com or a *.maas.aliyuncs.com workspace host) and bills the calls to Alibaba.Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'openai' });

const OpenAI = require('openai');
const client = new OpenAI({
  apiKey: process.env.DASHSCOPE_API_KEY,
  baseURL: 'https://dashscope-intl.aliyuncs.com/compatible-mode/v1',
});

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DASHSCOPE_API_KEY"],
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)

Azure OpenAI

Use Azure OpenAI through the OpenAI SDK. Weflayr auto-detects the Azure endpoint.Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'openai' });

const { AzureOpenAI } = require('openai');
const client = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT,
  apiKey: process.env.AZURE_OPENAI_API_KEY,
  apiVersion: '2024-10-21',
});

await client.chat.completions.create({
  model: 'your-deployment',
  messages: [{ role: 'user', content: 'Hello!' }],
});

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-10-21",
)

client.chat.completions.create(
    model="your-deployment",
    messages=[{"role": "user", "content": "Hello!"}],
)

Cohere

Install:

npm install weflayr cohere-ai @traceloop/instrumentation-cohere

pip install weflayr cohere opentelemetry-instrumentation-cohere

Instrument:

await weflayr.autoInstrument({ aiSdks: 'cohere' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.COHERE)

ElevenLabs

Install:

npm install weflayr @elevenlabs/elevenlabs-js

pip install weflayr elevenlabs

Instrument:

await weflayr.autoInstrument({ aiSdks: 'elevenlabs' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.ELEVENLABS)

Google AI Studio

Install:

npm install weflayr @google/genai @traceloop/instrumentation-google-generativeai

pip install weflayr google-genai opentelemetry-instrumentation-google-genai

Instrument:

const weflayr = require('weflayr');

await weflayr.autoInstrument({ aiSdks: 'google-genai' });

import weflayr

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.GOOGLE_GENAI)

Mistral AI

Instrument the Mistral SDK directly. Mistral’s models on Vertex AI, called through MistralGCP, are traced too: in Node.js, install @mistralai/mistralai-gcp next to @mistralai/mistralai, in Python, install mistralai[gcp].Install:

npm install weflayr @mistralai/mistralai @opentelemetry/api

pip install weflayr mistralai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'mistral' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.MISTRAL)

xAI (Grok)

Use xAI through the OpenAI SDK. Weflayr detects the api.x.ai endpoint and bills the calls to xAI.Install:

npm install weflayr openai @traceloop/instrumentation-openai

pip install weflayr openai opentelemetry-instrumentation-openai

Instrument:

await weflayr.autoInstrument({ aiSdks: 'openai' });

const OpenAI = require('openai');
const client = new OpenAI({ apiKey: process.env.XAI_API_KEY, baseURL: 'https://api.x.ai/v1' });

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI)

from openai import OpenAI

client = OpenAI(api_key=os.environ["XAI_API_KEY"], base_url="https://api.x.ai/v1")

Every parameter of autoInstrument() / auto_instrument() is listed in the Python and Node.js SDK references.

Redact personal data with mask

Use mask to redact personal data (emails, card numbers, names) before it leaves your process. Your callback receives the span attributes right before export, and what it returns is what Weflayr stores. Returning null keeps the attributes unchanged; throwing an error drops all of them.

const EMAIL = /[\w.+-]+@[\w-]+\.[\w.-]+/g;

await weflayr.autoInstrument({
  aiSdks: 'openai',
  mask: (attrs) => {
    // Replace every email in the prompts and answers
    for (const [key, value] of Object.entries(attrs)) {
      if (typeof value === 'string') attrs[key] = value.replace(EMAIL, '[email]');
    }
    return attrs;
  },
});

import re

EMAIL = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")

def mask(attrs):
    # Replace every email in the prompts and answers
    return {
        key: EMAIL.sub("[email]", value) if isinstance(value, str) else value
        for key, value in attrs.items()
    }

weflayr.auto_instrument(ai_sdks=weflayr.AiSdk.OPENAI, mask=mask)

Name your calls

Wrap each LLM call in setCallName / set_call_name: benchmarks are set up per call name.

await weflayr.setCallName('answer_question', async () => {
  // Your LLM call
});

with weflayr.set_call_name("answer_question"):
    ...  # Your LLM call

3. Benchmark a call

In Model benchmarks, click New benchmark, pick a call, and follow the steps: the examples to replay, the models to try, and the evaluators that judge their answers. See Create a benchmark. When it is done, the report shows the mix of models recommended for each objective, its quality against your current model and what it saves.

4. Route your calls

In Routing strategies, click New strategy, pick your benchmark, check its rules and save. Then send the strategy’s id as the model of your gateway calls:

client.responses.create(
    model="strategy/production-default",
    input="Hello",
)

await client.responses.create({
  model: 'strategy/production-default',
  input: 'Hello',
});

curl https://gateway.weflayr.com/v1/responses \
  -H "Authorization: Bearer $WEFLAYR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "strategy/production-default",
    "input": "Hello"
  }'

From now on, change the models your calls use from the routing strategy.

5. Analyze and go further

Tag features and customers: name the product feature (feature_name) and the customer (customer_name) of each call to benefit from advanced analytics and routing options.

Show example

import weflayr
from openai import OpenAI

weflayr.enable_propagate_metadata_to_gateway()
client = OpenAI(base_url="https://gateway.weflayr.com/v1", api_key="<WEFLAYR_API_KEY>")

def answer_ticket(ticket, customer):
    with weflayr.propagate_metadata(feature_name="support-chat", customer_name=customer):
        category = client.responses.create(
            model="claude-opus-5-5",
            input=f"Classify this ticket: {ticket}",
            extra_body={"weflayr": {"call_name": "classify_ticket"}},
        ).output_text
        return client.responses.create(
            model="claude-opus-5-5",
            input=f"Reply to this {category} ticket: {ticket}",
            extra_body={"weflayr": {"call_name": "draft_reply"}},
        ).output_text

const weflayr = require('weflayr');
const OpenAI = require('openai');

weflayr.setup();
weflayr.enablePropagateMetadataToGateway();
const client = new OpenAI({ baseURL: 'https://gateway.weflayr.com/v1', apiKey: '<WEFLAYR_API_KEY>' });

function answerTicket(ticket, customer) {
  return weflayr.propagateMetadata({ featureName: 'support-chat', customerName: customer }, async () => {
    const classification = await client.responses.create({
      model: 'claude-opus-5-5',
      input: `Classify this ticket: ${ticket}`,
      weflayr: { call_name: 'classify_ticket' },
    });
    const reply = await client.responses.create({
      model: 'claude-opus-5-5',
      input: `Reply to this ${classification.output_text} ticket: ${ticket}`,
      weflayr: { call_name: 'draft_reply' },
    });
    return reply.output_text;
  });
}

# Two calls of the same feature and customer, each with its own call name
curl https://gateway.weflayr.com/v1/responses \
  -H "Authorization: Bearer $WEFLAYR_API_KEY" \
  -H "Content-Type: application/json" \
  -H 'x-weflayr-metadata: {"feature_name": "support-chat"}' \
  -d '{
    "model": "claude-opus-5-5",
    "input": "Classify this ticket: My invoice is wrong",
    "weflayr": {"customer_name": "acme", "call_name": "classify_ticket"}
  }'

curl https://gateway.weflayr.com/v1/responses \
  -H "Authorization: Bearer $WEFLAYR_API_KEY" \
  -H "Content-Type: application/json" \
  -H 'x-weflayr-metadata: {"feature_name": "support-chat"}' \
  -d '{
    "model": "claude-opus-5-5",
    "input": "Reply to this billing ticket: My invoice is wrong",
    "weflayr": {"customer_name": "acme", "call_name": "draft_reply"}
  }'

Analyze your calls: the Overview shows your AI cost and usage and the calls coming in live, and AI cost breaks your cost down per feature.
Track your margin: sync your revenue from Stripe or a CSV to see your margin per customer or to include them in your routing strategy.
Track an internal tool: for a tool with no revenue (a sales copilot, a support assistant), create an internal project to see the cost of each user against the key metrics the tool drives, such as meetings booked.
Get alerted: set alerts on your cost and margin.

​1. Set up

​2. Use the gateway, or just log your calls with telemetry, a CSV or via API

​3. Benchmark a call

​4. Route your calls

​5. Analyze and go further

1. Set up

2. Use the gateway, or just log your calls with telemetry, a CSV or via API

3. Benchmark a call

4. Route your calls

5. Analyze and go further