> ## Documentation Index
> Fetch the complete documentation index at: https://docs.weflayr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Test evaluator

> Try an evaluator on a logged call.

Try an unsaved evaluator on one logged call of a call name: the candidate deployment answers the call again, and the evaluator scores its answer against the baseline's, which is the answer production logged unless you name a baseline deployment. If the verdict is what you expect, save the evaluator with set\_evaluator and use it in a benchmark.

> [!WARNING]
>
> Spends a little credit.

## Parameters

Passed in the request body.

**kind** `string` *required*

"judge" or "deterministic".

**instruction** `string` *required*

The judge's instruction in words, or the check's Python code.

**call\_name** `string` *required*

A call name, as get\_calls lists them: the test runs on one logged call of that name, drawn at random from the last 30 days.

**candidate\_deployment** `object` *required*

The deployment that answers the call again, to be scored.

**baseline\_deployment** `object`

The deployment whose answer the candidate is compared with. Omit it to compare with the answer production logged.

The response also carries `blockers` (what prevents the usual next step), in plain words. It is empty when the usual workflow applies.

```python Python theme={null}
import weflayr
from weflayr.openapi.models import TestEvaluatorInput

client = weflayr.client()  # reads WEFLAYR_API_KEY

client.test_evaluator(body=TestEvaluatorInput.from_dict({
    "kind": "judge",
    "instruction": "The answer is polite and never blames the customer.",
    "call_name": "summarize_profile",
    "candidate_deployment": {
        "model": "gpt-6-luna",
        "provider": "openai",
        "region": "GLOBAL",
    },
}))
```

```js Node.js theme={null}
const weflayr = require('weflayr');

const client = weflayr.client(); // reads WEFLAYR_API_KEY

await weflayr.api.testEvaluator({
  client,
  throwOnError: true,
  body: {
    kind: 'judge',
    instruction: 'The answer is polite and never blames the customer.',
    callName: 'summarize_profile',
    candidateDeployment: {
      model: 'gpt-6-luna',
      provider: 'openai',
      region: 'GLOBAL',
    },
  },
});
```

```bash cURL theme={null}
curl -X POST https://app.weflayr.com/api/evaluator-test/ \
  -H "Authorization: Bearer $WEFLAYR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "judge",
    "instruction": "The answer is polite and never blames the customer.",
    "call_name": "summarize_profile",
    "candidate_deployment": {
      "model": "gpt-6-luna",
      "provider": "openai",
      "region": "GLOBAL"
    }
  }'
```

```json 200 theme={null}
{
  "call_id": "evt_8f2a",
  "candidate_output": "Thank you for reaching out. Here is a summary of the profile.",
  "baseline_output": null,
  "verdict": "same",
  "justification": "Both answers are polite.",
  "blockers": []
}
```

```json 401 theme={null}
{"error": "Invalid or missing API key. Send it as \"Authorization: Bearer <your key>\"."}
```

```json 400 theme={null}
{"error": "No call named 'summarize_profile' with its prompt was logged in the last 30 days. List the call names with get_calls.", "code": "no_call_to_test"}
```

```json 402 theme={null}
{"error": "Your organization has no credit left. Top up on the Credits page, then call this again.", "code": "insufficient_credits"}
```

```json 422 theme={null}
{"error": "Invalid request payload.", "detail": [{"field": "kind", "error": "Field required"}]}
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.