> ## Documentation Index
> Fetch the complete documentation index at: https://docs.weflayr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Generate evaluators

> Have AI write evaluators.

Have AI write evaluators ([how](https://docs.weflayr.com/concepts/benchmarking.md#evaluators-specific-to-your-use-case)). With a benchmark\_uuid, it writes judge evaluators from what the calls of that saved setup share (a prompt text, a tool, an answer schema) and saves them to your project, out of your library until the person adds them from the dashboard. With a description, it writes the Python code of a deterministic evaluator from a rule in words.

> [!WARNING]
>
> Spends a little credit.

## Parameters

Passed in the request body.

**benchmark\_uuid** `string`

Write judge evaluators for the calls this saved setup sampled. They are saved to your project.

**description** `string`

Write the Python code of a deterministic evaluator from this rule in words.

**user\_email** `string`

The email of the member of your organization this is for, recorded as the author of the evaluators written for a benchmark\_uuid. Defaults to the email of the person who created the API key, or for a key without one of the project's creator. The response's recorded\_for\_email gives the email recorded: tell the person, and send this to record another.

The response also carries `blockers` (what prevents the usual next step), in plain words. It is empty when the usual workflow applies.

```python Python theme={null}
import weflayr
from weflayr.openapi.models import GenerateEvaluatorsInput

client = weflayr.client()  # reads WEFLAYR_API_KEY

client.generate_evaluators(body=GenerateEvaluatorsInput.from_dict({
    "description": "The answer has fewer than 50 words.",
}))
```

```js Node.js theme={null}
const weflayr = require('weflayr');

const client = weflayr.client(); // reads WEFLAYR_API_KEY

await weflayr.api.generateEvaluators({
  client,
  throwOnError: true,
  body: {
    description: 'The answer has fewer than 50 words.',
  },
});
```

```bash cURL theme={null}
curl -X POST https://app.weflayr.com/api/evaluator-generation/ \
  -H "Authorization: Bearer $WEFLAYR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "description": "The answer has fewer than 50 words."
  }'
```

```json 200 theme={null}
{
  "evaluators": [],
  "instruction": "def check(input: str, baseline_output: str, evaluated_output: str, tags: dict[str, str]) -> bool:\n    return len(evaluated_output.split()) < 50",
  "url": null,
  "recorded_for_email": null,
  "blockers": []
}
```

```json 401 theme={null}
{"error": "Invalid or missing API key. Send it as \"Authorization: Bearer <your key>\"."}
```

```json 400 theme={null}
{"error": "No member of your organization has the email sam@acme.com. Ask the person for the email they use in Weflayr, and send it again.", "code": "unknown_user"}
```

```json 402 theme={null}
{"error": "Your organization has no credit left. Top up on the Credits page, then call this again.", "code": "insufficient_credits"}
```

```json 404 theme={null}
{"error": "No saved setup of this project has the id 5c1d6a52-3f5f-4c53-9a0e-6f2b9c7d8e01. Save one with set_benchmark, or list the benchmarks with get_benchmarks."}
```

```json 422 theme={null}
{"error": "Invalid request payload.", "detail": [{"field": "", "error": "Value error, set either \"benchmark_uuid\" or \"description\", not both"}]}
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.