> ## Documentation Index
> Fetch the complete documentation index at: https://docs.weflayr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Set evaluator

> Create an evaluator, or change one of the project's.

Create an [evaluator](https://docs.weflayr.com/product/evaluators.md), or change one of your project's. Try it on a logged call with test\_evaluator, then use its id in evaluators\_impacting\_benchmark or evaluators\_for\_reference of set\_benchmark.

## Parameters

Passed in the request body.

**evaluator\_id** `integer`

The evaluator to change. Omit it to create one. A built-in evaluator cannot be changed.

**name** `string`

The evaluator's name. Required to create one.

**kind** `string`

"judge" (an AI compares the candidate's answer with the baseline's by following your instruction in words) or "deterministic" (Python code checks the answers). Required to create one.

**instruction** `string`

A judge's instruction in words, or a deterministic check's Python code. Required to create one.

**in\_library** `boolean`

Add the evaluator to your library, which the Evaluators page lists and every benchmark can pick from. Defaults to false on creation: the person adds it from the dashboard once they have checked it.

**user\_email** `string`

The email of the member of your organization this is for, recorded as the evaluator's author when it is created. Defaults to the email of the person who created the API key, or for a key without one of the project's creator. The response's recorded\_for\_email gives the email recorded: tell the person, and send this to record another.

The response also carries `blockers` (what prevents the usual next step), in plain words. It is empty when the usual workflow applies.

```python Python theme={null}
import weflayr
from weflayr.openapi.models import SetEvaluatorInput

client = weflayr.client()  # reads WEFLAYR_API_KEY

client.set_evaluator(body=SetEvaluatorInput.from_dict({
    "name": "Polite tone",
    "kind": "judge",
    "instruction": "The answer is polite and never blames the customer.",
    "user_email": "alex@acme.com",
}))
```

```js Node.js theme={null}
const weflayr = require('weflayr');

const client = weflayr.client(); // reads WEFLAYR_API_KEY

await weflayr.api.setEvaluator({
  client,
  throwOnError: true,
  body: {
    name: 'Polite tone',
    kind: 'judge',
    instruction: 'The answer is polite and never blames the customer.',
    userEmail: 'alex@acme.com',
  },
});
```

```bash cURL theme={null}
curl -X POST https://app.weflayr.com/api/evaluators/ \
  -H "Authorization: Bearer $WEFLAYR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Polite tone",
    "kind": "judge",
    "instruction": "The answer is polite and never blames the customer.",
    "user_email": "alex@acme.com"
  }'
```

```json 200 theme={null}
{
  "evaluator_id": 14,
  "name": "Polite tone",
  "kind": "judge",
  "instruction": "The answer is polite and never blames the customer.",
  "builtin": false,
  "always_used": false,
  "in_library": false,
  "generated": false,
  "url": "https://app.weflayr.com/dashboard/optimization/benchmark/?evaluators=1",
  "recorded_for_email": "alex@acme.com",
  "blockers": []
}
```

```json 401 theme={null}
{"error": "Invalid or missing API key. Send it as \"Authorization: Bearer <your key>\"."}
```

```json 400 theme={null}
{"error": "No member of your organization has the email sam@acme.com. Ask the person for the email they use in Weflayr, and send it again.", "code": "unknown_user"}
```

```json 404 theme={null}
{"error": "No evaluator of this project has the id 14, or it was deleted. List the evaluators with get_evaluators."}
```

```json 422 theme={null}
{"error": "Invalid request payload.", "detail": [{"field": "", "error": "Value error, to create an evaluator, set kind, instruction"}]}
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.