> ## Documentation Index
> Fetch the complete documentation index at: https://www.voiceflow.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluations

API operations for evaluations. Browse the routes below for parameters, request examples, and responses.

API version: Selected per operation

## List evaluations

Returns every evaluation in the project. An evaluation is a criterion a model scores past conversations against. The call takes no paging parameters, so one request yields the whole set.

### Endpoint

```http
GET /v1/stable/evaluation
```

Server: https://realtime-api.voiceflow.com

### Authentication

```json
[
  {
    "token": []
  }
]
```

`token`:

```json
{
  "scheme": "bearer",
  "bearerFormat": "JWT",
  "type": "http",
  "description": "Voiceflow bearer token"
}
```

### Query parameters

#### `projectID`

Required · string

The ID of the project to operate on.

```json
{
  "type": "string",
  "description": "The ID of the project to operate on."
}
```

### Responses

#### 200

Returns every evaluation defined on the project, each with the criteria it judges transcripts against, its result type, whether it runs automatically on new transcripts, and its average cost per run.

Schema:

```json
{
  "$ref": "#/components/schemas/StableEvaluationListResponse"
}
```

Example:

```json
{
  "evaluations": [
    {
      "id": "<string>",
      "name": "<string>",
      "prompt": "<string>",
      "default": true,
      "enabled": true,
      "projectID": "<string>",
      "settings": "<AgentSettings>",
      "averageCost": 0,
      "description": "<string>"
    }
  ]
}
```

### Request examples

#### cURL

```bash
curl --request GET \
  --url 'https://realtime-api.voiceflow.com/v1/stable/evaluation?projectID=<projectID>' \
  --header 'Authorization: Bearer <token>'
```

#### JavaScript

```javascript
const response = await fetch('https://realtime-api.voiceflow.com/v1/stable/evaluation?projectID=<projectID>', {
  method: 'GET',
  headers: {
    Authorization: 'Bearer <token>',
  },
});
const data = await response.json();
```

#### Python

```python
import requests

response = requests.get(
    'https://realtime-api.voiceflow.com/v1/stable/evaluation?projectID=<projectID>',
    headers={'Authorization': 'Bearer <token>'},
)
data = response.json()
```

#### Voiceflow CLI

```bash
vf evaluation list --project-id <projectID>
```


## Create evaluation

Adds a evaluation to the project, a criterion a model scores past conversations against. The response carries the ID that later calls address it by.

### Endpoint

```http
POST /v1/stable/evaluation
```

Server: https://realtime-api.voiceflow.com

### Authentication

```json
[
  {
    "token": []
  }
]
```

`token`:

```json
{
  "scheme": "bearer",
  "bearerFormat": "JWT",
  "type": "http",
  "description": "Voiceflow bearer token"
}
```

### Query parameters

#### `projectID`

Required · string

The ID of the project to operate on.

```json
{
  "type": "string",
  "description": "The ID of the project to operate on."
}
```

### Request body

Required

```json
{
  "required": true,
  "content": {
    "application/json": {
      "schema": {
        "$ref": "#/components/schemas/StableEvaluationCreateRequest"
      }
    }
  }
}
```

### Responses

#### 201

Creates the evaluation and returns it with its assigned `id`. No transcript is scored yet: results come from running it against a transcript, or automatically on new transcripts when `enabled` is true.

Schema:

```json
{
  "$ref": "#/components/schemas/StableEvaluationResponse"
}
```

Example:

```json
{
  "evaluation": {
    "id": "<string>",
    "name": "<string>",
    "prompt": "<string>",
    "default": true,
    "enabled": true,
    "projectID": "<string>",
    "settings": "<AgentSettings>",
    "averageCost": 0,
    "description": "<string>"
  }
}
```

### Request examples

#### cURL

```bash
curl --request POST \
  --url 'https://realtime-api.voiceflow.com/v1/stable/evaluation?projectID=<projectID>' \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "type": "boolean",
  "truePrompt": "<string>",
  "falsePrompt": "<string>",
  "name": "<string>",
  "prompt": "<string>",
  "enabled": true,
  "description": "<string>"
}'
```

#### JavaScript

```javascript
const response = await fetch('https://realtime-api.voiceflow.com/v1/stable/evaluation?projectID=<projectID>', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <token>',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    "type": "boolean",
    "truePrompt": "<string>",
    "falsePrompt": "<string>",
    "name": "<string>",
    "prompt": "<string>",
    "enabled": true,
    "description": "<string>"
  }),
});
const data = await response.json();
```

#### Python

```python
import requests

response = requests.post(
    'https://realtime-api.voiceflow.com/v1/stable/evaluation?projectID=<projectID>',
    headers={'Authorization': 'Bearer <token>'},
    json={
      "type": "boolean",
      "truePrompt": "<string>",
      "falsePrompt": "<string>",
      "name": "<string>",
      "prompt": "<string>",
      "enabled": True,
      "description": "<string>"
    },
)
data = response.json()
```

#### Voiceflow CLI

```bash
vf evaluation create --project-id <projectID> <<'JSON'
{
  "type": "boolean",
  "truePrompt": "<string>",
  "falsePrompt": "<string>",
  "name": "<string>",
  "prompt": "<string>",
  "enabled": true,
  "description": "<string>"
}
JSON
```


## Get evaluation

Returns a single evaluation by ID, with the fields that define it. An evaluation is a criterion a model scores past conversations against.

### Endpoint

```http
GET /v1/stable/evaluation/{evaluationID}
```

Server: https://realtime-api.voiceflow.com

### Authentication

```json
[
  {
    "token": []
  }
]
```

`token`:

```json
{
  "scheme": "bearer",
  "bearerFormat": "JWT",
  "type": "http",
  "description": "Voiceflow bearer token"
}
```

### Path parameters

#### `evaluationID`

Required · string

The ID of the evaluation to operate on.

```json
{
  "type": "string"
}
```

### Query parameters

#### `projectID`

Required · string

The ID of the project to operate on.

```json
{
  "type": "string",
  "description": "The ID of the project to operate on."
}
```

### Responses

#### 200

Returns one evaluation in full: the criteria prompts for its result type, the model settings the evaluator judges with, and whether it is one of the built-in defaults.

Schema:

```json
{
  "$ref": "#/components/schemas/StableEvaluationResponse"
}
```

Example:

```json
{
  "evaluation": {
    "id": "<string>",
    "name": "<string>",
    "prompt": "<string>",
    "default": true,
    "enabled": true,
    "projectID": "<string>",
    "settings": "<AgentSettings>",
    "averageCost": 0,
    "description": "<string>"
  }
}
```

### Request examples

#### cURL

```bash
curl --request GET \
  --url 'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>' \
  --header 'Authorization: Bearer <token>'
```

#### JavaScript

```javascript
const response = await fetch('https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>', {
  method: 'GET',
  headers: {
    Authorization: 'Bearer <token>',
  },
});
const data = await response.json();
```

#### Python

```python
import requests

response = requests.get(
    'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>',
    headers={'Authorization': 'Bearer <token>'},
)
data = response.json()
```

#### Voiceflow CLI

```bash
vf evaluation get --evaluation-id <evaluationID> --project-id <projectID>
```


## Update evaluation

Changes an existing evaluation in place. The response carries only a confirmation message, so refetch the evaluation to read its new values.

### Endpoint

```http
PATCH /v1/stable/evaluation/{evaluationID}
```

Server: https://realtime-api.voiceflow.com

### Authentication

```json
[
  {
    "token": []
  }
]
```

`token`:

```json
{
  "scheme": "bearer",
  "bearerFormat": "JWT",
  "type": "http",
  "description": "Voiceflow bearer token"
}
```

### Path parameters

#### `evaluationID`

Required · string

The ID of the evaluation to operate on.

```json
{
  "type": "string"
}
```

### Query parameters

#### `projectID`

Required · string

The ID of the project to operate on.

```json
{
  "type": "string",
  "description": "The ID of the project to operate on."
}
```

### Request body

Required

```json
{
  "required": true,
  "content": {
    "application/json": {
      "schema": {
        "$ref": "#/components/schemas/StableEvaluationUpdateRequest"
      }
    }
  }
}
```

### Responses

#### 200

Confirms the evaluation's name, criteria, result type, or `enabled` state were updated. The body is a message only, so refetch the evaluation to read its stored definition.

Schema:

```json
{
  "$ref": "#/components/schemas/StableUpdateResponse"
}
```

Example:

```json
{
  "message": "<string>"
}
```

### Request examples

#### cURL

```bash
curl --request PATCH \
  --url 'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>' \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "truePrompt": "<string>",
  "falsePrompt": "<string>",
  "name": "<string>",
  "prompt": "<string>",
  "enabled": true,
  "settings": {
    "model": "<AIModel>",
    "realtime": {
      "voice": "<string>",
      "eagerness": "<string>"
    },
    "maxTokens": 0,
    "temperature": 0,
    "reasoningEffort": "<AIReasoningEffort>",
    "serviceTier": "<AIParamsServiceTier>"
  },
  "description": "<string>",
  "type": "boolean"
}'
```

#### JavaScript

```javascript
const response = await fetch('https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>', {
  method: 'PATCH',
  headers: {
    Authorization: 'Bearer <token>',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    "truePrompt": "<string>",
    "falsePrompt": "<string>",
    "name": "<string>",
    "prompt": "<string>",
    "enabled": true,
    "settings": {
      "model": "<AIModel>",
      "realtime": {
        "voice": "<string>",
        "eagerness": "<string>"
      },
      "maxTokens": 0,
      "temperature": 0,
      "reasoningEffort": "<AIReasoningEffort>",
      "serviceTier": "<AIParamsServiceTier>"
    },
    "description": "<string>",
    "type": "boolean"
  }),
});
const data = await response.json();
```

#### Python

```python
import requests

response = requests.patch(
    'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>',
    headers={'Authorization': 'Bearer <token>'},
    json={
      "truePrompt": "<string>",
      "falsePrompt": "<string>",
      "name": "<string>",
      "prompt": "<string>",
      "enabled": True,
      "settings": {
        "model": "<AIModel>",
        "realtime": {
          "voice": "<string>",
          "eagerness": "<string>"
        },
        "maxTokens": 0,
        "temperature": 0,
        "reasoningEffort": "<AIReasoningEffort>",
        "serviceTier": "<AIParamsServiceTier>"
      },
      "description": "<string>",
      "type": "boolean"
    },
)
data = response.json()
```

#### Voiceflow CLI

```bash
vf evaluation update --evaluation-id <evaluationID> --project-id <projectID> <<'JSON'
{
  "truePrompt": "<string>",
  "falsePrompt": "<string>",
  "name": "<string>",
  "prompt": "<string>",
  "enabled": true,
  "settings": {
    "model": "<AIModel>",
    "realtime": {
      "voice": "<string>",
      "eagerness": "<string>"
    },
    "maxTokens": 0,
    "temperature": 0,
    "reasoningEffort": "<AIReasoningEffort>",
    "serviceTier": "<AIParamsServiceTier>"
  },
  "description": "<string>",
  "type": "boolean"
}
JSON
```


## Delete evaluation

Deletes a single evaluation by ID. An evaluation is a criterion a model scores past conversations against. The response carries only a confirmation message, so refetch the evaluations to see what is left.

### Endpoint

```http
DELETE /v1/stable/evaluation/{evaluationID}
```

Server: https://realtime-api.voiceflow.com

### Authentication

```json
[
  {
    "token": []
  }
]
```

`token`:

```json
{
  "scheme": "bearer",
  "bearerFormat": "JWT",
  "type": "http",
  "description": "Voiceflow bearer token"
}
```

### Path parameters

#### `evaluationID`

Required · string

The ID of the evaluation to operate on.

```json
{
  "type": "string"
}
```

### Query parameters

#### `projectID`

Required · string

The ID of the project to operate on.

```json
{
  "type": "string",
  "description": "The ID of the project to operate on."
}
```

### Responses

#### 200

Confirms the evaluation was removed from the project. The body carries a message only, so drop any cached copy of the project's evaluations.

Schema:

```json
{
  "$ref": "#/components/schemas/StableDeleteResponse"
}
```

Example:

```json
{
  "message": "<string>"
}
```

### Request examples

#### cURL

```bash
curl --request DELETE \
  --url 'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>' \
  --header 'Authorization: Bearer <token>'
```

#### JavaScript

```javascript
const response = await fetch('https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>', {
  method: 'DELETE',
  headers: {
    Authorization: 'Bearer <token>',
  },
});
const data = await response.json();
```

#### Python

```python
import requests

response = requests.delete(
    'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>?projectID=<projectID>',
    headers={'Authorization': 'Bearer <token>'},
)
data = response.json()
```

#### Voiceflow CLI

```bash
vf evaluation delete --evaluation-id <evaluationID> --project-id <projectID>
```


## Run evaluation

Scores one finished conversation against an evaluation. The response carries the verdict together with the reasoning behind it, so a low score says why.

### Endpoint

```http
POST /v1/stable/evaluation/{evaluationID}/run
```

Server: https://realtime-api.voiceflow.com

### Authentication

```json
[
  {
    "token": []
  }
]
```

`token`:

```json
{
  "scheme": "bearer",
  "bearerFormat": "JWT",
  "type": "http",
  "description": "Voiceflow bearer token"
}
```

### Path parameters

#### `evaluationID`

Required · string

The ID of the evaluation to operate on.

```json
{
  "type": "string"
}
```

### Query parameters

#### `projectID`

Required · string

The ID of the project to operate on.

```json
{
  "type": "string",
  "description": "The ID of the project to operate on."
}
```

### Request body

Required

```json
{
  "required": true,
  "content": {
    "application/json": {
      "schema": {
        "$ref": "#/components/schemas/StableEvaluationRunRequest"
      }
    }
  }
}
```

### Responses

#### 201

Scores the transcript named in the body against this evaluation and returns the result: the `value` the evaluator produced, typed to match the evaluation, the `reason` behind it, and the LLM cost the run incurred.

Schema:

```json
{
  "$ref": "#/components/schemas/StableEvaluationRunResponse"
}
```

Example:

```json
{
  "result": {
    "cost": 0,
    "value": true,
    "reason": "<string>",
    "transcriptID": "<string>",
    "evaluationID": "<string>"
  }
}
```

### Request examples

#### cURL

```bash
curl --request POST \
  --url 'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>/run?projectID=<projectID>' \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "transcriptID": "<string>"
}'
```

#### JavaScript

```javascript
const response = await fetch('https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>/run?projectID=<projectID>', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <token>',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    "transcriptID": "<string>"
  }),
});
const data = await response.json();
```

#### Python

```python
import requests

response = requests.post(
    'https://realtime-api.voiceflow.com/v1/stable/evaluation/<evaluationID>/run?projectID=<projectID>',
    headers={'Authorization': 'Bearer <token>'},
    json={
      "transcriptID": "<string>"
    },
)
data = response.json()
```

#### Voiceflow CLI

```bash
vf evaluation run --evaluation-id <evaluationID> --project-id <projectID> <<'JSON'
{
  "transcriptID": "<string>"
}
JSON
```


## Queue evaluation runs

Queue evaluations against transcripts.

### Endpoint

```http
POST /v1/stable/evaluation/queue-runs
```

Server: https://realtime-api.voiceflow.com

### Authentication

```json
[
  {
    "token": []
  }
]
```

`token`:

```json
{
  "scheme": "bearer",
  "bearerFormat": "JWT",
  "type": "http",
  "description": "Voiceflow bearer token"
}
```

### Query parameters

#### `projectID`

Required · string

The ID of the project to operate on.

```json
{
  "type": "string",
  "description": "The ID of the project to operate on."
}
```

### Request body

Required

```json
{
  "required": true,
  "content": {
    "application/json": {
      "schema": {
        "$ref": "#/components/schemas/StableEvaluationQueueRunsRequest"
      }
    }
  }
}
```

### Responses

#### 202

Schema:

```json
{
  "$ref": "#/components/schemas/StableEvaluationQueueRunsResponse"
}
```

Example:

```json
{
  "evaluationCount": 0,
  "transcriptCount": 0,
  "warning": {
    "type": "quota_exceeded",
    "message": "<string>",
    "skippedTranscriptIDs": [
      "<string>"
    ]
  }
}
```

### Request examples

#### cURL

```bash
curl --request POST \
  --url 'https://realtime-api.voiceflow.com/v1/stable/evaluation/queue-runs?projectID=<projectID>' \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "evaluationIDs": [
    "<string>"
  ],
  "transcriptIDs": [
    "<string>"
  ]
}'
```

#### JavaScript

```javascript
const response = await fetch('https://realtime-api.voiceflow.com/v1/stable/evaluation/queue-runs?projectID=<projectID>', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <token>',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    "evaluationIDs": [
      "<string>"
    ],
    "transcriptIDs": [
      "<string>"
    ]
  }),
});
const data = await response.json();
```

#### Python

```python
import requests

response = requests.post(
    'https://realtime-api.voiceflow.com/v1/stable/evaluation/queue-runs?projectID=<projectID>',
    headers={'Authorization': 'Bearer <token>'},
    json={
      "evaluationIDs": [
        "<string>"
      ],
      "transcriptIDs": [
        "<string>"
      ]
    },
)
data = response.json()
```

#### Voiceflow CLI

```bash
vf evaluation queue-runs --project-id <projectID> <<'JSON'
{
  "evaluationIDs": [
    "<string>"
  ],
  "transcriptIDs": [
    "<string>"
  ]
}
JSON
```


## Schema definitions

### AIModel

```json
{
  "type": "string",
  "enum": [
    "gpt-3.5-turbo-1106",
    "gpt-3.5-turbo",
    "gpt-4",
    "gpt-4o",
    "gpt-4o-mini",
    "gpt-4.1-2025-04-14",
    "gpt-4.1-mini-2025-04-14",
    "gpt-4.1-nano-2025-04-14",
    "gpt-o3-mini",
    "o3-2025-04-16",
    "o4-mini-2025-04-16",
    "gpt-5",
    "gpt-5-mini",
    "gpt-5-nano",
    "gpt-5.2",
    "gpt-5.4",
    "gpt-5.4-mini",
    "gpt-5.5",
    "gpt-5.6-sol",
    "gpt-5.6-terra",
    "gpt-5.6-luna",
    "gpt-6-sol",
    "gpt-6-luna",
    "gpt-6.1-sol",
    "gpt-realtime",
    "gpt-live-1",
    "claude-4-opus",
    "claude-4-sonnet",
    "claude-4.5-sonnet",
    "claude-4.6-sonnet",
    "claude-4.5-haiku",
    "claude-4.5-opus",
    "claude-4.7-opus",
    "bedrock-claude-4-sonnet",
    "bedrock-claude-4.5-sonnet",
    "bedrock-claude-4.6-sonnet",
    "bedrock-claude-5-sonnet",
    "bedrock-claude-5.5-sonnet",
    "bedrock-claude-4.5-haiku",
    "bedrock-claude-5.5-haiku",
    "bedrock-claude-4.5-opus",
    "bedrock-claude-4.7-opus",
    "bedrock-claude-4.8-opus",
    "voiceflow-core-4.0",
    "voiceflow-core-4.1",
    "voiceflow-core-4.2",
    "voiceflow-flash-4.2",
    "gemini-2.5-pro",
    "gemini-2.5-flash",
    "gemini-live-2.5-flash",
    "gemini-3-flash",
    "gemini-3.5-flash",
    "gemini-3.1-pro",
    "gpt-oss-20b",
    "gpt-oss-120b",
    "llama-guard-4",
    "llama-3.1-instant",
    "llama-3.3-versatile",
    "qwen3.6-27b",
    "grok-4.6",
    "voiceflow-flash-4.1",
    "kimi-k3",
    "gemini-flash-2",
    "gpt-4-turbo",
    "claude-3.5-haiku",
    "claude-3-opus",
    "claude-3.7-sonnet",
    "claude-v2",
    "llama-3.2-1b-preview",
    "gemini-pro-1.5",
    "claude-3-haiku",
    "claude-3-sonnet",
    "text-davinci-003",
    "claude-v1",
    "claude-instant-v1",
    "deep-seek-r1-distill-llama-70B",
    "claude-3.5-sonnet"
  ],
  "x-enumNames": [
    "GPT_3_5_TURBO_1106",
    "GPT_3_5_TURBO",
    "GPT_4",
    "GPT_4_O",
    "GPT_4_O_MINI",
    "GPT_4_1",
    "GPT_4_1_MINI",
    "GPT_4_1_NANO",
    "GPT_O_3_MINI",
    "GPT_O_3",
    "GPT_O_4_MINI",
    "GPT_5",
    "GPT_5_MINI",
    "GPT_5_NANO",
    "GPT_5_2",
    "GPT_5_4",
    "GPT_5_4_MINI",
    "GPT_5_5",
    "GPT_5_6_SOL",
    "GPT_5_6_TERRA",
    "GPT_5_6_LUNA",
    "GPT_6_SOL",
    "GPT_6_LUNA",
    "GPT_6_1_SOL",
    "GPT_REALTIME",
    "GPT_LIVE_1",
    "CLAUDE_4_OPUS",
    "CLAUDE_4_SONNET",
    "CLAUDE_4_5_SONNET",
    "CLAUDE_4_6_SONNET",
    "CLAUDE_4_5_HAIKU",
    "CLAUDE_4_5_OPUS",
    "CLAUDE_4_7_OPUS",
    "BEDROCK_CLAUDE_4_SONNET",
    "BEDROCK_CLAUDE_4_5_SONNET",
    "BEDROCK_CLAUDE_4_6_SONNET",
    "BEDROCK_CLAUDE_5_SONNET",
    "BEDROCK_CLAUDE_5_5_SONNET",
    "BEDROCK_CLAUDE_4_5_HAIKU",
    "BEDROCK_CLAUDE_5_5_HAIKU",
    "BEDROCK_CLAUDE_4_5_OPUS",
    "BEDROCK_CLAUDE_4_7_OPUS",
    "BEDROCK_CLAUDE_4_8_OPUS",
    "VOICEFLOW_CORE_4_0",
    "VOICEFLOW_CORE_4_1",
    "VOICEFLOW_CORE_4_2",
    "VOICEFLOW_FLASH_4_2",
    "GEMINI_2_5_PRO",
    "GEMINI_2_5_FLASH",
    "GEMINI_LIVE_2_5_FLASH",
    "GEMINI_3_FLASH",
    "GEMINI_3_5_FLASH",
    "GEMINI_3_1_PRO",
    "GPT_OSS_20_B",
    "GPT_OSS_120_B",
    "LLAMA_GUARD_4",
    "LLAMA_3_1_INSTANT",
    "LLAMA_3_3_VERSATILE",
    "QWEN_3_6_27_B",
    "GROK_4_6",
    "VOICEFLOW_FLASH_4_1",
    "KIMI_K_3",
    "GEMINI_FLASH_2",
    "GPT_4_TURBO",
    "CLAUDE_3_5_HAIKU",
    "CLAUDE_3_OPUS",
    "CLAUDE_3_7_SONNET",
    "CLAUDE_V_2",
    "LLAMA_3_2_1_B_PREVIEW",
    "GEMINI_PRO_1_5",
    "CLAUDE_3_HAIKU",
    "CLAUDE_3_SONNET",
    "DA_VINCI_003",
    "CLAUDE_V_1",
    "CLAUDE_INSTANT_V_1",
    "DEEP_SEEK_R_1_DISTILL_LLAMA_70_B",
    "CLAUDE_3_5_SONNET"
  ]
}
```

### AIParamsServiceTier

```json
{
  "type": "string",
  "enum": [
    "default",
    "flex",
    "priority"
  ],
  "x-enumNames": [
    "DEFAULT",
    "FLEX",
    "PRIORITY"
  ]
}
```

### AIReasoningEffort

```json
{
  "type": "string",
  "enum": [
    "off",
    "adaptive",
    "minimal",
    "low",
    "medium",
    "high"
  ],
  "x-enumNames": [
    "OFF",
    "ADAPTIVE",
    "MINIMAL",
    "LOW",
    "MEDIUM",
    "HIGH"
  ]
}
```

### AgentSettings

```json
{
  "type": "object",
  "properties": {
    "model": {
      "allOf": [
        {
          "$ref": "#/components/schemas/AIModel"
        }
      ]
    },
    "realtime": {
      "type": "object",
      "properties": {
        "voice": {
          "type": "string"
        },
        "eagerness": {
          "type": "string"
        }
      }
    },
    "maxTokens": {
      "type": "number"
    },
    "temperature": {
      "type": "number"
    },
    "reasoningEffort": {
      "nullable": true,
      "allOf": [
        {
          "$ref": "#/components/schemas/AIReasoningEffort"
        }
      ]
    },
    "serviceTier": {
      "description": "The service tier to request from models that support it. `priority` is faster processing: faster responses at a higher credit cost. When unset, the deprecated project-level `priorityProcessing` setting applies.",
      "nullable": true,
      "allOf": [
        {
          "$ref": "#/components/schemas/AIParamsServiceTier"
        }
      ]
    }
  }
}
```

### StableDeleteResponse

```json
{
  "type": "object",
  "properties": {
    "message": {
      "type": "string",
      "description": "A human-readable message confirming the deletion."
    }
  },
  "required": [
    "message"
  ]
}
```

### StableEvaluation

```json
{
  "allOf": [
    {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "default": {
          "type": "boolean",
          "description": "Whether this is a built-in default evaluation."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "projectID": {
          "type": "string",
          "description": "The ID of the project this evaluation belongs to."
        },
        "settings": {
          "nullable": true,
          "allOf": [
            {
              "$ref": "#/components/schemas/AgentSettings"
            }
          ]
        },
        "averageCost": {
          "nullable": true,
          "description": "The average LLM cost incurred each time this evaluation runs.",
          "type": "number"
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        }
      },
      "required": [
        "id",
        "name",
        "prompt",
        "default",
        "enabled",
        "projectID",
        "settings",
        "averageCost",
        "description"
      ]
    },
    {
      "oneOf": [
        {
          "type": "object",
          "properties": {
            "type": {
              "type": "string",
              "enum": [
                "boolean"
              ],
              "description": "Identifies an evaluation that produces a true/false result.",
              "x-enumNames": [
                "BOOLEAN"
              ]
            },
            "truePrompt": {
              "type": "string",
              "description": "The criteria describing when the evaluator should return `true`."
            },
            "falsePrompt": {
              "type": "string",
              "description": "The criteria describing when the evaluator should return `false`."
            }
          },
          "required": [
            "type",
            "truePrompt",
            "falsePrompt"
          ]
        },
        {
          "type": "object",
          "properties": {
            "type": {
              "type": "string",
              "enum": [
                "number"
              ],
              "description": "Identifies an evaluation that produces a numeric score.",
              "x-enumNames": [
                "NUMBER"
              ]
            },
            "minimumValue": {
              "type": "number",
              "description": "The lowest score the evaluator can assign."
            },
            "maximumValue": {
              "type": "number",
              "description": "The highest score the evaluator can assign."
            },
            "minimumPrompt": {
              "type": "string",
              "description": "The criteria describing what warrants the `minimumValue` score."
            },
            "maximumPrompt": {
              "type": "string",
              "description": "The criteria describing what warrants the `maximumValue` score."
            }
          },
          "required": [
            "type",
            "minimumValue",
            "maximumValue",
            "minimumPrompt",
            "maximumPrompt"
          ]
        },
        {
          "type": "object",
          "properties": {
            "type": {
              "type": "string",
              "enum": [
                "string"
              ],
              "description": "Identifies an evaluation that produces a free-form text result.",
              "x-enumNames": [
                "STRING"
              ]
            }
          },
          "required": [
            "type"
          ]
        },
        {
          "type": "object",
          "properties": {
            "type": {
              "type": "string",
              "enum": [
                "option"
              ],
              "description": "Identifies an evaluation that selects one of a predefined set of options.",
              "x-enumNames": [
                "OPTION"
              ]
            },
            "options": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "value": {
                    "type": "string",
                    "description": "The value assigned to the evaluation result when this option is selected."
                  },
                  "prompt": {
                    "type": "string",
                    "description": "The criteria describing when the evaluator should select this option."
                  },
                  "included": {
                    "type": "boolean",
                    "description": "Whether this option is available for the evaluator to select."
                  },
                  "color": {
                    "type": "string",
                    "description": "The color used to display this option in the Voiceflow UI."
                  }
                },
                "required": [
                  "value",
                  "prompt",
                  "included",
                  "color"
                ]
              }
            }
          },
          "required": [
            "type",
            "options"
          ]
        }
      ]
    }
  ]
}
```

### StableEvaluationCreateRequest

```json
{
  "oneOf": [
    {
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "boolean"
          ],
          "description": "Identifies an evaluation that produces a true/false result.",
          "x-enumNames": [
            "BOOLEAN"
          ]
        },
        "truePrompt": {
          "type": "string",
          "description": "The criteria describing when the evaluator should return `true`."
        },
        "falsePrompt": {
          "type": "string",
          "description": "The criteria describing when the evaluator should return `false`."
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        }
      },
      "required": [
        "type",
        "truePrompt",
        "falsePrompt",
        "name",
        "prompt",
        "enabled"
      ],
      "additionalProperties": false
    },
    {
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "number"
          ],
          "description": "Identifies an evaluation that produces a numeric score.",
          "x-enumNames": [
            "NUMBER"
          ]
        },
        "minimumValue": {
          "type": "number",
          "description": "The lowest score the evaluator can assign."
        },
        "maximumValue": {
          "type": "number",
          "description": "The highest score the evaluator can assign."
        },
        "minimumPrompt": {
          "type": "string",
          "description": "The criteria describing what warrants the `minimumValue` score."
        },
        "maximumPrompt": {
          "type": "string",
          "description": "The criteria describing what warrants the `maximumValue` score."
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        }
      },
      "required": [
        "type",
        "minimumValue",
        "maximumValue",
        "minimumPrompt",
        "maximumPrompt",
        "name",
        "prompt",
        "enabled"
      ],
      "additionalProperties": false
    },
    {
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "string"
          ],
          "description": "Identifies an evaluation that produces a free-form text result.",
          "x-enumNames": [
            "STRING"
          ]
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        }
      },
      "required": [
        "type",
        "name",
        "prompt",
        "enabled"
      ],
      "additionalProperties": false
    },
    {
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "option"
          ],
          "description": "Identifies an evaluation that selects one of a predefined set of options.",
          "x-enumNames": [
            "OPTION"
          ]
        },
        "options": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "value": {
                "type": "string",
                "description": "The value assigned to the evaluation result when this option is selected."
              },
              "prompt": {
                "type": "string",
                "description": "The criteria describing when the evaluator should select this option."
              },
              "included": {
                "type": "boolean",
                "description": "Whether this option is available for the evaluator to select."
              },
              "color": {
                "type": "string",
                "description": "The color used to display this option in the Voiceflow UI."
              }
            },
            "required": [
              "value",
              "prompt",
              "included",
              "color"
            ]
          }
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        }
      },
      "required": [
        "type",
        "options",
        "name",
        "prompt",
        "enabled"
      ],
      "additionalProperties": false
    }
  ]
}
```

### StableEvaluationListResponse

```json
{
  "type": "object",
  "properties": {
    "evaluations": {
      "type": "array",
      "items": {
        "$ref": "#/components/schemas/StableEvaluation"
      }
    }
  },
  "required": [
    "evaluations"
  ]
}
```

### StableEvaluationQueueRunsRequest

```json
{
  "type": "object",
  "properties": {
    "evaluationIDs": {
      "minItems": 1,
      "maxItems": 10,
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "The evaluations to run, 1 to 10. Every evaluation runs against every transcript."
    },
    "transcriptIDs": {
      "minItems": 1,
      "maxItems": 100,
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "The transcripts to score, 1 to 100."
    }
  },
  "required": [
    "evaluationIDs",
    "transcriptIDs"
  ],
  "additionalProperties": false
}
```

### StableEvaluationQueueRunsResponse

```json
{
  "type": "object",
  "properties": {
    "evaluationCount": {
      "type": "integer",
      "minimum": -9007199254740991,
      "maximum": 9007199254740991,
      "description": "Evaluation runs queued across all transcripts."
    },
    "transcriptCount": {
      "type": "integer",
      "minimum": -9007199254740991,
      "maximum": 9007199254740991,
      "description": "Transcripts queued for at least one evaluation. Lower than the number sent when some were skipped: another project, not ended yet, or expired."
    },
    "warning": {
      "description": "Set when the evaluation quota made the queue skip transcripts; it names them.",
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "quota_exceeded"
          ],
          "x-enumNames": [
            "QUOTA_EXCEEDED"
          ]
        },
        "message": {
          "type": "string"
        },
        "skippedTranscriptIDs": {
          "type": "array",
          "items": {
            "type": "string"
          }
        }
      },
      "required": [
        "type",
        "message",
        "skippedTranscriptIDs"
      ]
    }
  },
  "required": [
    "evaluationCount",
    "transcriptCount"
  ]
}
```

### StableEvaluationResponse

```json
{
  "type": "object",
  "properties": {
    "evaluation": {
      "$ref": "#/components/schemas/StableEvaluation"
    }
  },
  "required": [
    "evaluation"
  ]
}
```

### StableEvaluationResult

```json
{
  "type": "object",
  "properties": {
    "cost": {
      "type": "number",
      "description": "The LLM cost incurred by this evaluation run."
    },
    "value": {
      "anyOf": [
        {
          "type": "boolean"
        },
        {
          "type": "number"
        },
        {
          "type": "string"
        }
      ],
      "description": "The result produced by the evaluator; its type matches the evaluation `type`."
    },
    "reason": {
      "type": "string",
      "description": "The evaluator's explanation for the returned `value`."
    },
    "transcriptID": {
      "type": "string",
      "description": "The ID of the transcript that was evaluated."
    },
    "evaluationID": {
      "type": "string",
      "description": "The ID of the evaluation that was run."
    }
  },
  "required": [
    "cost",
    "value",
    "reason",
    "transcriptID",
    "evaluationID"
  ]
}
```

### StableEvaluationRunRequest

```json
{
  "type": "object",
  "properties": {
    "transcriptID": {
      "type": "string",
      "description": "The ID of the transcript to run the evaluation against."
    }
  },
  "required": [
    "transcriptID"
  ],
  "additionalProperties": false
}
```

### StableEvaluationRunResponse

```json
{
  "type": "object",
  "properties": {
    "result": {
      "$ref": "#/components/schemas/StableEvaluationResult"
    }
  },
  "required": [
    "result"
  ]
}
```

### StableEvaluationUpdateRequest

```json
{
  "oneOf": [
    {
      "type": "object",
      "properties": {
        "truePrompt": {
          "type": "string",
          "description": "The criteria describing when the evaluator should return `true`."
        },
        "falsePrompt": {
          "type": "string",
          "description": "The criteria describing when the evaluator should return `false`."
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "settings": {
          "nullable": true,
          "allOf": [
            {
              "$ref": "#/components/schemas/AgentSettings"
            }
          ]
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        },
        "type": {
          "type": "string",
          "enum": [
            "boolean"
          ],
          "description": "Identifies an evaluation that produces a true/false result.",
          "x-enumNames": [
            "BOOLEAN"
          ]
        }
      },
      "required": [
        "type"
      ],
      "additionalProperties": false
    },
    {
      "type": "object",
      "properties": {
        "minimumValue": {
          "type": "number",
          "description": "The lowest score the evaluator can assign."
        },
        "maximumValue": {
          "type": "number",
          "description": "The highest score the evaluator can assign."
        },
        "minimumPrompt": {
          "type": "string",
          "description": "The criteria describing what warrants the `minimumValue` score."
        },
        "maximumPrompt": {
          "type": "string",
          "description": "The criteria describing what warrants the `maximumValue` score."
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "settings": {
          "nullable": true,
          "allOf": [
            {
              "$ref": "#/components/schemas/AgentSettings"
            }
          ]
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        },
        "type": {
          "type": "string",
          "enum": [
            "number"
          ],
          "description": "Identifies an evaluation that produces a numeric score.",
          "x-enumNames": [
            "NUMBER"
          ]
        }
      },
      "required": [
        "type"
      ],
      "additionalProperties": false
    },
    {
      "type": "object",
      "properties": {
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "settings": {
          "nullable": true,
          "allOf": [
            {
              "$ref": "#/components/schemas/AgentSettings"
            }
          ]
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        },
        "type": {
          "type": "string",
          "enum": [
            "string"
          ],
          "description": "Identifies an evaluation that produces a free-form text result.",
          "x-enumNames": [
            "STRING"
          ]
        }
      },
      "required": [
        "type"
      ],
      "additionalProperties": false
    },
    {
      "type": "object",
      "properties": {
        "options": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "value": {
                "type": "string",
                "description": "The value assigned to the evaluation result when this option is selected."
              },
              "prompt": {
                "type": "string",
                "description": "The criteria describing when the evaluator should select this option."
              },
              "included": {
                "type": "boolean",
                "description": "Whether this option is available for the evaluator to select."
              },
              "color": {
                "type": "string",
                "description": "The color used to display this option in the Voiceflow UI."
              }
            },
            "required": [
              "value",
              "prompt",
              "included",
              "color"
            ]
          }
        },
        "name": {
          "type": "string"
        },
        "prompt": {
          "type": "string",
          "description": "The criteria the evaluator uses to judge a transcript."
        },
        "enabled": {
          "type": "boolean",
          "description": "Whether this evaluation runs automatically against new transcripts."
        },
        "settings": {
          "nullable": true,
          "allOf": [
            {
              "$ref": "#/components/schemas/AgentSettings"
            }
          ]
        },
        "description": {
          "nullable": true,
          "description": "A human-readable description of what this evaluation measures.",
          "type": "string"
        },
        "type": {
          "type": "string",
          "enum": [
            "option"
          ],
          "description": "Identifies an evaluation that selects one of a predefined set of options.",
          "x-enumNames": [
            "OPTION"
          ]
        }
      },
      "required": [
        "type"
      ],
      "additionalProperties": false
    }
  ]
}
```

### StableUpdateResponse

```json
{
  "type": "object",
  "properties": {
    "message": {
      "type": "string",
      "description": "A human-readable message confirming the update."
    }
  },
  "required": [
    "message"
  ]
}
```
