> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# Scenarios

> A scenario is a situation a simulated user acts out against your agent, and the criteria a judge scores the conversation against.

## Situation and criteria

The situation describes the user and what they want, in one or a few sentences. The simulated user plays it turn by turn and improvises the words, so a scenario is not a script. The criteria, one per line, are what the judge checks at the end of the conversation. Each one passes, fails or is inconclusive.

For example, a checkout support agent gets this scenario:

```text title="Situation" theme={null}
The customer was charged twice for the same order, minutes apart, and wants one of the charges back.
```

```text title="Criteria" theme={null}
Verifies the order before it promises anything
Issues exactly one refund, not two
States the 5 to 10 business day timeline
```

Write criteria about outcomes and actions, not wording. A criterion about an internal action, such as "looks up the order before answering", is verified against the agent's own traces, so the judge does not rely on what the reply claims. See [Linking your traces](https://langwatch.ai/docs/agent-testing/linking-your-traces.md).

## The editor

**New scenario** on the Scenarios tab, or **Edit** on a row, opens the editor.

![The scenario editor with the title, the test suite, the situation, the criteria and the labels](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/scenario-editor.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=bd2587e27c55076239b29387cb4897ab)

| Field | What it is |
| - | - |
| **Title** | The name of the scenario, shown in every table and result. |
| **Test suite** | The test suite of the scenario. **No test suite** files it under **Default**. |
| **Situation** | Who the user is and what they are trying to do. |
| **Criteria** | One criterion per line. Blank lines are dropped. |
| **Labels** | Free text tags, any number. A run plan can scope by labels across test suites. |
| **Fields** | One control per field the test suite declares, after the criteria: a growing text box for a text field, a number input for a number, a switch for a boolean. See [Field values](#field-values). |

**Save** stores the scenario. **Save & Run** stores it and opens the [run dialog](https://langwatch.ai/docs/agent-testing/run-plans.md#the-run-dialog). **Open recent run** opens the latest run of this scenario.

## Field values

When the test suite declares [fields](https://langwatch.ai/docs/agent-testing/test-suites.md#fields), the editor shows one control per field after the criteria, and the value is stored on the scenario. An [evaluator](https://langwatch.ai/docs/agent-testing/evaluators.md) attached to the suite reads the value through its mapping, for example `expected_output` from `golden_sql`.

![The scenario editor with the golden\_sql and table\_schema boxes filled in under the criteria](https://mintcdn.com/langwatch/OHUpQQmmdnh4pjQ7/images/agent-testing/scenario-editor-field-values.png?fit=max\&auto=format\&n=OHUpQQmmdnh4pjQ7\&q=85\&s=3f6e29c72a3b83ceca5e920738aa4667)

The `golden_sql` and `table_schema` boxes under the criteria are the fields the `Case lookups` suite declares. A text box grows with its text.

A blank value is no value: the evaluators that read the field are skipped for this scenario, with the reason `no golden_sql on this scenario`. A scenario moved to another suite keeps the values it holds; the editor lists the ones the new suite does not declare under **Not in this suite**, each with a remove action.

From the CLI, `langwatch scenario create` and `langwatch scenario update` take `--field identifier=value`, repeated once per field. A number field reads a number and a boolean field reads `true` or `false`, and a field the suite does not declare is refused with the list it does. On `update` the flags together replace the values the scenario holds.

The API takes the same values as `fields` on `POST /api/scenarios` and `PUT /api/scenarios/{id}`, where an empty object clears them. The SDKs and the MCP tools take `fields` on create and update.

## Customize scenario

Three chips under the criteria open optional blocks:

![The Customize scenario chips expanded: parameters, min and max turns, and the model overrides](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/scenario-editor-customize.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=10f3f88eed9a24791d6ca425af8047e1)

* **Add parameters** declares run parameters as one line of `name=default` pairs, for example `plan=free, locale=de`. The situation, the criteria and the target read a value as `{{ params.NAME }}`. See [Run parameters](https://langwatch.ai/docs/agent-testing/run-parameters.md).
* **Define min and max turns** bounds the conversation. **Max turns** defaults to 10. **Min turns** has no default: the judge can end the conversation as soon as it has a verdict.
* **Override models** picks the **User simulator** and the **Judge** for this scenario alone. Without an override, the run's models apply.

## Versions

Every save is a new version. The editor shows the current one as `v3 · History`; open **History** to see every version with when it was saved and by whom. **Restore** on an older version writes its content forward as a new version, so every version stays in the history.

![The version history of a scenario, with Restore on an older version](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/scenario-version-history.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=a8e673b4ddcc72442c786d91ce52d776)

When someone else saves the scenario while you have it open, the editor tells you and offers to reload before you save over it.

## Actions on a scenario

The row menu of a scenario offers **Edit**, **Duplicate**, **Open recent runs**, **Move to suite...** and **Archive**. Select several rows to move them together.

![The row menu of a scenario](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/scenario-row-menu.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=305fa5970783be2692e0017119924805)

Archiving removes the scenario from the tables and from every future run. Its past runs stay on the Results tab.

## Create scenarios with Langy

Langy, the assistant in the bottom corner of every page, writes and saves scenarios from what you paste: a support ticket, a complaint, acceptance criteria, or a transcript of a conversation that went wrong. Ask for the scenario and check it in the editor afterwards.

## Create scenarios from your editor

Through the [LangWatch MCP integration](https://langwatch.ai/docs/integration/mcp.md), a coding assistant such as Claude Code or Cursor reads and writes the scenarios of your project. The REST API does the same: `POST /api/scenarios` creates one, `PATCH /api/scenarios/{id}` updates one. See the [Scenarios API reference](https://langwatch.ai/docs/api-reference/scenarios/overview.md).

## Limits

| Limit | Value |
| - | - |
| Parameters declared on one scenario | 20 |
| Fields declared on a test suite | 30 |
| A text field value | 65536 characters |
| Max turns without a value | 10 |
| Versions kept | All |

Also check: [Test suites](https://langwatch.ai/docs/agent-testing/test-suites.md), [Run plans](https://langwatch.ai/docs/agent-testing/run-plans.md), [Run parameters](https://langwatch.ai/docs/agent-testing/run-parameters.md), [Setup and teardown hooks](https://langwatch.ai/docs/agent-testing/setup-and-teardown-hooks.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.