> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# Answer Correctness: check answers against expected output

> Score every answer against the expected output in your dataset with the Exact Match or the LLM Answer Match evaluator, and compare runs as you change the prompt.

This guide builds an experiment for a customer support assistant where every question has a known correct answer. You run the questions through a prompt, score each answer against the expected output, and read the pass rate.

## Prerequisites

* A dataset with an `input` column and an `expected_output` column. Create one from a CSV on the [Datasets](https://app.langwatch.ai/@project/datasets) page, or create it inside the workbench in step 2. See [Datasets](https://langwatch.ai/docs/datasets/overview.md).
* A model provider configured in Settings, so the prompt column and the LLM Answer Match evaluator can call a model.

## Steps

**1. Create the experiment**

Open [Experiments](https://app.langwatch.ai/@project/experiments), click **New Experiment**, then **Create Experiment**. The workbench opens. Click the generated name in the header and rename it, for example `Support answer correctness`.

**2. Add the dataset**

Under **Datasets**, click **Add** and pick **Select existing dataset**. To start from a file instead, pick **Upload CSV**. To type the rows in, pick **Create new** and add the `input` and `expected_output` columns.

![The Datasets Add menu with Select existing dataset, Upload CSV and Create new](https://mintcdn.com/langwatch/iG2HKiUWiJlQldW1/images/evaluations/workbench-dataset-add-menu.png?fit=max\&auto=format\&n=iG2HKiUWiJlQldW1\&q=85\&s=7cbbbe1af188592876d3c5b9ae4d914d)

**3. Add a prompt column**

Under **Prompts or Agents**, click **Add** and pick **Prompt**, then **New Prompt**. In the editor, pick the model and write the system prompt. For example:

```text theme={null}
You are the customer support assistant for ACME.
Answer only from the ACME policies below. If the answer is not in the policies, say that you do not know and refer the customer to support@acme.example.

Policies:
- Returns are accepted within 30 days of purchase, in the original packaging.
- Standard shipping takes 3 to 5 business days.
- Orders can be changed or cancelled until they ship.
```

In the **Variables** section, check that `input` maps to the dataset's `input` column, so each row's question reaches the model. Click **Apply**.

![The prompt editor with the model selector and the Variables section mapping input to the dataset column](https://mintcdn.com/langwatch/iG2HKiUWiJlQldW1/images/evaluations/workbench-prompt-editor.png?fit=max\&auto=format\&n=iG2HKiUWiJlQldW1\&q=85\&s=813606c11cd59115f1df3a329b2656e3)

**4. Add the evaluator**

On the prompt column, click **Add evaluator**, then **New Evaluator**. Pick the **Expected Answer** category.

![The Expected Answer category with Exact Match Evaluator, LLM Answer Match, LLM Factual Match, ROUGE Score and BLEU Score](https://mintcdn.com/langwatch/iG2HKiUWiJlQldW1/images/evaluations/evaluator-type-selector.png?fit=max\&auto=format\&n=iG2HKiUWiJlQldW1\&q=85\&s=98d4df91bde7a06c60f57652df40dc5c)

Choose one of:

| Evaluator | Use it when |
| - | - |
| **Exact Match** | The answer must match the expected output character for character. Case, surrounding whitespace and punctuation are ignored by default; each is a setting. |
| **LLM Answer Match** | The answer may be worded differently and still be correct. A model compares the answer with the expected output and passes it when the meaning matches. |

Name the evaluator. In **Variables**, map `output` to the prompt column's output, `expected_output` to the dataset's `expected_output` column and `input` to the dataset's `input` column, then click **Create Evaluator**. If a required field stays unmapped, the workbench opens the mapping editor. Map the field there and click **Apply**.

![The LLM Answer Match editor with output mapped to the prompt output and expected\_output to the dataset column](https://mintcdn.com/langwatch/iG2HKiUWiJlQldW1/images/evaluations/workbench-evaluator-mapping.png?fit=max\&auto=format\&n=iG2HKiUWiJlQldW1\&q=85\&s=b4b7413759cc583f1f716cb61a7b69c1)

**5. Run**

Click **Run** in the header. Each cell fills with the model's answer and the evaluator chip on
each row turns into a pass or a fail. To rerun a single row, use the button on that cell.

**6. Read the results**

Hover over the summary in the column header. **Pass Rate** is the share of rows the evaluator passed. Click a failed chip to read the evaluator's details: for LLM Answer Match, the reason the model gave for the mismatch.

![The column summary with the pass rate and the per-evaluator breakdown](https://mintcdn.com/langwatch/iG2HKiUWiJlQldW1/images/evaluations/workbench-column-summary.png?fit=max\&auto=format\&n=iG2HKiUWiJlQldW1\&q=85\&s=f8b7132d89046a5395287c4d840ebaf5)

## Iterate and compare runs

Edit the prompt to fix the failing rows and click **Run** again. Every run is kept. Click **Results** in the header and tick the previous run and the new one in the **Experiment Runs** sidebar to see both side by side, with the pass rate of each run in the chart and both outputs per row, and check that no row that passed before now fails.

![Two experiment runs compared side by side on the results page](https://mintcdn.com/langwatch/iG2HKiUWiJlQldW1/images/evaluations/experiment-runs-compare.png?fit=max\&auto=format\&n=iG2HKiUWiJlQldW1\&q=85\&s=4176424542860cd1a2a7fa4b82844f27)

Each saved prompt version is what your application loads through the [prompt library](https://langwatch.ai/docs/prompt-management/overview.md), so the version that wins the comparison is the one you ship.

## Common failures

| Symptom | Cause |
| - | - |
| The run button reads **Configure missing mappings first** | An evaluator field or a prompt variable has no source. Open the editor and map the field marked **Required**. |
| Every evaluator chip shows an error | The evaluator's model has no provider key. Add one under Settings, Model Providers, or change the evaluator's model. |
| A cell shows **No output yet** | The column did not run for that row. Click the cell's run button. |

**Also check:** [LLM-as-a-judge](https://langwatch.ai/docs/evaluations/experiments/ui/llm-as-a-judge.md) for datasets without expected outputs, and [Run experiments from CI/CD](https://langwatch.ai/docs/evaluations/experiments/ci-cd.md) to trigger this experiment from a pipeline.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.