> ## Documentation Index > Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details. # Results > The Results tab lists every run plan of the project with its pass rate and trend. Open a plan for its runs, and a run for every conversation and verdict. ## The Results tab **Test Runs** shows the number of scenario executions in the period. The period is **Last 7 days**, **Last 15 days** or **Last 30 days**; runs older than 30 days are in cold storage and do not appear here. ![The Results tab grouped by run plan, with the pass rate and the trend of each plan](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/results-tab.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=ad98a25fed7aef968ea0fac6d479500d) **Group by** sets what a row is: | Group by | One row per | Columns | | - | - | - | | **Run plan** | Run plan | Run plan, Last run, Scope, Targets, Pass, Trend | | **Scenario** | Scenario, across every plan it ran in | Scenario, Labels, Runs, Pass, Trend | | **Target** | Target, with a parameter variant as its own target | Target, Scenarios covered, Runs, Pass, Trend | | **None** | Scenario execution | Scenario, Run plan, Target, When, Result | The filters narrow every grouping: **Scenario**, **Label**, **Target** and **All statuses**, **Passed** or **Failed**. **Reset filters** clears them. **Pass** is green at 100%, amber from 40%, red below, and grey with no result. **Trend** shows the last runs as bars, oldest first, at most 14. **Charts** opens the tiles **Executions**, **Pass rate**, **Failing scenarios** and **Cost** for the period, and **Pass rate over time**. ![The Charts block with the four tiles and the pass rate over time](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/results-charts.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=3d56432917343b1a8e72921505a20fcb) ## A run plan and its runs Press a run plan. The sidebar lists its runs as **Run #n** with the note, the time and the pass rate; **Load More...** loads older ones. The header of the selected run shows the pass rate, the total duration and the total cost. Hover it for **Completed**, **Avg Agent Latency**, **Avg Agent Cost**, **Total Duration** and **Total Cost**, with percentiles behind the two averages. ![A run plan with its runs in the sidebar, the summary pills and the table of results](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/run-detail-table.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=fd99a21f8a9e0d4292a07207cc883f58) **Show run settings** opens the configuration the run used: **Started**, **Targets**, **Parameters**, **Repeat**, **Simulator model** and **Judge model**. The **Table** view has one row per scenario execution: **Result**, **Scenario**, **Evaluators** and **Time · cost**. The **Evaluators** column appears only on a run that has evaluators, and its cell shows one pill per attached evaluator, with its verdict, its score or its status when it was skipped or hit an error; a row whose evaluators are still running reads `Grading`. The row menu offers **Open the conversation**, **Rerun this scenario** and **Edit scenario**. The **Grid** view shows the same executions as cards, one colour per result, each ending in `Completed` or `Failed`, or in `Evaluating` while the evaluators of the run are still reporting. ![The Grid view of a run, one card per scenario execution](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/run-detail-grid.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=6ef3b8cf154e4d2e45f5d96e728bc22d) **Export as CSV** downloads the run. **Run again** starts a new run with the plan's configuration, and **Stop all** cancels the conversations still running. ## A conversation Press a row to open the drawer. The header shows the **Status**, the criteria count, the **Duration**, the **Cost** and when it **Ran**. The transcript follows, with **View trace** on every turn whose trace has arrived. ![The conversation drawer of a failed run, with the failed criteria and the judge reasoning](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/run-drawer-failed.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=acaab28654c01ba39dad520c73b1a2a1) Under the transcript: **FAILED CRITERIA**, **PASSED CRITERIA** and **JUDGE REASONING**, then **Evaluators** when the suite or the plan attached any, then **Parameters** with the values the run used, a secret parameter masked. **Edit Scenario** opens the scenario in the editor. ## Evaluator results An [evaluator](https://langwatch.ai/docs/agent-testing/evaluators.md) attached to the test suite or to the run plan runs after the judge, and the run keeps one result per evaluator. The drawer lists them under the criteria, one row each: the status, the name, a mark when it is required, the verdict or the score, and the reason under it. ![The results page of a run: the header pills SQL Query Equivalence 67%, Answer clarity score 9.25 and PII Leak Scanner 100%, and one pill per evaluator on each row, one of them Skipped and one Fail](https://mintcdn.com/langwatch/OHUpQQmmdnh4pjQ7/images/agent-testing/results-run-evaluator-pills.png?fit=max\&auto=format\&n=OHUpQQmmdnh4pjQ7\&q=85\&s=11df22585b0fd96fd90713dd08e6b5e3) The header shows one pill per evaluator, and the **Evaluators** column shows one pill per evaluator on every row. A skipped evaluator keeps its pill, in grey. | Status | Meaning | | - | - | | `passed`, `failed` | The evaluator's verdict. When the trace never delivered a required input, the result is `failed` with the reason `no call in the trace`, `no retrieved contexts in the trace` or `no spans in the trace`. A failed **required** evaluator fails the scenario: the verdict reads `Failed · ` and the judge reasoning is kept. The reason says which input was missing. | | `scored` | The score of an evaluator with no pass or fail verdict. A score never fails a scenario. | | `skipped` | The scenario has no value for a field the mapping reads. The reason names the field. | | `error` | The evaluator itself failed to run. The reason says why. | ![The run drawer of a failed run: the verdict line reads FAILED · SQL Query Equivalence, the passed criterion, and the evaluator rows with the failed required SQL Query Equivalence and its explanation, a 7.00 clarity score and a passed PII Leak Scanner](https://mintcdn.com/langwatch/OHUpQQmmdnh4pjQ7/images/agent-testing/run-drawer-evaluator-rows-failed.png?fit=max\&auto=format\&n=OHUpQQmmdnh4pjQ7\&q=85\&s=2e0810141463d796851655e1362b16a7) A run the required **SQL Query Equivalence** failed: the verdict line names it, its row shows the explanation and the **Inputs**, and the criterion the judge passed is kept above. The run header shows a pill per evaluator: the pass rate for one with a verdict, the mean for one with a score. The verdict of a run is settled when the evaluators are in. A run whose evaluators all pass, or that has none, keeps the judge's verdict. Between the judge's verdict and the evaluator results the run reads as `PENDING_EVALUATION`, `evaluating` in the interface: the **Result** column shows `Evaluating` with a spinner and the **Evaluators** column shows `Grading`. `langwatch test-suite run ... --wait` and the batch counts treat such a run as still going, so a batch is not reported as green while a required evaluator can still fail one of its runs. The run stays pending until its evaluators are recorded. A grading job that is lost is recorded as errored evaluators once its deadline passes, so a required evaluator that never ran fails the run instead of leaving it pending. The evaluators a run is graded with are fixed when the run is scheduled. Editing the suite or the run plan while a batch executes changes the runs scheduled after the edit, never the ones already queued. The results reach the CLI and the API as `results.evaluations` on `langwatch simulation-run get --format json` and `GET /api/simulation-runs/{scenarioRunId}`, one entry per evaluator with `status`, `required`, `passed`, `score`, `label` and `details`. While the run is going, the drawer shows `The conversation is running…` and then `The judge is reading the conversation…`, and the verdict appears in place when it is in. ## What a status means The **Result** column of the table shows `Passed`, `Failed`, `Running` or `Evaluating`. The **Status** chip of the drawer is more precise: | Status | Meaning | What to do | | - | - | - | | `completed` | The judge delivered a verdict and every evaluator is in. The verdict itself is `PASSED` or `FAILED`, and a failed required evaluator is what makes it `FAILED` even when the judge passed the run. | On a fail, read the reasoning in the **RESULTS** block and the evaluator rows, then the transcript. | | `running`, `queued`, `pending` | The conversation, or the verdict, is not done. | Wait, or **Stop**. | | `evaluating` | The conversation is over and the judge has decided, but the evaluators the suite and the plan attach have not been recorded yet. A required one can still fail the run. The table shows `Evaluating` in the **Result** column and `Grading` in the **Evaluators** column. | Wait. The API and the CLI report this run as `PENDING_EVALUATION` and do not count it as finished. | | `failed` with an error in the drawer | The target did not answer: the URL is wrong, the agent is down, the authentication was refused, or a local tunnel ended. The drawer shows the error the target returned. | Fix the agent, then **Rerun this scenario**. | | `cancelled` | Stopped with **Stop** or **Stop all**. | **Rerun this scenario** when needed. | | `stalled` | The run stopped reporting before a verdict. | **Rerun this scenario**. A stalled run counts as not passed. | ![The conversation drawer of a run whose target could not be reached, with the error](https://mintcdn.com/langwatch/vSvONwxpUQtFi1t0/images/agent-testing/run-drawer-unreachable.png?fit=max\&auto=format\&n=vSvONwxpUQtFi1t0\&q=85\&s=3c6aa4d8aa770d0060fac7531a251763) A run whose setup hook failed shows **Not scored**: no turn ran, and the run does not count towards the pass rate. The drawer shows the setup error. See [Setup and teardown hooks](https://langwatch.ai/docs/agent-testing/setup-and-teardown-hooks.md#what-happens-when-a-hook-fails). A criterion the judge could not decide is inconclusive. A criterion about an internal action goes inconclusive when the agent's traces did not arrive; see [Linking your traces](https://langwatch.ai/docs/agent-testing/linking-your-traces.md#when-traces-do-not-arrive). Also check: [Run plans](https://langwatch.ai/docs/agent-testing/run-plans.md), [Compare agents](https://langwatch.ai/docs/agent-testing/compare-agents.md), [Run from CI](https://langwatch.ai/docs/agent-testing/run-from-ci.md) for the batch API behind the same runs. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.