> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# Annotations

> Review traces by hand with a comment, scores and a corrected output, and read every review in the annotations inbox.

An annotation is a human review attached to a trace, or to one part of it: a comment, one or more values per score type, and, for a suggestion, the output the model should have given. Annotations feed the review inbox, the trace list, and the datasets you build from reviewed traces.

## Annotate a trace

Open a trace from the [Trace Explorer](https://app.langwatch.ai/@project/traces). In the conversation view, hover over a reply. **Translate**, **Annotate** and **Suggest** appear next to it:

| Button | Opens |
| - | - |
| **Annotate** | **Add annotation**: a **Comment** and the **Scores** chips. |
| **Suggest** | **Suggest correction**: an **Expected output** field prefilled with the reply, a diff against the original, the comment and the scores. |

![A reply in the conversation view with the Translate, Annotate and Suggest buttons, and a saved annotation with its comment and Accuracy score beside it](https://mintcdn.com/langwatch/b_7qlc4BhHkq6ZDw/images/annotations/trace-annotate-cluster.png?fit=max\&auto=format\&n=b_7qlc4BhHkq6ZDw\&q=85\&s=a1e8dad4564115dcfe871a2edbf365b1)

The composer names its target under the title, for example **Output**. Each score chip opens its question, its options and a **Reason (optional)** field. Click **OK** to set the score, or **Clear** to remove it, then **Save**. See [Annotation scores](https://langwatch.ai/docs/annotations/scores.md) to define the score types.

![The Add annotation composer targeting Output, with a comment typed, the Accuracy and Issues score chips, and the Accuracy chip open on its question, the options Correct, Partially correct and Wrong, the Reason (optional) field, Clear and OK](https://mintcdn.com/langwatch/b_7qlc4BhHkq6ZDw/images/annotations/annotation-form.png?fit=max\&auto=format\&n=b_7qlc4BhHkq6ZDw\&q=85\&s=5cf2a3c2d6a85a5a5ae5b7712912cfe8)

**Suggest correction** prefills **Expected output** with the reply. Edit the text; the **Diff** section counts the added and removed characters and highlights the change. **Reset** restores the original reply.

![The Suggest correction composer with one sentence added to the Expected output and the Diff section showing +63 -0 with the added sentence highlighted](https://mintcdn.com/langwatch/b_7qlc4BhHkq6ZDw/images/annotations/annotation-suggest-form.png?fit=max\&auto=format\&n=b_7qlc4BhHkq6ZDw\&q=85\&s=9e3f2f52f38a77d5fa9398ac82d493c4)

Every message, span, attribute row and panel in the trace drawer has a **Comment** button too, so an annotation can point at exactly the part that is wrong. The **Annotations** chip in the drawer header counts the annotations on the trace.

Saving a suggestion also records the corrected output on the trace, so a dataset built from that trace gets the corrected text. See [Edit a trace](https://langwatch.ai/docs/annotations/queues.md#edit-a-trace).

You can edit or delete your own annotations from the trace. Writing annotations needs the `annotations:manage` permission; reading them needs `annotations:view`.

## The annotations inbox

[Annotations](https://app.langwatch.ai/@project/annotations) lists the traces queued for review. The sidebar has **Inbox** (everything queued for you or for a queue you are in), **You** (traces sent to you directly), **All** (every annotation in the project) and one entry per queue under **My queues**, each with its pending count.

![The annotations inbox: the sidebar with Inbox, You, All and the Support review queue, the Queues, Status: Pending, Columns and All time filters, Export, and the table with the reviewer avatars, Date queued, Input, Output, Scores, Comments and Suggestions](https://mintcdn.com/langwatch/b_7qlc4BhHkq6ZDw/images/annotations/annotations-inbox.png?fit=max\&auto=format\&n=b_7qlc4BhHkq6ZDw\&q=85\&s=d3b24fc88789dabe53c0c423a2fb58bb)

The table shows **People** (the reviewer avatars, in a column without a header), **Date queued**, **Input**, **Output**, **Scores**, **Comments** and **Suggestions**. **Columns** adds one column per score type or hides the ones you do not need; the choice is kept per project in your browser.

![The Columns popover with the 7 shown count, a search box, the Standard section with every column ticked and the Score types section with Issues and Accuracy](https://mintcdn.com/langwatch/b_7qlc4BhHkq6ZDw/images/annotations/annotations-columns-menu.png?fit=max\&auto=format\&n=b_7qlc4BhHkq6ZDw\&q=85\&s=2a08edc3c0cd84df6465ee4236cbc38f)

**Status** filters **Pending**, **Completed** or **All**; the inbox opens on **Pending**. **Queues** filters by queue. **Export** downloads the visible rows as CSV with the trace id, the status, every visible column and the annotators.

A pending row opens the review page for that item. A completed row opens the trace. The row menu has **View trace**, **Add to dataset** and, on a queue item, **Remove from queue**. Select several rows to add them to a dataset or to a queue at once.

## Where annotations show up

* The **Annotations** column of the Trace Explorer, and the **annotated** and **unannotated** filter under **Quality**.
* The **annotations** source when you add traces to a dataset: the comment, the scores and their reasons, the expected output, or all of them as one text. See [Datasets](https://langwatch.ai/docs/datasets/overview.md).
* The annotations inbox and the CSV export.

**Also check:** [Annotation queues](https://langwatch.ai/docs/annotations/queues.md) to organize a review session across a team, and [Annotation scores](https://langwatch.ai/docs/annotations/scores.md) to define what reviewers score.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.