> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# Automations: build a dataset from your traces

> Create an automation that adds every new trace matching a filter to a dataset, mapped to the columns you choose.

An automation with the **Add to dataset** action appends a row for every trace that matches its filter, as the trace arrives. Use it to grow a dataset from production, for example every trace an online evaluation failed, or every trace with a given metadata value.

## Create the automation

1. Open the [Trace Explorer](https://app.langwatch.ai/@project/traces) and set the filter that selects the traces you want, for example an evaluation result or a metadata value.
2. Click **Automate** in the toolbar. The automation drawer opens with the current filter as its subject.

![LangWatch](https://mintcdn.com/langwatch/zEjkqM6j5EFLGO6F/images/add-automation-filter.png?fit=max\&auto=format\&n=zEjkqM6j5EFLGO6F\&q=85\&s=6784604d50d9f139b6ae6695f3aa44b3)

Add Automation

3. Pick **Add to dataset** as the action and choose the dataset. Its columns are mapped to trace fields for you, as described in [The column mapping](#the-column-mapping).

![LangWatch](https://mintcdn.com/langwatch/zEjkqM6j5EFLGO6F/images/add-to-dataset-automation.png?fit=max\&auto=format\&n=zEjkqM6j5EFLGO6F\&q=85\&s=66ba8b2206118a0f83304f4d041b5e97)

Add To Dataset

4. Save. From then on, every trace that matches the filter adds one row to the dataset.

## The column mapping

When you pick a dataset, LangWatch builds the mapping from the dataset's column names:

| Column name | Filled from |
| - | - |
| `input`, `question`, `user_input` | The trace input |
| `output`, `answer`, `response`, `result`, `expected_output` | The trace output |
| `contexts` | The retrieved contexts, as a list of strings |
| `total_cost` | The trace's total cost |
| `spans` | The trace's spans |
| `trace_id`, `timestamp` | The trace id and its timestamp |
| Any other name | The trace **metadata key with the same name** as the column |

So a column named `customer_tier` is filled from the trace's `customer_tier` metadata. A trace without that key leaves the cell empty.

The mapping is stored on the automation as `actionParams.datasetMapping`. Each column maps to a `{ source, key, subkey }` entry: `source` is the trace field, `key` narrows a keyed source (the metadata key for `metadata`, for example), and `subkey` narrows it further (one field of an evaluation, for example).

```json theme={null}
{
  "datasetId": "<datasetId>",
  "datasetMapping": {
    "mapping": {
      "input": { "source": "input", "key": "", "subkey": "" },
      "output": { "source": "output", "key": "", "subkey": "" },
      "customer_tier": { "source": "metadata", "key": "customer_tier", "subkey": "" }
    },
    "expansions": []
  }
}
```

A `source` is one of `trace_id`, `thread_id`, `timestamp`, `input`, `output`, `contexts`, `contexts.string_list`, `metrics.total_cost`, `metrics.first_token_ms`, `metrics.total_time_ms`, `metrics.prompt_tokens`, `metrics.completion_tokens`, `metrics.total_tokens`, `spans`, `spans.llm.input`, `spans.llm.output`, `metadata`, `evaluations`, `annotations` or `events`.

### Changing the mapping later

The automation drawer has no per-column mapping editor: it derives the mapping from the dataset's columns, and choosing the dataset again derives it again. To map a column to a different source after the automation exists, update the automation's delivery configuration with the whole mapping you want, with the CLI:

```bash theme={null}
langwatch trigger update <automationId> --action-params '{"datasetId":"<datasetId>","datasetMapping":{"mapping":{"input":{"source":"input"},"customer_tier":{"source":"metadata","key":"tier"}},"expansions":[]}}'
```

or with `PATCH /api/triggers/{id}` and the same `actionParams`. `actionParams` replaces the stored configuration as a whole, so send every column, not only the one you change. See the [Triggers API](https://langwatch.ai/docs/api-reference/triggers/overview.md).

**Also check:** [Automations](https://langwatch.ai/docs/features/automations.md) for the delivery options and the automation history, and [Datasets](https://langwatch.ai/docs/datasets/overview.md) for the full list of trace fields a column can map to.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.