Insights across your traces

Ask Langy what changed and it analyzes patterns across thousands of your traces, compares this week to previous weeks, and can set up alerts and automations for what matters.

Langy reads across your project’s whole history when you ask what changed, instead of a single trace. You ask it the same way you ask anything else, in the Langy panel.

Also check: Ask Langy (the panel you ask from), Scenario tests and evals (what an investigation can produce).

Ask what changed

Point Langy at a window and it reads across your traces to find what moved, instead of making you page through them:

• “Did my agent get worse this week?”
• “What changed in my errors since the last deploy?”
• “Where is my cost going, and how does it compare to last week?”
• “Why is p50 latency 12s on the checkout agent?”

Langy runs the langwatch CLI (trace search, analytics query, and more) over the origin you named, compares this week to previous weeks, and comes back with the cost, latency, or quality change that stands out and the traces behind it. From there you can ask it to write a test or open a PR on the same finding.

Ask it to watch for you

When something is worth watching past a single question, ask Langy to set up an alert or an automation and it creates the monitor or trigger for you:

• “Alert me if p95 latency on the checkout agent goes above 8s.”
• “Create a trigger that flags any trace where the answer-correctness eval fails.”
• “Watch my daily token spend and tell me if it jumps.”

Langy sets these up with the langwatch CLI, so the alert or automation lives in your project like any other, not inside the chat. You change or remove it the same way you would one you made yourself:

• A latency, cost or volume alert is a graph alert: it needs a custom graph of the metric (langwatch graph create, or one you already have) and an alert on one of that graph’s series (langwatch trigger create --custom-graph-id --graph-alert). For “p95 latency on the checkout agent above 8s”, that is a graph of performance.completion_time at p95 filtered to the checkout agent, and an alert like this:

  langwatch trigger create "Checkout p95 latency" \
    --action SEND_SLACK_MESSAGE --slack-connection <connectionId> --slack-channel C0123456789 \
    --custom-graph-id <graphId> --alert-type WARNING \
    --graph-alert '{"seriesName":"0/performance.completion_time/p95","operator":"gt","threshold":8000,"timePeriod":15}'
  

  The threshold is in milliseconds, and timePeriod is the window in minutes. The Slack connection is one listed under Settings → Integrations → Slack; a webhook connection needs no --slack-channel.

• A per-trace automation, such as flagging every trace where an evaluation fails, is langwatch trigger create --filters, keyed by the id of the monitor that runs the evaluation. If no monitor runs it yet, Langy adds one with langwatch monitor create.

The Triggers API has every option.

These are things you ask Langy to do, on the window and metric you name. Langy analyzes your traces and sets up monitors when you ask; it does not watch and act on its own.