> ## Documentation Index > Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details. # Ask Langy > Ask Langy in plain language about your production traces and evals; it reads the data with the langwatch CLI, writes a Scenario test, and can open a pull request. Type the question into the Langy panel in plain language; there is no query syntax to learn. The sections below group example questions by what you are trying to find out, from a regression, to a cost spike, to a fix you want turned into a pull request. > [!NOTE] > > **Also check:** [Scenario tests and evals](https://langwatch.ai/docs/langy/scenario-tests-and-evals.md) (what Langy produces), > [Pull requests](https://langwatch.ai/docs/langy/pull-requests.md) (how it changes code), [Insights across your > traces](https://langwatch.ai/docs/langy/insights.md) (what it finds when you ask what changed). ![The Langy chat panel ready for a question](https://mintcdn.com/langwatch/6W_w4oAKDAm7lUJ_/images/langy/overview/chat-panel-empty.png?fit=max\&auto=format\&n=6W_w4oAKDAm7lUJ_\&q=85\&s=147604d2e9b627932b73f33a5b0cf838) The Langy panel, ready for a question, with suggested prompts and a 'Choose what Langy can see' context row on the composer. ## What you can ask Group your questions by intent. Each maps to the `langwatch` CLI commands Langy runs behind it. ### Investigate a regression * "Did my agent get worse this week?" * "Find failing traces from the last 24 hours." * "Why is p50 latency 12s on the checkout agent?" Langy runs `langwatch trace search` and `langwatch analytics query` over the relevant origin, compares the window you named, and comes back with the traces and the metric that moved. ### Investigate cost * "Where is my cost going?" * "What changed in my token spend after the last deploy?" * "Can I move this feature to a cheaper model without losing quality?" For a cheaper-model migration, Langy pulls the traces for that feature, then writes an [evaluation or a Scenario test](https://langwatch.ai/docs/langy/scenario-tests-and-evals.md) that measures the current behavior, so you can compare the cheaper model against it. ### Optimize a prompt * "Optimize the support classifier prompt." * "Can you improve this prompt's pass rate?" * "Try a better version of this prompt and compare." Langy scores the current prompt, duplicates it, rewrites the duplicate, and reruns both against your evaluator. See [Prompt Optimization](https://langwatch.ai/docs/langy/prompt-optimization.md). ### Set up testing and evals * "Set up an evaluator for answer correctness on the support agent." * "Write a Scenario test that probes this agent like an annoyed user." * "Compare two runs and tell me which prompt is better." Langy writes the test or evaluator with `langwatch scenario create`, `langwatch evaluator create`, or `langwatch experiment run`, and can run it for you. See [Scenario tests and evals](https://langwatch.ai/docs/langy/scenario-tests-and-evals.md). ### Investigate an issue and open a PR * "Investigate this failing trace and open a PR with a fix." * "This agent ignores the system prompt under pressure; write a failing test and fix it." Langy reads the trace, writes a failing Scenario test that captures the problem, drafts a fix, and opens a [bot-authored pull request](https://langwatch.ai/docs/langy/pull-requests.md). A human reviews and merges; Langy never merges. ## What happens when you ask **1. Langy reads your data** It shells the `langwatch` CLI (`trace search`, `trace get`, `analytics query`, and more) to read the traces and evals for the question you asked. Traces are scoped by origin, so it excludes its own turns by default. **2. It may write a test or evaluation** When a question needs a check, Langy writes a Scenario simulation test or an evaluation that captures the behavior. See [Scenario tests and evals](https://langwatch.ai/docs/langy/scenario-tests-and-evals.md). **3. It may open a pull request** For a code change, Langy opens a bot-authored PR on the repo you installed the GitHub App on, and runs the Scenario tests it wrote in your CI. See [Pull requests](https://langwatch.ai/docs/langy/pull-requests.md). Each tool call streams into the turn as an activity card, so you see the CLI command it ran and its typed result. ![Langy running tool work](https://mintcdn.com/langwatch/6W_w4oAKDAm7lUJ_/images/langy/chat/chat-toolwork.png?fit=max\&auto=format\&n=6W_w4oAKDAm7lUJ_\&q=85\&s=0a3e2e8e5430494ba2f54d3fb172a1ff) The Langy panel mid-turn, running real tool work. Each CLI call appears as an activity in the conversation. When Langy runs a Scenario simulation, the result appears as a run card in the chat with its outcome, duration, and cost. ![Simulation run card in chat](https://mintcdn.com/langwatch/6W_w4oAKDAm7lUJ_/images/langy/chat/simulation-run-card.png?fit=max\&auto=format\&n=6W_w4oAKDAm7lUJ_\&q=85\&s=dc7f275122d37cf1a3386aef7d2a445d) A live simulation-run card inside chat: outcome SUCCESS with duration and cost, and a link to open the run in Agent Testing > Results. ## Ask about what you are looking at The panel stays docked while you move through the app, so you can open a trace and ask Langy about that trace. Paste the trace ID or add it through the context chip, and Langy reads the same spans you see in the waterfall. ![A trace drawer with its span waterfall and a Langy conversation about that trace](https://mintcdn.com/langwatch/6W_w4oAKDAm7lUJ_/images/langy/panel/drawer-companion.png?fit=max\&auto=format\&n=6W_w4oAKDAm7lUJ_\&q=85\&s=2c0b37035845b8cdba33228320a8d019) A trace's span waterfall with Langy alongside, walking through what happened in that trace and why an evaluator errored. ## Choose what Langy can see Context is explicit. The `# Context` chip on the composer lets you choose what Langy can see, and the chips shown are exactly the context the model receives. ## Stop a turn You can stop a turn at any point. Stopping ends the turn: Langy stops the work in progress and keeps the partial output it already produced. Start a new turn to continue from there. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.