Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.

Skip to main content

Get started with LangWatch Skills in seconds: Set up evals, scenario tests, and tracing just by asking your AI coding assistant.

LangWatch home page

llms.txt
Sign In
langwatch/langwatch
langwatch/langwatch

Platform
Observability
AI Gateway
AI Governance
Coding Agents
Langy
Self Hosting
API Reference
Get Support

Get Started

Introduction

Traces

Overview
POST
Search traces
GET
Get trace details
GET
Get Transcript
PATCH
Update trace metadata
POST
Create public path for single trace
POST
Delete an existing public path for a trace
Projection DSL & field catalog

Datasets

Overview
GET
List all non-archived datasets for the project
POST
Create a new dataset
POST
Start a direct browser→S3 dataset upload
POST
Evaluate a dataset
POST
Create a new dataset from an uploaded file
GET
Get a dataset by its slug or id
GET
List records for a dataset
PATCH
Update a dataset by its slug or id
PATCH
Update or create a record in a dataset
PUT
Update Staging
POST
Add dataset entries programmatically using the LangWatch API to build evaluation sets for LLM testing and agent validation.
POST
Create records in a dataset in batch
POST
Upload a file
POST
Create Finalize
POST
Retry normalization of a failed or stuck dataset
DEL
Archive a dataset
DEL
Delete records from a dataset by IDs
DEL
Delete Direct Upload

Evaluators

Overview
GET
Get all evaluators for a project
POST
Create a new evaluator
GET
Get a specific evaluator by ID or slug
PUT
Update an existing evaluator
DEL
Archive an evaluator

Built-in Evaluators

Overview

Evaluations

Overview
GET
List the built-in evaluators
POST
Run an evaluator
POST
Run a namespaced evaluator
POST
Run an evaluator as a guardrail
POST
Report batch evaluation results

Experiments

Overview
POST
Create an experiment
GET
List experiments for the project
POST
Run an experiment
GET
List runs of an experiment
GET
Poll a run
GET
Read run results
POST
Report DSPy optimizer steps
POST
Create an experiment and its setup
GET
Read one experiment
GET
List an experiment's versions
GET
Read an experiment's setup
PUT
Save an experiment's setup
POST
Restore an experiment version

Monitors

Overview
GET
List Monitors
POST
Create a new online evaluation monitor
GET
Get a monitor by its ID
PATCH
Update a monitor
POST
Enable or disable a monitor
DEL
Delete a monitor

Prompts

Overview
GET
Get prompts
GET
List Tags
POST
Create prompt
POST
Create Tags
GET
Get prompt
GET
Get prompt versions
PUT
Update prompt
PUT
Assign a tag
PUT
Rename a prompt tag definition
POST
Sync/upsert a prompt with local content
POST
Restore a prompt to a previous version
DEL
Delete prompt
DEL
Delete Tags

Annotations

Overview
GET
Get annotations
GET
Get single annotation
GET
Get annotations for a trace
PATCH
Patch single annotation
POST
Create annotation for single trace
DEL
Delete single annotation

Scenarios

Overview
GET
Get all scenarios for a project
POST
Create a new scenario
GET
Get a specific scenario by ID
GET
List a scenario's versions
GET
Read one scenario version
PUT
Update an existing scenario
DEL
Archive a scenario

Scenario Events

Overview
POST
Create a new scenario event
POST
Create Browser Tab
DEL
Delete all events

Simulation Runs

Overview
GET
List Simulation Runs
GET
List batch summaries for a scenario set
GET
Get a single simulation run by its ID
GET
Get Batches

Run Plans

Overview
GET
List the project's run plans
POST
Run a configuration under a name
GET
Read one run plan
POST
Run a plan again
DEL
Archive a run plan

Test Suites

Overview
GET
List the project's test suites
POST
Create a test suite
GET
Read one test suite
PATCH
Rename a test suite
POST
Run a test suite
DEL
Archive a test suite

Suites

Overview
GET
List all non-archived suites
POST
Create a new suite
GET
Get a suite
PATCH
Update a suite
POST
Duplicate a suite
POST
Trigger a suite run
DEL
Archive a suite

Agents

Overview
GET
List all non-archived agents for the project
POST
Create a new agent
GET
Get an agent by its id
PATCH
Update an agent by its id
DEL
Archive an agent

Coding Agents

Overview
GET
Get pull request coding agent usage
GET
List coding agent session events

Triggers

Overview
GET
List all active triggers
POST
Create a Slack alert trigger
POST
Create a new trigger
GET
Get a trigger by its ID
PATCH
Update a trigger
DEL
Delete a trigger

Events

Overview
POST
Record a user event
POST
Track an event (legacy path)

Workflows

Overview
GET
List all non-archived workflows for the project
GET
Get a workflow by its ID
PATCH
Update a workflow's metadata
POST
Run a workflow version (legacy path)
POST
Create Evaluate
POST
Run a specific workflow version
POST
Run a workflow
DEL
Archive a workflow

Dashboards

Overview
GET
List Dashboards
POST
Create a new dashboard
GET
Get a dashboard by its id, including its graphs
PUT
Reorder dashboards
PATCH
Rename a dashboard
DEL
Delete a dashboard and its graphs

Graphs

Overview
GET
List Graphs
POST
Create a custom graph on a dashboard
GET
Get a custom graph by its ID
PATCH
Update Graphs
DEL
Delete a custom graph

Analytics

Overview
POST
Query analytics timeseries (legacy path)
POST
Create Timeseries

Secrets

Overview
GET
List all secrets for the project
POST
Create a new project secret
GET
Get a secret by its ID
PUT
Update a secret's value
DEL
Delete a secret

Agent Cache

Overview
GET
Read a cache entry
PUT
Store a cache entry
POST
Claim a cache entry
DEL
Remove a cache entry

Model Providers

Overview
GET
List Model Providers
PUT
Create or update a model provider

Projects

Overview
GET
List projects
POST
Create project
GET
Get project
GET
Get project API key
PATCH
Update project
POST
Regenerate project API key
DEL
Archive project

Teams

Overview
GET
List teams
POST
Create team
GET
Get team
GET
List team members
GET
List team projects
PATCH
Update team
POST
Add team member
DEL
Archive team
DEL
Remove team member

Groups

Overview
GET
List groups
POST
Create group
GET
Get group
GET
List group bindings
GET
List group members
PATCH
Rename group
POST
Add group binding
POST
Add group member
DEL
Delete group
DEL
Remove group binding
DEL
Remove group member

Organization

Overview
GET
Get organization
PATCH
Update organization

Members

Overview
GET
List members
GET
Get member
GET
Get member access
PATCH
Update member
DEL
Remove member

Invites

Overview
GET
List invites
POST
Create invites
DEL
Revoke invite

Roles

Overview
GET
List permissions
GET
List roles
POST
Create role
GET
Get role
PATCH
Update role
DEL
Delete role

Role Bindings

Overview
GET
List role bindings
POST
Create role binding
PATCH
Update role binding
DEL
Delete role binding

SCIM Tokens

Overview
GET
List SCIM tokens
POST
Create SCIM token
DEL
Revoke SCIM token

SCIM

Overview
GET
Get the SCIM service provider configuration
GET
List the SCIM resource types
GET
List the SCIM resource schemas
GET
List provisioned users
POST
Provision a user
GET
Get a provisioned user
PUT
Replace a provisioned user
PATCH
Update a provisioned user
DEL
Deprovision a user
GET
List provisioned groups
POST
Provision a group
GET
Get a provisioned group
PUT
Replace a provisioned group
PATCH
Update a provisioned group
DEL
Deprovision a group

Organizations (Self-Hosted)

Overview
GET
List organizations
POST
Create an organization
GET
Get an organization

API Keys

Overview
GET
List API keys
POST
Create API key
GET
Get an API key
PATCH
Update an API key
DEL
Revoke API key

Gateway: Virtual Keys

Overview
GET
List virtual keys
POST
Create virtual key
GET
Get virtual key
GET
Read a virtual key's spend
PATCH
Update virtual key
POST
Disable virtual key
POST
Enable virtual key
POST
Revoke virtual key
POST
Rotate virtual key secret

Gateway: Budgets

Overview
GET
List budgets applicable to the project
POST
Create budget
GET
Get budget
PATCH
Update budget
POST
Reset budget period
DEL
Archive budget

Gateway: Cache Rules

Overview
GET
List cache-control rules
POST
Create a cache rule
GET
Get a cache rule
PATCH
Update a cache rule
DEL
Archive a cache rule

Gateway: Spend

Overview
GET
List spend events
GET
List spend summaries
POST
Replay spend events to an endpoint
GET
Read one end user's spend

Webhooks

Overview
GET
List webhook endpoints
GET
List subscribable event types
GET
List emitted events
POST
Create a webhook endpoint
GET
Get a webhook endpoint
GET
List an endpoint's delivery attempts
GET
Read an endpoint's delivery health
GET
Get one emitted event
PATCH
Update a webhook endpoint
POST
Roll an endpoint's signing secret
POST
Send a test event to an endpoint
DEL
Archive a webhook endpoint

Intro

Scenarios

Overview

Manage test scenarios for agent simulations. Create, update, and organize scenarios that define test cases for your AI agents.

Intro

Manage test scenarios for agent simulations. Create, update, and organize scenarios that define test cases for your AI agents.

Last modified on May 19, 2026

Was this page helpful?

Delete single annotation

Get all scenarios for a project

Powered byThis documentation is built and hosted on Mintlify, a developer documentation platform