> ## Documentation Index > Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details. # Run a test suite > Run every scenario filed in the test suite against the targets sent with the request. The run is filed under a run plan named after the suite and its targets unless a name is sent. A request that names no target answers 422 suite_targets_required. ## OpenAPI ````yaml POST /api/v1/test-suites/{id}/run openapi: 3.1.0 info: title: LangWatch API description: LangWatch openapi spec version: 1.0.0 servers: - url: https://app.langwatch.ai security: - project_api_key: [] paths: /api/v1/test-suites/{id}/run: post: tags: - Test Suites summary: Run a test suite description: >- Run every scenario filed in the test suite against the targets sent with the request. The run is filed under a run plan named after the suite and its targets unless a name is sent. A request that names no target answers 422 suite_targets_required. operationId: runTestSuite parameters: - in: path name: id schema: type: string minLength: 1 required: true description: The test suite id. requestBody: required: true content: application/json: schema: type: object properties: targets: type: array items: type: object properties: type: type: string enum: - prompt - http - code - workflow - connected - voice referenceId: type: string minLength: 1 scenarioMappings: type: object propertyNames: type: string additionalProperties: oneOf: - type: object properties: type: type: string const: source sourceId: type: string minLength: 1 path: type: array items: type: string required: - type - sourceId - path additionalProperties: false - type: object properties: type: type: string const: value value: type: string required: - type - value additionalProperties: false runParameters: type: object propertyNames: type: string additionalProperties: anyOf: - type: string maxLength: 4096 - type: number - type: boolean runSecretParameterNames: maxItems: 50 type: array items: type: string maxLength: 64 required: - type - referenceId additionalProperties: false description: >- The prompts, agents or workflows the suite runs against. A test suite stores none of its own, so a run states them. Every target runs every scenario, so naming more than one compares them in the same run. name: description: >- The run plan this run joins or creates. Leave it out and the name is derived from the suite name and the targets. type: string minLength: 1 maxLength: 200 repeatCount: description: >- How many times each scenario and target pairing runs. Between 1 and 5; defaults to 1. type: integer minimum: 1 maximum: 5 simulatorModel: description: >- The model that plays the user for every scenario in this run. Leave it out for the scenario or project default. anyOf: - type: string maxLength: 200 pattern: ^[a-zA-Z0-9_-]+(?:\/[^\s/]+)+$ - type: 'null' judgeModel: description: >- The model that judges every scenario in this run. Leave it out for the scenario or project default. anyOf: - type: string maxLength: 200 pattern: ^[a-zA-Z0-9_-]+(?:\/[^\s/]+)+$ - type: 'null' idempotencyKey: description: >- Repeat the same key to make a retry join the batch the first call started instead of running everything again. Send the same request with it: the same key over a different configuration starts its own batch. Defaults to a new key per call. type: string parameters: description: >- Constant values applied to every scenario in the run, e.g. a fixture id or a tenant. A value supplied here overrides the scenario's own default for that name, and a target that names the same parameter in its runParameters overrides it for that target. type: object propertyNames: type: string additionalProperties: anyOf: - type: string maxLength: 4096 - type: number - type: boolean note: description: >- One short line describing why this batch was run, e.g. a commit hash or what you changed. It is stored on every run of the batch and shown beside the run in the platform. Up to 200 characters. type: string maxLength: 200 required: - targets responses: '200': description: Success content: application/json: schema: type: object properties: scheduled: type: boolean description: True once the runs are queued. batchRunId: type: string description: The id of this batch. Every run started here carries it. setId: type: string description: The result set the batch is filed under in the platform. jobCount: type: number description: How many runs were queued. skippedArchived: type: object properties: scenarios: type: array items: type: string description: Scenarios left out because they are archived. targets: type: array items: type: string description: Targets left out because they are archived. required: - scenarios - targets additionalProperties: false description: What the run left out, and why. items: type: array items: type: object properties: scenarioRunId: type: string description: The id of this single run. scenarioId: type: string description: The scenario that was run. target: type: object properties: type: type: string enum: - prompt - http - code - workflow - connected - voice referenceId: type: string minLength: 1 scenarioMappings: type: object propertyNames: type: string additionalProperties: oneOf: - type: object properties: type: type: string const: source sourceId: type: string minLength: 1 path: type: array items: type: string required: - type - sourceId - path additionalProperties: false - type: object properties: type: type: string const: value value: type: string required: - type - value additionalProperties: false runParameters: type: object propertyNames: type: string additionalProperties: anyOf: - type: string maxLength: 4096 - type: number - type: boolean runSecretParameterNames: maxItems: 50 type: array items: type: string maxLength: 64 required: - type - referenceId additionalProperties: false description: What it was run against. name: description: The scenario name, when known. type: - string - 'null' required: - scenarioRunId - scenarioId - target - name additionalProperties: false description: Every run this call queued. runPlanId: type: string description: The run plan this run was filed under. planName: type: string description: The name that plan answers to. created: type: boolean description: >- True when this run created the plan, false when it joined a plan already there. platformUrl: type: string format: uri description: Where to watch this run in the LangWatch platform. required: - scheduled - batchRunId - setId - jobCount - skippedArchived - items - runPlanId - planName - created - platformUrl additionalProperties: false '404': description: Not Found content: application/json: schema: type: object properties: error: type: string message: type: string required: - error additionalProperties: false security: - project_api_key: [] components: securitySchemes: project_api_key: type: apiKey in: header name: X-Auth-Token description: >- Project API key for sending traces and accessing project-scoped resources. Format: sk-lw-... (no underscore). Obtain one by creating a project via the Admin API or the LangWatch UI. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.