Run a configuration under a name

Run a configuration under a name. The name identifies the run plan: send a name already in use and that plan’s configuration is replaced with this one, send a new name and the plan is created, send no name and one is derived from what the run covers and what it runs against.

POST /api/v1/run-plans/run

Python

Python

import requests

url = "https://app.langwatch.ai/api/v1/run-plans/run"

payload = { "config": {
        "scope": { "mode": "all" },
        "targets": [
            {
                "referenceId": "<string>",
                "scenarioMappings": {},
                "runParameters": {},
                "runSecretParameterNames": ["<string>"]
            }
        ],
        "repeatCount": 3,
        "simulatorModel": "<string>",
        "judgeModel": "<string>",
        "scenarioIds": ["<string>"],
        "evaluators": [
            {
                "id": "<string>",
                "evaluatorId": "<string>",
                "required": True,
                "mappings": {}
            }
        ]
    } }
headers = {
    "X-Auth-Token": "<api-key>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

200

{
  "scheduled": true,
  "batchRunId": "<string>",
  "setId": "<string>",
  "jobCount": 123,
  "skippedArchived": {
    "scenarios": [
      "<string>"
    ],
    "targets": [
      "<string>"
    ]
  },
  "items": [
    {
      "scenarioRunId": "<string>",
      "scenarioId": "<string>",
      "target": {
        "type": "prompt",
        "referenceId": "<string>",
        "scenarioMappings": {},
        "runParameters": {},
        "runSecretParameterNames": [
          "<string>"
        ]
      },
      "name": "<string>"
    }
  ],
  "runPlanId": "<string>",
  "planName": "<string>",
  "created": true,
  "platformUrl": "<string>"
}

Authorizations

X-Auth-Token · string · header · required

Project API key for sending traces and accessing project-scoped resources. Format: sk-lw-... (no underscore). Obtain one by creating a project via the Admin API or the LangWatch UI.

Body

application/json

config · object · required

What this run covers and what it runs against. Written onto the run plan the name resolves.

Show child attributes

name · string

The run plan this run joins or creates. A run plan is identified by its name, so sending the same name again replaces that plan's configuration with this one. Leave it out and the name is derived from what the run covers and what it runs against. Up to 200 characters.

Required string length: 1 - 200

idempotencyKey · string

Repeat the same key to make a retry join the batch the first call started instead of running everything again. Send the same request with it: the same key over a different configuration starts its own batch. Defaults to a new key per call.

parameters · object

Constant values applied to every scenario in the run, e.g. a fixture id or a tenant. A value supplied here overrides the scenario's own default for that name, and a target that names the same parameter in its runParameters overrides it for that target.

Show child attributes

note · string

One short line describing why this batch was run, e.g. a commit hash or what you changed. It is stored on every run of the batch and shown beside the run in the platform. Up to 200 characters.

Maximum string length: 200

Response

200 - application/json

Success

scheduled · boolean · required

True once the runs are queued.

batchRunId · string · required

The id of this batch. Every run started here carries it.

setId · string · required

The result set the batch is filed under in the platform.

jobCount · number · required

How many runs were queued.

skippedArchived · object · required

What the run left out, and why.

Show child attributes

items · object[] · required

Every run this call queued.

Show child attributes

runPlanId · string · required

The run plan this run was filed under.

planName · string · required

The name that plan answers to.

created · boolean · required

True when this run created the plan, false when it joined a plan already there.

platformUrl · string<uri> · required

Where to watch this run in the LangWatch platform.