Query Resolution

This evaluator checks if all the user queries in the conversation were resolved. Useful to detect when the bot doesn’t know how to answer or can’t help the user.

POST /langevals/query_resolution/evaluate

Python

Experiment

import langwatch

df = langwatch.datasets.get_dataset("dataset-id").to_pandas()

experiment = langwatch.experiment.init("my-experiment")

for index, row in experiment.loop(df.iterrows()):
    # your execution code here
    experiment.evaluate(
        "langevals/query_resolution",
        index=index,
        data={
            "conversation": row["conversation"],
        },
        settings={}
    )

200

{
  "status": "processed",
  "score": 123,
  "passed": true,
  "label": "<string>",
  "details": "<string>",
  "cost": {
    "currency": "<string>",
    "amount": 123
  }
}

Authorizations

X-Auth-Token · string · header · required

API key for authentication

Body

application/json

data · object · required

Show child attributes

trace_id · string

Optional trace ID to associate this evaluation with a trace

settings · object

Show child attributes

Response

200

application/json

Successful evaluation

status · enum<string>

Available options: processed, skipped, error

score · number

Numeric score from the evaluation

passed · boolean

Whether the evaluation passed

label · string

Label assigned by the evaluation

details · string

Additional details about the evaluation

cost · object

Show child attributes