Ragas Faithfulness

This evaluator assesses the extent to which the generated answer is consistent with the provided context. Higher scores indicate better faithfulness to the context, useful for detecting hallucinations.

POST /legacy/ragas_faithfulness/evaluate

Try it

Python

Offline Evaluation

import langwatch

df = langwatch.dataset.get_dataset("dataset-id").to_pandas()

evaluation = langwatch.evaluation.init("my-incredible-evaluation")

for index, row in evaluation.loop(df.iterrows()):
    # your execution code here       
    evaluation.run(
        "legacy/ragas_faithfulness",
        index=index,
        data={
            "output": output,
            "contexts": row["contexts"],
        },
        settings={}
    )

200

[
  {
    "status": "processed",
    "score": 123,
    "passed": true,
    "label": "<string>",
    "details": "<string>",
    "cost": {
      "currency": "<string>",
      "amount": 123
    },
    "raw_response": {},
    "error_type": "<string>",
    "traceback": [
      "<string>"
    ]
  }
]

Authorizations

X-Auth-Token · string · header · required

Body

application/json

data · object · required

Show child attributes

data. output · string · required

The output text to evaluate

data. contexts · string[] · required

Context information for evaluation

settings · object

Evaluator settings

Show child attributes

settings. model · string · default: openai/gpt-5

The model to use for evaluation.

settings. max_tokens · number · default: 2048

The maximum number of tokens allowed for evaluation, a too high number can be costly. Entries above this amount will be skipped.

settings. embeddings_model · string · default: openai/text-embedding-ada-002

The model to use for embeddings.

Response

200

application/json

Successful evaluation

status · enum<string> · required

Available options: processed, skipped, error

score · number

Evaluation score

passed · boolean

Whether the evaluation passed

label · string

Evaluation label

details · string

Additional details about the evaluation

cost · object

Show child attributes

cost. currency · string · required

cost. amount · number · required

raw_response · object

Raw response from the evaluator

error_type · string

Type of error if status is 'error'

traceback · string[]

Error traceback if status is 'error'