Ragas Context Relevancy

This metric gauges the relevancy of the retrieved context, calculated based on both the question and contexts. The values fall within the range of (0, 1), with higher values indicating better relevancy.

POST /legacy/ragas_context_relevancy/evaluate

Python

Experiment

import langwatch

df = langwatch.datasets.get_dataset("dataset-id").to_pandas()

experiment = langwatch.experiment.init("my-experiment")

for index, row in experiment.loop(df.iterrows()):
    # your execution code here
    experiment.evaluate(
        "legacy/ragas_context_relevancy",
        index=index,
        data={
            "output": output,
            "contexts": row["contexts"],
        },
        settings={}
    )

200

[
  {
    "status": "processed",
    "score": 123,
    "passed": true,
    "label": "<string>",
    "details": "<string>",
    "cost": {
      "currency": "<string>",
      "amount": 123
    }
  }
]

Authorizations

X-Auth-Token · string · header · required

API key for authentication

Body

application/json

output · string · required

The output/response text to evaluate

contexts · string[] · required

Array of context strings used for RAG evaluation

settings · object

Show child attributes

Response

200

application/json

Successful evaluation

status · enum<string>

Available options: processed, skipped, error

score · number

Numeric score from the evaluation

passed · boolean

Whether the evaluation passed

label · string

Label assigned by the evaluation

details · string

Additional details about the evaluation

cost · object

Show child attributes