Lingua Language Detection

This evaluator detects the language of the input and output text to check for example if the generated answer is in the same language as the prompt, or if it’s in a specific expected language.

POST /lingua/language_detection/evaluate

Try it

Python

Offline Evaluation

import langwatch

df = langwatch.dataset.get_dataset("dataset-id").to_pandas()

evaluation = langwatch.evaluation.init("my-incredible-evaluation")

for index, row in evaluation.loop(df.iterrows()):
    # your execution code here       
    evaluation.run(
        "lingua/language_detection",
        index=index,
        data={
            "output": output,
            "input": row["input"],
        },
        settings={}
    )

200

[
  {
    "status": "processed",
    "score": 123,
    "passed": true,
    "label": "<string>",
    "details": "<string>",
    "cost": {
      "currency": "<string>",
      "amount": 123
    },
    "raw_response": {},
    "error_type": "<string>",
    "traceback": [
      "<string>"
    ]
  }
]

Authorizations

X-Auth-Token · string · header · required

Body

application/json

data · object · required

Show child attributes

data. output · string · required

The output text to evaluate

data. input · string

The input text to evaluate

settings · object

Evaluator settings

Show child attributes

settings. check_for · string · default: input_matches_output

What should be checked

settings. expected_language · string · default: undefined

The specific language that the output is expected to be

settings. min_words · number · default: 7

Minimum number of words to check, as the language detection can be unreliable for very short texts. Inputs shorter than the minimum will be skipped.

settings. threshold · number · default: 0.25

Minimum confidence threshold for the language detection. If the confidence is lower than this, the evaluation will be skipped.

Response

200

application/json

Successful evaluation

status · enum<string> · required

Available options: processed, skipped, error

score · number

Evaluation score

passed · boolean

Whether the evaluation passed

label · string

Evaluation label

details · string

Additional details about the evaluation

cost · object

Show child attributes

cost. currency · string · required

cost. amount · number · required

raw_response · object

Raw response from the evaluator

error_type · string

Type of error if status is 'error'

traceback · string[]

Error traceback if status is 'error'