Info
This evaluator checks if the response answer’s the query sufficiently. Required Argsquery
: The query, ideally in a question format.response
: The LLM generated response.
gpt-4
Example
- Query: Which spaceship landed on the moon first?
- Response: Neil Armstrong was the first man to set foot on the moon in 1969
Eval Result
- Result: Fail
- Explanation: The query is asking which spaceship landed on the moon first, but the response only mentions the name of the astronaut, and does not say anything about the name of the spaceship.
Run the eval on a dataset
- Load your data with the
Loader
- Run the evaluator on your dataset