Athina supports 2 types of JSON evals:

  1. JSON Schema: This checks the structure of the JSON object to make sure it has the required fields, and that the types of the fields match the expected structure.
  2. JSON Validation:: This checks the LLM-generated JSON object against a ground-truth object to see if they match.

​
JSON Schema

The JSON Schema eval is used to check the structure of the JSON object to make sure it has the required fields, and that the types of the fields match the expected structure.

The JSON Schema must be defined in JSON Schema format. In the UI, you can also import a sample JSON object and generate a JSON Schema from it.

To try this eval please refer to the examples .

​
JSON Validation

The JSON Validation eval is used to check the LLM-generated JSON object against a ground-truth object.

You may specify validation rules to indicate which fields should be compared and how they should be compared.

The field to be compared must be specified in JSON Path format (e.g., $.field1.field2). See the JSON Path documentation to learn more.

To try this eval please refer to the examples .

GuardrailsSummarization QA