There are 2 ways to use Athina Evals.

  1. Configure automatic evals in the dashboard: These will run automatically on your logged inferences, and you can view the results in the dashboard.
  2. Run evals programmatically using the Python SDK: This is useful for running evals on your own datasets to iterate rapidly during development.

Run Evals Programmatically

To get started with Athina Evals:

1. Install the athina package

pip install athina

2. Set your API keys

If you are using the python SDK, then can set the API keys like this:

from athina.keys import AthinaApiKey, OpenAiApiKey


If you are using the CLI, then run athina init, and enter the API keys when prompted.

3. Load your dataset like this:

You can also load data using a CSV or Python Dictionary

from athina.loaders import RagLoader

dataset = RagLoader().load_json(json_filepath)

4. Now you can run evals like this.

from athina.evals import DoesResponseAnswerQuery


For more details, see this guide on running evals.