There are 2 ways to use Athina Evals.

  1. Configure automatic evals in the dashboard: These will run automatically on your logged inferences, and you can view the results in the dashboard.
  2. Run evals programmatically using the Python SDK: This is useful for running evals on your own datasets to iterate rapidly during development.

​
Run Evals Programmatically

To get started with Athina Evals:

1. Install the athina package

pip install athina

2. Set your API keys

If you are using the python SDK, then can set the API keys like this:

from athina.keys import AthinaApiKey, OpenAiApiKey

OpenAiApiKey.set_key(os.getenv('OPENAI_API_KEY'))
AthinaApiKey.set_key(os.getenv('ATHINA_API_KEY'))

If you are using the CLI, then run athina init, and enter the API keys when prompted.

3. Load your dataset like this:

You can also load data using a Python Dictionary

from athina.loaders import RagLoader

dataset = RagLoader().load_json(json_filepath)

4. Now you can run evals like this.

from athina.evals import DoesResponseAnswerQuery

DoesResponseAnswerQuery().run_batch(data=dataset)

For more details, see this guide on running evals.

OverviewAutomatic Evals