There are many ways to use Athina Evals.

​
Running Evals in UI

  1. Run evals on datasets
  2. Configure online evals to run automatically: These will run automatically on your logged inferences, and you can view the results in the dashboard.
  3. Run evaluations manually from the Trace view
  4. Run evaluations in the Prompt playground
  5. Configure Automations to run evals automatically on new datasets in a project

​
Run Evals Programmatically

  1. Run evals in CI / CD pipelines
  2. Run evals as guardrails around inference
  3. Run evals using the Python SDK

Here’s a quickstart guide for running evals using the Python SDK:

1. Install the athina package

pip install athina

2. Set your API keys

If you are using the python SDK, then can set the API keys like this:

from athina.keys import AthinaApiKey, OpenAiApiKey

OpenAiApiKey.set_key(os.getenv('OPENAI_API_KEY'))
AthinaApiKey.set_key(os.getenv('ATHINA_API_KEY'))

If you are using the CLI, then run athina init, and enter the API keys when prompted.

3. Load your dataset like this:

You can also load data using a Python Dictionary

from athina.loaders import RagLoader

dataset = RagLoader().load_json(json_filepath)

4. Now you can run evals like this.

from athina.evals import DoesResponseAnswerQuery

DoesResponseAnswerQuery().run_batch(data=dataset)

For more details, see this guide on running evals.

OverviewOnline Evals