Running Evals in UI
- Run evals on datasets
- Configure online evals to run automatically: These will run automatically on your logged inferences, and you can view the results in the dashboard.
- Run evaluations manually from the Trace view
- Run evaluations in the Prompt playground
- Configure Automations to run evals automatically on new datasets in a project
Run Evals Programmatically
- Run evals in CI / CD pipelines
- Run evals as guardrails around inference
- Run evals using the Python SDK
athina
package
athina init
, and enter the API keys when prompted.
3. Load your dataset like this:
You can also load data using a Python Dictionary