Getting Started Guides

• Create and commit your first Prompt Template

• Build and run your first Flow TODO

• Log a sample LLM inference (Python Notebook)
• Configure an online eval
• Re-run LLM calls with a different prompt or model
• View analytics and compare different segments TODO
• Create a dataset from your logs
• Export your logs TODO

• Getting started: Datasets Overview & Concepts
• Creating a dataset
• Running an offline eval in Athina IDE
• Running a prompt column in Athina IDE
• Running dynamic columns in Athina IDE
• Running an experiment in Athina IDE

Experimentation

• Comparing different models and prompts
• Comparing different datasets side-by-side
• Prototyping a prompt chain in 3 mins without writing code

Evaluation

• RAG Evaluation: A Guide
• Measure and Improve retrieval in your RAGs
• LLM-as-a-Judge Evaluation
• Pairwise Evaluation

Evaluation Best Practices

• Improving eval performance
• Different stages of evaluation

Safeguarding

• Prompt Injection: Attack and Defense
• Running evals as real-time guardrails