# Athina

## Docs

- [Configure an Annotation Project](https://docs.athina.ai/annotations/configure-project.md): Learn how to set up and configure an annotation project in Athina AI.
- [Export Annotated Data](https://docs.athina.ai/annotations/export-data.md): Export labeled datasets from Athina in your preferred format.
- [Annotation Metrics](https://docs.athina.ai/annotations/metrics.md): Understand annotation progress and agreement rates across your datasets.
- [Annotation](https://docs.athina.ai/annotations/overview.md): Allow your team to annotate and label your datasets.
- [Annotation Permissions](https://docs.athina.ai/annotations/permissions.md): Manage who can create, access, and contribute to annotation projects.
- [Review Annotation Entries](https://docs.athina.ai/annotations/review-entries.md): Learn how to review and validate completed annotations directly within the dataset.
- [View Annotated Data](https://docs.athina.ai/annotations/view-annotated-data.md): Learn how to inspect completed annotations and monitor dataset coverage.
- [Add Rows to Dataset](https://docs.athina.ai/api-reference/datasets/add-rows-to-dataset.md): You can add rows to an existing dataset using the Python SDK.
- [Create a Dataset](https://docs.athina.ai/api-reference/datasets/create-dataset.md): You can create a dataset programmatically using the Python SDK.
- [Delete Dataset](https://docs.athina.ai/api-reference/datasets/delete-dataset.md): You can delete a dataset by ID via the Python SDK.
- [Get Dataset](https://docs.athina.ai/api-reference/datasets/get-dataset.md): You can get a specific dataset by ID or name.
- [List All Datasets](https://docs.athina.ai/api-reference/datasets/list.md): You can list all datasets using the Python SDK.
- [Update Cells in a Dataset](https://docs.athina.ai/api-reference/datasets/update-cell-via-api.md): You can update cells in a dataset via the Python SDK or directly via the API.
- [API Call](https://docs.athina.ai/api-reference/evals/custom-evals/api-call.md)
- [Write your own LLM Eval class](https://docs.athina.ai/api-reference/evals/custom-evals/create-your-own-eval.md)
- [Evaluation with Custom Python Code](https://docs.athina.ai/api-reference/evals/custom-evals/custom-code-eval.md)
- [LLM-as-a-Judge (Custom Prompt Eval)](https://docs.athina.ai/api-reference/evals/custom-evals/custom-prompt.md)
- [Custom Grading Criteria](https://docs.athina.ai/api-reference/evals/custom-evals/grading-criteria.md)
- [Pairwise Evaluation](https://docs.athina.ai/api-reference/evals/custom-evals/pairwise-evaluation.md)
- [Loading data for Evals](https://docs.athina.ai/api-reference/evals/loading-data/loading-data-for-eval.md)
- [Loading data via Llama-Index](https://docs.athina.ai/api-reference/evals/loading-data/loading-data-via-llama-index.md)
- [Conversation Evals](https://docs.athina.ai/api-reference/evals/preset-evals/conversation-evals.md)
- [Function Based Evaluators](https://docs.athina.ai/api-reference/evals/preset-evals/function-evals.md)
- [Grounded Evals](https://docs.athina.ai/api-reference/evals/preset-evals/grounded-evals.md)
- [Guardrails](https://docs.athina.ai/api-reference/evals/preset-evals/guardrails.md)
- [JSON Evals](https://docs.athina.ai/api-reference/evals/preset-evals/json-evals.md)
- [Preset Evals](https://docs.athina.ai/api-reference/evals/preset-evals/overview.md)
- [Answer Completeness](https://docs.athina.ai/api-reference/evals/preset-evals/rag/answer-completeness.md)
- [Context Sufficiency](https://docs.athina.ai/api-reference/evals/preset-evals/rag/context-sufficiency.md)
- [Groundedness](https://docs.athina.ai/api-reference/evals/preset-evals/rag/groundedness.md)
- [RAGAS](https://docs.athina.ai/api-reference/evals/preset-evals/rag/ragas.md)
- [Response Faithfulness](https://docs.athina.ai/api-reference/evals/preset-evals/rag/response-faithfulness.md)
- [OpenAI Content Moderation](https://docs.athina.ai/api-reference/evals/preset-evals/safety/open-ai-content-moderation.md)
- [PII Detection](https://docs.athina.ai/api-reference/evals/preset-evals/safety/pii-detection.md)
- [Prompt Injection](https://docs.athina.ai/api-reference/evals/preset-evals/safety/prompt-injection.md)
- [Summarization QA](https://docs.athina.ai/api-reference/evals/preset-evals/summarization-qa.md)
- [Running a suite of evals](https://docs.athina.ai/api-reference/evals/running-evals/run-eval-suite.md): Here is a sample of all the code you need to run a suite of evals.
- [Running Evals](https://docs.athina.ai/api-reference/evals/running-evals/run-single-eval.md)
- [Query the Graph API using cURL or Python](https://docs.athina.ai/api-reference/graphql-api/curl-python-examples.md): We have provided examples of how to query the Athina AI GraphQL API using cURL and Python. You can use these examples to fetch data from the API and integrate it into your applications.
- [Getting Started](https://docs.athina.ai/api-reference/graphql-api/getting-started.md)
- [Overview](https://docs.athina.ai/api-reference/graphql-api/overview.md)
- [Sample GraphQL Queries](https://docs.athina.ai/api-reference/graphql-api/sample-queries.md)
- [Langchain](https://docs.athina.ai/api-reference/logging/langchain.md)
- [🚅 LiteLLM](https://docs.athina.ai/api-reference/logging/lite-llm.md)
- [Log via API Request](https://docs.athina.ai/api-reference/logging/log-via-api-request.md)
- [Log via Python SDK](https://docs.athina.ai/api-reference/logging/log-via-python-sdk.md)
- [Log via TypeScript SDK](https://docs.athina.ai/api-reference/logging/log-via-typescript-sdk.md)
- [Logging Attributes](https://docs.athina.ai/api-reference/logging/logging-attributes.md)
- [OpenAI Completions](https://docs.athina.ai/api-reference/logging/misc/openai-completion-0x.md): _If you're using OpenAI completions in Python, you can get set up in just **2 minutes**_
- [OpenAI Completions](https://docs.athina.ai/api-reference/logging/misc/openai-completion-1x.md): _If you're using OpenAI completions in Python, you can get set up in just **2 minutes**_
- [OpenAI Assistant](https://docs.athina.ai/api-reference/logging/openai-assistant.md)
- [OpenAI Chat Completion](https://docs.athina.ai/api-reference/logging/openai-chat-0x.md): If you're using OpenAI chat completions in Python, you can get set up in just 2 minutes
- [OpenAI Chat Completion](https://docs.athina.ai/api-reference/logging/openai-chat-1x.md): If you're using OpenAI chat completions in Python, you can get set up in just 2 minutes
- [Logging](https://docs.athina.ai/api-reference/logging/overview.md): To get started with Athina's Monitoring, the first step is to start logging your inferences.
- [Supported Models](https://docs.athina.ai/api-reference/logging/supported-models.md)
- [Tracing](https://docs.athina.ai/api-reference/logging/tracing.md): Athina LLM Application Tracing captures the full context of an execution including retrieval, generation, api calls, and more
- [Tracing for Langchain (Python)](https://docs.athina.ai/api-reference/logging/tracing-for-langchain.md): Athina Tracing integrates with Langchain using Langchain Callbacks (Python). Thereby, our SDK automatically creates a nested trace for every run of your Langchain application.
- [Tracing using Python decorators](https://docs.athina.ai/api-reference/logging/tracing-using-python-decorators.md): Tracing to Athina can be done using Python decorators.
- [Tracing via API](https://docs.athina.ai/api-reference/logging/tracing-via-api.md): Tracing to Athina can be done via simple API requests.
- [Update Logs By External Reference ID](https://docs.athina.ai/api-reference/logging/updates/update-logs-by-external-reference-id.md)
- [Update Logs By ID](https://docs.athina.ai/api-reference/logging/updates/update-logs-by-id.md)
- [Dataset Automations](https://docs.athina.ai/datasets/automations.md): Automate workflows with project automations.
- [Compare datasets](https://docs.athina.ai/datasets/compare-datasets.md): You can compare multiple datasets side-by-side in Athina.
- [Create a Dataset](https://docs.athina.ai/datasets/create-dataset/create-dataset.md): You can currently create datasets in Athina in the following ways:
- [Upload File](https://docs.athina.ai/datasets/create-dataset/create-dataset-from-file.md): You can upload a JSONL, CSV, or JSON file to create a dataset in Athina.
- [From Logs](https://docs.athina.ai/datasets/create-dataset/create-dataset-from-logs.md): You can import your inference logs to create a dataset in Athina.
- [Via API](https://docs.athina.ai/datasets/create-dataset/create-dataset-via-api.md): You can use simple POST API requests to add rows to a dataset.
- [Via Python SDK](https://docs.athina.ai/datasets/create-dataset/create-dataset-via-python-sdk.md): You can use our Python SDK to create a dataset in Athina.
- [Generate a Synthetic Dataset](https://docs.athina.ai/datasets/create-dataset/generate-synthetic-dataset.md): You can generate synthetic datasets in Athina.
- [Import a HuggingFace Dataset](https://docs.athina.ai/datasets/create-dataset/import_huggingface_dataset.md)
- [Delete a Dataset](https://docs.athina.ai/datasets/delete-dataset.md): You can delete a dataset in Athina by following these steps.
- [Overview](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns.md): Dynamic columns let you run prompts, code execution, retrievals, and more on your datasets
- [API Call](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-api-call.md): Fetch some data from an API endpoint
- [Classification](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-classification.md): Classify text into pre-defined labels using an LLM
- [Code Execution](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-code-execution.md): Execute a python function on every row
- [Extract Entities](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-extract-entities.md): Extract entities from a previous column using an LLM
- [Run Prompt](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-run-prompt.md): Send a prompt to a language model to generate an AI response
- [Export / Download Datasets](https://docs.athina.ai/datasets/export-datasets.md): Export datasets to a file.
- [Join Datasets](https://docs.athina.ai/datasets/join-datasets.md): Import columns from another dataset.
- [Manage Datasets](https://docs.athina.ai/datasets/management.md): Organize datasets with Projects and Tags
- [View Metrics](https://docs.athina.ai/datasets/metrics.md): You can view and compare metrics across datasets in Athina.
- [Introduction](https://docs.athina.ai/datasets/overview.md)
- [Run evaluations (UI)](https://docs.athina.ai/datasets/run-eval.md): Run evaluations on Athina IDE in a few clicks
- [Run Experiments](https://docs.athina.ai/datasets/run-experiment.md): Re-generate a dataset with a new prompt or a new model and compare the results side-by-side
- [SQL Notebook](https://docs.athina.ai/datasets/sql.md): Run SQL queries on your datasets
- [Eval Cookbooks](https://docs.athina.ai/evals/cookbooks.md)
- [Custom Evals](https://docs.athina.ai/evals/custom-evals.md)
- [Offline Evals](https://docs.athina.ai/evals/offline-evals.md)
- [Online Evals](https://docs.athina.ai/evals/online-evals.md)
- [Athina Evals](https://docs.athina.ai/evals/overview.md)
- [Preset Evaluators](https://docs.athina.ai/evals/preset-evals.md)
- [Quick Start](https://docs.athina.ai/evals/quickstart.md)
- [Running Evals in CI/CD](https://docs.athina.ai/evals/running-evals-ci-cd.md)
- [Running Evals in UI](https://docs.athina.ai/evals/running-evals-in-ui.md)
- [Running Evals via SDK](https://docs.athina.ai/evals/running-evals-via-sdk.md)
- [Open Source Evaluations](https://docs.athina.ai/evals/why-athina/open-source-evals.md): The philosophy behind `athina-evals`, our open-source evaluation library.
- [Why Athina Evals](https://docs.athina.ai/evals/why-athina/why-athina-evals.md)
- [Where is data stored?](https://docs.athina.ai/faqs/data-policy.md)
- [Can I choose which model to use for running evaluations?](https://docs.athina.ai/faqs/evals/choosing-models.md)
- [How do you manage costs for LLM evaluation?](https://docs.athina.ai/faqs/evals/managing-costs.md)
- [Why use LLM-as-a-judge for evaluations?](https://docs.athina.ai/faqs/evals/why-llm-judge.md)
- [Why Not Use Traditional Evaluation Metrics?](https://docs.athina.ai/faqs/evals/why-not-traditional-metrics.md)
- [Can I log inferences from any model?](https://docs.athina.ai/faqs/logging/can-i-log-using-any-model.md)
- [Grouping Inferences](https://docs.athina.ai/faqs/logging/how-can-i-log-conversations.md)
- [Q. Will the Athina logging SDK increase my latency?](https://docs.athina.ai/faqs/logging/logging-latency.md)
- [Is this SDK going to make a proxy request to OpenAI through Athina?](https://docs.athina.ai/faqs/logging/proxy.md)
- [Q. Can Athina's observability platform be deployed on-prem?](https://docs.athina.ai/faqs/on-prem.md)
- [Code Execution](https://docs.athina.ai/flows/blocks/code_execution.md): Execute Python code in a sandbox.
- [Knowledge Retrieval](https://docs.athina.ai/flows/blocks/knowledge.md): Retrieve documents from a knowledge base.
- [Blocks](https://docs.athina.ai/flows/blocks/overview.md): Blocks and flows are composable and can be used to create powerful workflows.
- [Search](https://docs.athina.ai/flows/blocks/search.md): Search the web for information.
- [Concepts](https://docs.athina.ai/flows/concepts.md): Learn the basics of flows and blocks.
- [Flows](https://docs.athina.ai/flows/overview.md): Build complex pipelines with Flows.
- [Sharing Flows](https://docs.athina.ai/flows/share-flows.md): Learn how to share flows publicly or with selected users.
- [Flow Templates](https://docs.athina.ai/flows/templates.md): Flow Templates are public flows that you can copy and use in your own flows.
- [Variables in Flows](https://docs.athina.ai/flows/variables.md): Understand how variables work in flows.
- [Getting Started with Athina](https://docs.athina.ai/getting-started.md)
- [Integrate AWS Bedrock Models](https://docs.athina.ai/guides/datasets/aws-bedrock-model.md): Add and use custom Large Language Models in Athina.
- [Comparing datasets using Athina IDE](https://docs.athina.ai/guides/datasets/comparing-datasets.md)
- [Run an Experiment to compare and evaluate responses from different models](https://docs.athina.ai/guides/datasets/comparing-models.md)
- [Get Data from S3 Bucket](https://docs.athina.ai/guides/datasets/get-data-from-s3-bucket.md): Step-by-step guide on retrieving S3 data in to Athina.
- [Preparing Data for Fine-Tuning](https://docs.athina.ai/guides/datasets/preparing-data-for-fine-tuning.md): Step-by-Step Guide to Optimizing Your Dataset for Fine-Tuning Models in Athina.
- [Prototype and Evaluate a Prompt Chain](https://docs.athina.ai/guides/datasets/prototype-and-evaluate-prompt-chain.md): You can prototype and evaluate a prompt chain in Athina IDE
- [Run Prompts and Evaluate](https://docs.athina.ai/guides/datasets/run-prompts-and-evaluate.md)
- [Evaluations in CI/CD Pipeline](https://docs.athina.ai/guides/evals/evals-in-cicd-pipeline.md): Automating Evaluations using Athina AI in CI/CD Pipelines.
- [Evaluate Conversations](https://docs.athina.ai/guides/evals/evaluate-conversations.md): Step-by-step evaluation of a multi-turn conversation using Athina.
- [How can I improve the performance / reliability of my evals?](https://docs.athina.ai/guides/evals/improving-eval-performance.md)
- [Different stages of evaluation](https://docs.athina.ai/guides/evals/llm-eval-workflows.md)
- [How to Measure Retrieval Accuracy in RAG Applications Using Athina IDE](https://docs.athina.ai/guides/evals/measuring-retrieval-accuracy-in-rag.md)
- [Pairwise Evaluation](https://docs.athina.ai/guides/evals/pairwise-evals.md): Step by step pairwise evaluation guide to compare model outputs using Athina AI.
- [Pairwise Evaluation](https://docs.athina.ai/guides/evals/pairwise-evaluation.md)
- [Prompt Injection: Attacks and Defenses](https://docs.athina.ai/guides/evals/prompt-injection.md)
- [Which evaluations to use for RAG applications?](https://docs.athina.ai/guides/evals/rag-eval-guide.md)
- [RAG Evaluators](https://docs.athina.ai/guides/evals/rag-evals.md)
- [Running evals as real-time guardrails](https://docs.athina.ai/guides/evals/running-evals-guardrails.md)
- [Compare Multiple Models](https://docs.athina.ai/guides/experiments/compare-multiple-models.md): Step by step guide on how to compare multiple models using Athina AI.
- [Create and Share Flow](https://docs.athina.ai/guides/flows/create-and-share-flow.md): Learn how to create and share AI workflows using the Flow Builder in Athina AI.
- [Getting Started Guides](https://docs.athina.ai/guides/overview.md)
- [Prompt Comparison](https://docs.athina.ai/guides/prompts/prompt-comparison.md): Learn how to compare multiple prompts using Athina AI.
- [Prompt Versioning](https://docs.athina.ai/guides/prompts/prompt-versioning.md): A guide on managing and tracking prompts with prompt versioning in Athina AI.
- [Integrations](https://docs.athina.ai/integrations.md): Athina has integrations with the following projects/packages:
- [Analytics and Insights](https://docs.athina.ai/monitoring/analytics.md)
- [Continuous Evaluation](https://docs.athina.ai/monitoring/continuous-eval.md): Athina can run continuous evaluations on your logs to monitor model performance in production
- [How can I export my logged inferences?](https://docs.athina.ai/monitoring/export-data.md)
- [Inference Trace](https://docs.athina.ai/monitoring/inference-trace.md)
- [Athina Monitoring](https://docs.athina.ai/monitoring/overview.md): Advanced Monitoring & Analytics. Your production environment will thank you.
- [Model Performance Metrics](https://docs.athina.ai/monitoring/performance-metrics.md)
- [Query Topic Classification](https://docs.athina.ai/monitoring/topic-classification.md)
- [Athina AI](https://docs.athina.ai/overview.md): Athina is a collaborative AI development platform that lets teams build, test and monitor production-grade AI applications.
- [Concepts](https://docs.athina.ai/prompts/concepts.md): Understand how prompts are structured in Athina
- [Create Prompt Template](https://docs.athina.ai/prompts/create-prompt.md): You can create prompts in Athina's Prompt Playground, or via API
- [Delete Prompt Slug](https://docs.athina.ai/prompts/delete-prompt.md): You can delete a prompt slug in Athina's Prompt Playground, or via API
- [Duplicate Prompt Slug](https://docs.athina.ai/prompts/duplicate-prompt.md): You can duplicate a prompt slug in Athina's Prompt Playground, or via API
- [List Prompt Slugs](https://docs.athina.ai/prompts/list-prompts.md): You can list all prompt slugs stored on Athina via API or Python SDK
- [Organize Prompts](https://docs.athina.ai/prompts/organization.md): You can organize your prompts in Athina's Prompt Management System
- [Overview](https://docs.athina.ai/prompts/overview.md): Athina offers a powerful prompt management system that allows you to create, edit, manage, test and version prompts.
- [Run Evaluations on a Prompt Response](https://docs.athina.ai/prompts/prompt-evals.md): You can evaluate prompt responses in Athina's Playground.
- [Prompt Versioning](https://docs.athina.ai/prompts/prompt-versioning.md): You can version your prompts in Athina's Prompt Playground, or via API.
- [Multi-Prompt Playground](https://docs.athina.ai/prompts/run-multiple-prompts.md): You can run multiple prompts in Athina's Prompt Playground, or via API
- [Run Prompt](https://docs.athina.ai/prompts/run-prompt.md): You can run prompts in Athina's Prompt Playground, or via API
- [Prompt Syntax](https://docs.athina.ai/prompts/syntax.md): Learn how to write prompts in Athina's Playground.
- [Self-Hosting](https://docs.athina.ai/self-hosting/on-AWS.md)
- [Self-Hosting On Azure](https://docs.athina.ai/self-hosting/on-Azure.md)
- [Credits](https://docs.athina.ai/settings/credits.md): Understand how execution credits work in Athina.
- [Custom Models](https://docs.athina.ai/settings/custom-models.md): You can configure custom models on Athina in the Settings page.
- [Sampling Evals](https://docs.athina.ai/settings/sampling-evals.md): You can set up sampling rules for continuous evals in the Settings.

## OpenAPI Specs

- [openapi](https://docs.athina.ai/api-reference/openapi.json)

## Optional

- [Open-Source Evals](https://github.com/athina-ai/athina-evals)
- [Blog](https://blog.athina.ai/)
- [Email us](mailto:hello@athina.ai)
- [Book a call](https://cal.com/shiv-athina/30min)