How do you manage costs for LLM evaluation?

Evals run with your LLM keys. You can configure the keys, and track the cost of evals in the settings page.

There are a few controls that help you manage the cost of LLM evals in Athina.

• You can configure which models to use for each eval.

​

Online Evals

• Max Evals: You can configure max evals per month in the settings. For example, if your max evals per month is set at 30k per month, we will sample 1000 logs per day.

• Sampling Rate: You can configure a sampling rate in the settings. For example, if your sampling rate is set at 10%, we will sample 10% of all logs (while still respecting the max evals limit).

• Filters: You can filter evals by model in the settings.