- You can configure which models to use for each eval.
Online Evals
- Max Evals: You can configure max evals per month in the settings. For example, if your max evals per month is set at 30k per month, we will sample 1000 logs per day.
- Sampling Rate: You can configure a sampling rate in the settings. For example, if your sampling rate is set at 10%, we will sample 10% of all logs (while still respecting the max evals limit).
- Filters: You can filter evals by model in the settings.