Logging Attributes
These are all the fields you can log to Athina.
Required Fields
Identifier for the language model used for inference. This is just a string label, all models are supported.
The prompt sent for inference. This can be either a string or the messages
array sent to OpenAI.
The response from the LLM. This can be either a string or the ChatCompletion response object from OpenAI.
Eval Fields
For most RAG evals, you must also log these fields:
The user’s query. For conversational applications, this is usually the user’s last message.
Tip: Although this isn’t required, this is highly recommended as several evals depend on this field.
The retrieved context (if using RAG).
Tip: Although this isn’t required, this is highly recommended as several evals depend on this field.
Latency
The millisecond response time of the inference.
Token Usage and Cost
Athina will automatically calculate token usage and cost if the language_model_id is a known model.
However, you can also log the cost and token usage manually. These are used for analytics.
The number of input (prompt) tokens used.
The number of output (completion) tokens used.
The number of total tokens used. If this is not logged, we will simply add
prompt_tokens and completion_tokens
The cost of the inference. If this is not provided, we will automatically
calculate the cost if the language_model_id is a known model.
Tip: If you log the entire OpenAI ChatCompletion response object to us,
we’ll automatically extract the token counts and cost.
Segmentation Fields
Optionally, you can also add the following fields for better segmentation on the dashboard
The identifier for the prompt used for inference. This is useful for segmenting inference calls by prompt.
The environment your app is running in (ex: production, staging, etc).
This is useful for segmenting inference calls by environment.
The customer ID. This is useful for segmenting inference calls by customer.
The end user ID. This is useful for segmenting inference calls by the end user.
Topics
For additional segmentation of data, you can log a topic string to associate with the log. This topic will then be used to filter logs on the dashboard, show comparisons, and granular analytics.
Logging Conversations
To group inferences into a conversation or chat session, just include the session_id field.
The session or conversation ID. This is used for grouping different inferences into a conversation or chain. Read more
Logging Custom Attributes
You can log any custom attributes to Athina to be shown as metadata.
An external reference ID for the inference. This can be used to update the logs later. Read more.
Additional metadata to add to this log.
Optionally, you can also log custom attributes with your prompt. You can pass attribute name and attribute value as key-value pair in the custom_attributes object.
Note:- A prompt run cannot have duplicate attribute names
Ground Truth
If you have ground truth responses, you can log them here. Ground truth responses are required for some evals like Answer Similarity
Ground-truth or reference response.
Tools
For tools, you may log the following fields
Tools
Tool Calls
Function Calling
For function calling, you may also log the following fields:
Function call request
Function call response
Tip: To avoid adding any latency to your application, log your inference as a fire-and-forget request.
Feedback
Learn how to update logs by ID or by external reference ID.
A number representing the end user’s feedback. For example, 1 for positive
feedback, -1 for negative feedback.
A comment from the end user about the response.
A number representing the grader’s feedback. For example, 1 for positive
feedback, -1 for negative feedback.