Langchain
If you’re using Langchain, you can log your data to Athina with just a few lines of code.
All you need to do is to add the CallbackHandler
to your LLMChain
callbacks.
Install Athina Logging SDK
pip install athina-logger
Import Athina classes and set API key
from athina_logger.api_key import AthinaApiKey
from athina_logger.langchain_handler import CallbackHandler
AthinaApiKey.set_api_key(os.getenv('ATHINA_API_KEY'))
Instantiate the `CallbackHandler` with Athina metadata properties
athina_handler = CallbackHandler(
prompt_slug='customer-query-prompt/v1',
user_query='I would like to get a refund on a pair of shoes I purchased online',
environment='production',
session_id='1234',
customer_id='nike-usa',
customer_user_id='tim@apple.com',
external_reference_id='your-reference-id',
custom_attributes= {
"loggedBy": "John Doe",
"age": 24,
"isAdmin": true,
"references": null
# any other attribute to be logged
}
kwargs: Any, # Any key-value data you want to associate with the LLM calls in a chain
)
-
prompt_slug
: An identifier for the prompt that is being used. -
user_query
: The query that the user entered to the LLM. -
environment
: The environment in which the LLM is running. For example, production or development. -
session_id
: The session ID of the LLM. This is used to group multiple LLM calls together. -
customer_id
: The ID of the customer that is using the LLM. -
customer_user_id
: The ID of the user that is using the LLM. -
external_reference_id
: The ID of the external reference that is using the LLM. -
custom_attributes
: Any key-value data you want to associate with the LLM call -
kwargs
: Any key-value data you want to associate with the LLM calls in a chain. This key-value data will be stored as context in Athina Server
Sample kwargs:
context1 = "Germany is located in central europe"
context2 = "Berlin is the capital of Germany"
This will be stored as:
{
"context1": "Germany is located in central europe",
"context2": "Berlin is the capital of Germany"
}
This will be perceived as retrieved context
Add `CallbackHandler` to `LLMChain` callbacks
chain = LLMChain(
llm=ChatOpenAI(...otherProperties, callbacks=[athina_handler]),
prompt=chat_prompt,
)
chain.run('OpenAI')
Supported Models (Without Streaming)
text-davinci-003
gpt-3.5-turbo
gpt-3.5-turbo-0613
gpt-3.5-turbo-16k
gpt-3.5-turbo-16k-0613
gpt-3.5-turbo-1106
gpt-4
gpt-4-0613
gpt-4-32k
gpt-4-32k-0613
gpt-4-1106-preview
meta-llama/Llama-2-13b
meta-llama/Llama-2-13b-chat
meta-llama/Llama-2-13b-chat-hf
meta-llama/Llama-2-13b-hf
meta-llama/Llama-2-70b
meta-llama/Llama-2-70b-chat
meta-llama/Llama-2-70b-chat-hf
meta-llama/Llama-2-70b-hf
meta-llama/Llama-2-7b
meta-llama/Llama-2-7b-chat
meta-llama/Llama-2-7b-chat-hf
meta-llama/Llama-2-7b-hf
claude-2
Supported Models (With Streaming)
text-davinci-003
gpt-3.5-turbo
gpt-3.5-turbo-0613
gpt-3.5-turbo-16k
gpt-3.5-turbo-16k-0613
gpt-3.5-turbo-1106
gpt-4
gpt-4-0613
gpt-4-32k
gpt-4-32k-0613
gpt-4-1106-preview
Not using Python?
Reach out to us at hello@athina.ai - we’re happy to add support for other stacks as well if we hear from you.