- Non-Streaming
- Streaming
1
Install Python SDK
Copy
pip install athina-logger
2
Set Athina API Key
Copy
from athina_logger.api_key import AthinaApiKey
AthinaApiKey.set_api_key(os.getenv('ATHINA_API_KEY'))
3
Log your inference
inference.py
Copy
response = client.chat.completions.create(
model='gpt-4-1106-preview',
messages=[{"role": "user", "content": "What is machine learning?"}],
)
response = response.model_dump() # For openai > 1 version
try:
InferenceLogger.log_inference(
prompt_slug="sdk_test",
prompt=messages,
language_model_id="gpt-4-1106-preview",
response=response,
external_reference_id="abc",
cost=0.0123,
custom_attributes={
"name": "John Doe"
# Your custom attributes
}
)
except Exception as e:
if isinstance(e, CustomException):
print(e.status_code)
print(e.message)
else:
print(e)
Tip: Include your logging code in a try/catch block to ensure that your
application doesn’t crash if the logging request fails.
1. Install the Python SDK
Runpip install athina-logger2. Import Athina Logger and openai package
Copy
from athina_logger.api_key import AthinaApiKey
from openai import OpenAI
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))
3. Set Athina API key
Initialize the Athina API key somewhere in your code
Copy
AthinaApiKey.set_api_key(os.getenv('ATHINA_API_KEY'))
4. Call the logging function
Copy
from athina_logger.log_stream_inference.openai_chat_completion_stream import LogOpenAiChatCompletionStreamInference
# Using python openai SDK
def stream_openai_chat_response(user_query, content):
response = client.chat.completions.create(
model='gpt-3.5-turbo',
messages=[{"role": "user", "content": content}],
stream=True,
)
logger = LogOpenAiChatCompletionStreamInference(
prompt=[{"role": "user", "content": content}],
language_model_id="gpt-3.5-turbo",
prompt_slug="test",
context=context, # OPTIONAL
response_time=response_time_ms, # OPTIONAL
customer_id="stripe", # OPTIONAL
customer_user_id="[email protected]", # OPTIONAL
session_id="c45g-1234-s6g4-43d3", # OPTIONAL
user_query=user_query, # OPTIONAL
environment="production", # OPTIONAL
external_reference_id="5e838eaf-7dd0-4b6f-a32c-26110dd54e58", # OPTIONAL; If passed, should be unique across all inference calls
custom_attributes={
"name": "John Doe"
} # OPTIONAL;
)
5. Collect the streaming responses
There are 2 ways to collect openai chat streamsOption 1: Collect automatically from response
Copy
try:
logger.collect_stream_inference(response)
logger.log_stream_inference()
except Exception as e:
print(e)
Option 2: Collect individually by chunk
Copy
try:
for chunk in response:
logger.collect_stream_inference_by_chunk(chunk)
logger.log_stream_inference()
except Exception as e:
print(e)
Tip: Include your logging code in a try/catch block to ensure that your
application doesn’t crash if the logging request fails.