> ## Documentation Index
> Fetch the complete documentation index at: https://docs.athina.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# 🚅 LiteLLM

**LiteLLM** ([GitHub ](https://github.com/BerriAI/litellm)) is a popular open-source library that lets you use any LLM as a drop in replacement for GPT. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs).

## Integration[](#integration)

If you're already using LiteLLM, you just need to add an Athina API key and callback handler to instantly log your responses across all providers to Athina.

```python
# Set the API key
os.environ["ATHINA_API_KEY"] = "your athina api key"

# Set callback
litellm.success_callback = ["athina"]
```

## Full Example[](#full-example)

```python
import os
import litellm
from litellm import completion

os.environ["ATHINA_API_KEY"] = "your athina api key"
os.environ["OPENAI_API_KEY"] = "your openai api key"
os.environ["COHERE_API_KEY"] = "your cohere api key"

# Add athina as success callback
litellm.success_callback = ["athina"]

# openai call
response = completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ]
)

# cohere call
response = completion(
  model="command-nightly",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm cohere"}
  ]
)
```

## Additional information in metadata[](#additional-information-in-metadata)

You can send some additional information by using the `metadata` field in completion. This can be useful for sending metadata about the request, such as the customer\_id, prompt\_slug, or any other information you want to track.

```python
#openai call with additional metadata
response = completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ],
  metadata={
    "environment": "staging",
    "prompt_slug": "my_prompt_slug/v1"
  }
)
```

Following are the allowed fields in metadata, their types, and their descriptions:

* `environment: Optional[str]` - Environment your app is running in (ex: production, staging, etc). This is useful for segmenting inference calls by environment.
* `prompt_slug: Optional[str]` - Identifier for the prompt used for inference. This is useful for segmenting inference calls by prompt.
* `customer_id: Optional[str]` - This is your customer ID. This is useful for segmenting inference calls by customer.
* `customer_user_id: Optional[str]` - This is the end user ID. This is useful for segmenting inference calls by the end user.
* `session_id: Optional[str]` - is the session or conversation ID. This is used for grouping different inferences into a conversation or chain.
* `external_reference_id: Optional[str]` - This is useful if you want to associate your own internal identifier with the inference logged to Athina.
* `context: Optional[Union[dict, str]]` - This is the context used as information for the prompt. For RAG applications, this is the "retrieved" data. You may log context as a string or as an object (dictionary).
* `expected_response: Optional[str]` - This is the reference response to compare against for evaluation purposes. This is useful for segmenting inference calls by expected response.
* `user_query: Optional[str]` - This is the user's query. For conversational applications, this is the user's last message.
* `custom_attributes: Optional[Dict[str, Any]]` - This is a dictionary of custom attributes to be logged with the inference.