- classify user query intent
- (LLM) normalize the query
- (ApiCall) call some APIs based on user query intent
- (Retrieval) retrieve relevant documents from Vector DB
- (LLM) generate a response based on the info in the previous steps
How does it work?
You can add dynamic column to run prompts on an LLM, call an API endpoint, extract structured data, classify values, retrieve documents, etc You can add as many dynamic columns as you would like to build up a complex data pipeline Here’s a 30 second demo showing you how a dynamic column works.Why is this useful?
- You can test complex chains (instead of just prompt-response pairs)
- You can prototype and compare different pipelines (in a spreadsheet UI)
- You can create multi-step evaluations
Types of Dynamic Columns
Currently, we support the following dynamic columns:API Call
Useful to call external APIs (ex: transcription, get info from DB, etc)
Run Prompt
Generate an LLM response by running a prompt on any model!
Classification
Classify values from other columns into user-defined labels
Extract Entities
Extract an array of entities from any column
Retrieval
Retrieve documents from a vector database
Conditional
Conditional execute any dynamic column based on the value of other columns
You can also Run Evaluations in the Datasets UI similar
to dynamic columns.