Datasets
Create Dataset Request Body Fields
Datasets
Create Dataset Request Body Fields
{
// required
"source": "api",
"name": "Demo Dataset",
// optional
"description": "This is a sample demo dataset",
"language_model_id": "gpt-4",
"prompt_template": [{
"role": "system",
"content": "You are an AI that answers user queries given the information from the context"
},
{
"role": "user",
"content": "Given this context {context}, Answer this query: {query}"
}],
"dataset_rows": [
{
"query": "When did Virgin Australia start operating?",
"context": [
"\"Virgin Australia, the trading name of Virgin Australia Airlines Pty Ltd, is an Australian-based airline. It is the largest airline by fleet size to use the Virgin brand. It commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route. It suddenly found itself as a major airline in Australia's domestic market after the collapse of Ansett Australia in September 2001. The airline has since grown to directly serve 32 cities in Australia, from hubs in Brisbane, Melbourne and Sydney.\""
],
"response": "Virgin Australia commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route.",
"expected_response": "Virgin Australia started services in August 2000"
},
{
"query": "When was Tomoaki Komorida born?",
"context": [
"\"Komorida was born in Kumamoto Prefecture on July 10, 1981. After graduating from high school, he joined the J1 League club Avispa Fukuoka in 2000. Although he debuted as a midfielder in 2001, he did not play much and the club was relegated to the J2 League at the end of the 2001 season. In 2002, he moved to the J2 club Oita Trinita. He became a regular player as a defensive midfielder and the club won the championship in 2002 and was promoted in 2003. He played many matches until 2005. In September 2005, he moved to the J2 club Montedio Yamagata. In 2006, he moved to the J2 club Vissel Kobe. Although he became a regular player as a defensive midfielder, his gradually was played less during the summer. In 2007, he moved to the Japan Football League club Rosso Kumamoto (later Roasso Kumamoto) based in his local region. He played as a regular player and the club was promoted to J2 in 2008. Although he did not play as much, he still played in many matches. In 2010, he moved to Indonesia and joined Persela Lamongan. In July 2010, he returned to Japan and joined the J2 club Giravanz Kitakyushu. He played often as a defensive midfielder and center back until 2012 when he retired.\""
],
"response": "Tomoaki Komorida was born on July 10,1981.",
"expected_response": "Tomoaki Komorida was born in July, 1981"
},
]
}
Required Fields
source
string
requiredSource of the dataset creation. For creation via API, this field must be ‘api’
name
string
requiredName of the dataset. Name must be unique. If dataset with the same name already exists, API will throw 400
description
string
Description of the dataset
language_model_id
string
Language Model that was used to generate the LLM responses
prompt_template
json
Prompt template that was used to generate the LLM responses of the dataset
dataset_rows
array
Rows of the dataset.
Currently, a dataset can have a maximum of 1000 rows. If the number of rows > 1000, API will throw 400
Fields in a single dataset row (All fields are optional)
query
string
User query to generate the LLM response
context
array[string]
Relevant context fetched from the retrieval systems
response
string
Generated LLM Response
expected_response
string
Ground Truth
{
// required
"source": "api",
"name": "Demo Dataset",
// optional
"description": "This is a sample demo dataset",
"language_model_id": "gpt-4",
"prompt_template": [{
"role": "system",
"content": "You are an AI that answers user queries given the information from the context"
},
{
"role": "user",
"content": "Given this context {context}, Answer this query: {query}"
}],
"dataset_rows": [
{
"query": "When did Virgin Australia start operating?",
"context": [
"\"Virgin Australia, the trading name of Virgin Australia Airlines Pty Ltd, is an Australian-based airline. It is the largest airline by fleet size to use the Virgin brand. It commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route. It suddenly found itself as a major airline in Australia's domestic market after the collapse of Ansett Australia in September 2001. The airline has since grown to directly serve 32 cities in Australia, from hubs in Brisbane, Melbourne and Sydney.\""
],
"response": "Virgin Australia commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route.",
"expected_response": "Virgin Australia started services in August 2000"
},
{
"query": "When was Tomoaki Komorida born?",
"context": [
"\"Komorida was born in Kumamoto Prefecture on July 10, 1981. After graduating from high school, he joined the J1 League club Avispa Fukuoka in 2000. Although he debuted as a midfielder in 2001, he did not play much and the club was relegated to the J2 League at the end of the 2001 season. In 2002, he moved to the J2 club Oita Trinita. He became a regular player as a defensive midfielder and the club won the championship in 2002 and was promoted in 2003. He played many matches until 2005. In September 2005, he moved to the J2 club Montedio Yamagata. In 2006, he moved to the J2 club Vissel Kobe. Although he became a regular player as a defensive midfielder, his gradually was played less during the summer. In 2007, he moved to the Japan Football League club Rosso Kumamoto (later Roasso Kumamoto) based in his local region. He played as a regular player and the club was promoted to J2 in 2008. Although he did not play as much, he still played in many matches. In 2010, he moved to Indonesia and joined Persela Lamongan. In July 2010, he returned to Japan and joined the J2 club Giravanz Kitakyushu. He played often as a defensive midfielder and center back until 2012 when he retired.\""
],
"response": "Tomoaki Komorida was born on July 10,1981.",
"expected_response": "Tomoaki Komorida was born in July, 1981"
},
]
}