Currently you can query the inference logs and the evaluations for those inferences via the GraphQL API. GraphQL API expects JSON with two essentials things: “query” and “variables”.

Here are some sample queries (along with corresponding variables and response):

​
Inference log queries

1. Get paginated inference logs:

query GetPromptRuns($limit: Int!, $page: Int!) {
  getPromptRunsByFilters(limit: $limit, page: $page) {
    id
    org_id
    prompt_slug
    language_model_id
    prompt_response
    prompt_tokens
  }
}

Variables for the above query:

{
  "limit": 2,
  "page": 0
}

2. Get evaluations of these inference logs:

query GetPromptRuns($limit: Int!, $page: Int!) {
  getPromptRunsByFilters(limit: $limit, page: $page) {
    id
    prompt_slug
    language_model_id
    environment
    prompt_run_topic {
      topic {
        label
      }
    }
    eval_result {
      id
      results
      eval_name
      eval_description
      eval_result_metric {
        value
        eval_result_id
        eval_metric {
          type
          label
          description
        }
      }
    }
  }
}

Variables for the above query:

{
  "limit": 1,
  "page": 0
}

​
Dataset Queries

1. Get all Datasets:

query GetDatasets{
  getDatasets{
    id
    name
  }
}

2. Get a dataset with rows:

query GetDataset($datasetId: String!, $limit: Int) {
  getDataset(datasetId: $datasetId, limit: $limit) {
    id
    name
    rows {
      query
      response
      eval_results {
        id
        metric_id
        metric_value
        explanation
        eval_name
        eval_type
        eval_config
      }
    }
  }
}

Variables for the above query:

{
  "datasetId": "fb292fef-cf9e-49c7-a167-63f39034e693",
  "limit": 2
}
Getting StartedCurl and Python Examples