Currently you can query the inference logs and the evaluations for those inferences via the GraphQL API. GraphQL API expects JSON with two essentials things: “query” and “variables”.

Here are some sample queries (along with corresponding variables and response):

​
Inference log queries

1. Get paginated inference logs:

query GetPromptRuns($limit: Int!, $page: Int!) {
  getPromptRunsByFilters(limit: $limit, page: $page) {
    id
    org_id
    prompt_slug
    language_model_id
    prompt_response
    prompt_tokens
  }
}

Variables for the above query:

{
  "limit": 2,
  "page": 0
}

2. Get evaluations of these inference logs:

query GetPromptRuns($limit: Int!, $page: Int!) {
  getPromptRunsByFilters(limit: $limit, page: $page) {
    id
    prompt_slug
    language_model_id
    environment
    prompt_run_topic {
      topic {
        label
      }
    }
    eval_result {
      id
      results
      eval_name
      eval_description
      eval_result_metric {
        value
        eval_result_id
        eval_metric {
          type
          label
          description
        }
      }
    }
  }
}

Variables for the above query:

{
  "limit": 1,
  "page": 0
}

3. Get inference logs filtered by created at:

query GetPromptRunsFilteredByCreatedAt($limit: Int!, $page: Int!, $start_date: String, $end_date: String) {
  getPromptRunsByFilters(limit: $limit, page: $page, start_date: $start_date, end_date: $end_date) {
    id
    org_id
    workspace_slug
    prompt_slug
    language_model_id
    prompt_response
    prompt_tokens
    eval_result {
      id
      results
      eval_name
      eval_description
      eval_result_metric {
        value
        eval_result_id
        eval_metric {
          type
          label
          description
        }
      }
    }
  }
}

Variables for the above query:

{
  "limit": 1,
  "page": 0,
  "start_date": "2024-09-04",
  "end_date": "2024-09-06"
}

Note that the start_date and end_date should be in the format YYYY-MM-DD and the end_date should be greater than the start_date. Both the dates are inclusive.

4. Get paginated inference logs with custom attribute filters:

query GetPromptRuns($limit: Int!, $page: Int!, $customAttributes: [CustomAttributeFilter]) {
  getPromptRunsByFilters(limit: $limit, page: $page, customAttributes: $customAttributes) {
    id
    org_id
    prompt_slug
    language_model_id
    prompt_response
    prompt_tokens
    prompt_run_custom_attribute {
      key
      value
    }
  }
}

Variables for the above query:

{
  "limit": 2,
  "page": 0,
  "customAttributes": [
    {"key": "example_key", "value": "example_value"}
  ]
}

​
Dataset Queries

1. Get all Datasets:

query GetDatasets{
  getDatasets{
    id
    name
  }
}

2. Get a dataset with rows:

query GetDataset($datasetId: String!, $limit: Int) {
  getDataset(datasetId: $datasetId, limit: $limit) {
    id
    name
    rows {
      query
      response
      eval_results {
        id
        metric_id
        metric_value
        explanation
        eval_name
        eval_type
        eval_config
      }
    }
  }
}

Variables for the above query:

{
  "datasetId": "fb292fef-cf9e-49c7-a167-63f39034e693",
  "limit": 2
}
Getting StartedCurl and Python Examples