Fails if the text contains Personal Identifiable Information (PII).
  • Inputs: text
  • Type: boolean
  • Metrics: passed (0 or 1)

​
Example

With PII
  • Query: “Sam Altman’s Ethereum address is 0x2390jd24jJD3m29kd20kd02k30rk02.”
  • Result: Failed

Without PII
  • Query: “What is the capital of France?”
  • Result: Passed

​
How does it work?

This evaluator uses an open-source Hugging Face library to detect PII in the text. The model is a fine-tuned version of Microsoft’s Deberta V3.

​
Notes

  • The model is not perfect and might not detect all PII.
  • You can use Athina as real time guardrails, but the PII detection eval can take a few seconds to complete, so it is not recommended for chat apps. (Example Notebook )