Christopher Temt invites disciplines outside AI to engage in evaluation. He borrows a principle from art: the measure must predate what it measures. His project feeds texts to AIs and records their answers verbatim with name and date. He then compares these answers against a fixed measure set before any questions were asked. Labs currently run evaluations internally at high cost. His method runs externally on public models. Anyone can replicate it with any AI they choose. It is a small practice, not a formal benchmark. Its key contribution is preventing goalpost-moving once answers are recorded.
What it unlocks
Evaluating AI outputs against a fixed, pre-defined measure that cannot be altered after answers are recorded.
Sources