Researchers introduced BlindToChange, a benchmark for AI agents. It tests if agents find recent changes that affect their answers. The benchmark uses real-world events drawn from Wikidata. It can be refreshed periodically. Agents perform up to 27.5 points worse when changes are not explicitly mentioned. Over half of failures occur because agents do not verify if facts have changed. The benchmark tracks progress on this new axis.
What it unlocks
Measuring whether AI agents proactively discover recent events that affect their responses.
- 27.5 point performance gap
- 50%+ of failures from unverified facts
Sources