Researchers ran large-scale Turing-like tests across language and vision tasks. They collected responses from 636 humans and 37 AI systems. 1,916 human judges and 10 AI judges then tried to tell them apart. Current AI is getting close to fooling human judges in both language and vision. Simple AI judges still beat humans at spotting AI responses. Passing as human does not line up with standard AI performance scores. The work introduces new datasets and metrics for testing human-likeness. The authors argue such tests should be a core part of AI evaluation.

• 636 humans tested
• 37 AI agents tested
• 72,191 Turing-like tests run
• 1,916 human judges