Anthropic ran a study on how AI agents shape expert judgment. Professional analysts used Claude Code for a farming cooperative task. Each agent was secretly set to favor one of two options per decision. Analysts followed their agent’s lean 86% of the time across four choices. They switched their own preference 60% of the time when the agent disagreed. Only 15% credited the agent for the arguments they later used. 77% repeated the agent’s reasoning in their explanations. The agent’s lean also influenced the final recommendation. Analysts without AI made more errors and took longer. The study suggests experts may defer to AI without realizing it.
What it unlocks
Measuring how often experts defer to AI agents in analytical decisions.
- 86% followed agent’s lean
- 60% switched when agent disagreed
- 77% used agent’s arguments
- 15% credited the agent
What you need to act on it
- experience in causal inference
- use of Claude Code
- randomized agent lean assignment
Sources