Microsoft’s Diagnostic Orchestrator reached the correct diagnosis in 83.2% of NEJM cases. Physicians with reference tools scored 36.1% on the same cases. On emergency department cases, DxO hit 80.3% accuracy versus 46.8% for physicians. The AI asked far more questions before ordering its first test. It also ordered fewer costly tests when budget limits were imposed. The study used a text-only benchmark without imaging or real-world uncertainty. The authors caution the results do not reflect routine practice. The system runs on a commercial large language model. All authors are Microsoft employees with equity in the company.
What changed
Earlier AI diagnosis studies gave models a complete case write-up in a single prompt.
What it unlocks
An AI can now simulate a panel of physicians to reach a diagnosis step by step.
- 83.2% diagnostic accuracy on NEJM cases
- 80.3% on emergency department cases
- 36.1% to 46.8% physician accuracy
- $5,431 vs. $4,725 average test cost
What you need to act on it
- text-only cases
- controlled benchmark
- no imaging or waveforms
Sources