Five large language models were tested on drug product recommendations. They used 177,120 synthetic clinical cases across 123 disease–drug pairs. The same product was recommended first in 97.5% of cases for a given active ingredient. Models agreed almost perfectly on which product to pick. Recommended products often cost more than real-world prescribing patterns. Products with higher online visibility were recommended more often. Originator products were strongly favored over generics. Later-approved and commonly prescribed products were recommended less. Modified prompts shifted recommendations but did not remove the bias. The study calls for rigorous evaluation before widespread use.
What it unlocks
Identifying systematic bias in LLM drug recommendations that may affect cost and equity.
- 177,120 synthetic clinical vignettes
- 123 disease–drug pairs
- 1,440 sociodemographic profiles
- 97.5% first-choice recommendation share
Sources