Researchers introduced Synthetic Hospital, a fully synthetic EHR benchmark. It contains 1,268 patients and 5,602 encounters. Every diagnosis and finding is grounded in standard medical ontologies. Physicians could not reliably distinguish its records from real charts. The best model scored 0.73 on reconstructing patient problem lists. That matched the mean of seven physicians but fell short of the best. Models missed roughly half of clinically relevant findings in summaries. The benchmark is served through a simulated hospital record system. It mirrors real EHR infrastructure with standard APIs and role-based access.
What changed
Real EHR data could not be openly shared due to privacy and lacked verifiable ground truth.
What it unlocks
Openly sharing a realistic, verifiable longitudinal EHR benchmark for clinical AI testing.
- 1,268 longitudinal patients
- 5,602 encounters
- 0.73 severity-weighted F1
- 0.89 best physician score
Sources