Researchers built GlobeBench, a benchmark for text-based world models. It contains 1,632 tasks across five domains and nineteen skills. Tasks test if models predict environment responses to actions. A hybrid system compares predictions to execution-derived references. GPT-6 Astra scored highest at 49.57% success. Model accuracy drops when actions affect larger environment portions. Success falls to 5.1% when changes must propagate to dependent objects. 631 tasks remain unsolved by any model. The code and data are publicly available.
What it unlocks
Comparing text-based world models on standardized environment simulation tasks.
- 1,632 tasks across five domains
- 19 simulation skills
- 49.57% task success for GPT-6 Astra
- 5.1% average success on propagated changes
Sources