Researchers at the Allen Institute for AI released Molmo, a family of models that read images and text, together with PixMo, the training datasets behind them. The datasets were collected from human annotators rather than generated by existing commercial models, which the authors present as the main contribution. The largest 72-billion-parameter version scores above Claude 3.5 Sonnet and Gemini 1.5 Pro on standard tests and human comparisons, behind GPT-4o. Weights, data and code are published.
What changed
Strong openly available image-and-text models largely depended on synthetic training data generated by proprietary models.
What it unlocks
Building or fine-tuning a competitive image-and-text model using fully published weights, datasets and code, including a dataset for pointing at locations in images.
- 72B parameter best-in-class model
- first submitted 25 Sep 2024, revised 5 Dec 2024
What you need to act on it
- substantial GPU capacity to train or fine-tune
- arxiv.org2026-08-10