ByteDance Seed and Tsinghua AIR released DAPO. It is an open-source reinforcement learning system for large language models. The system introduces the Decoupled Clip and Dynamic Sampling Policy Optimization algorithm. DAPO achieved 50 points on AIME 2024 using the Qwen2.5-32B base model. It outperformed the previous state-of-the-art with half the training steps. An earlier version without token-level PG loss and dynamic sampling scored 44 points. The project provides code, datasets, and model weights. Training records and checkpoints are available on Weights & Biases and Hugging Face. The system builds on the verl framework. Reproduction scripts and evaluation tools are included.
What it unlocks
Training large language models with decoupled clip and dynamic sampling policy optimization.
- 50% on AIME 2024
- 44% on AIME 2024
- 50% fewer training steps
What you need to act on it
- Python 3.10
- verl framework
- Ray Serve and vLLM for evaluation
Sources