Researchers at Tencent and eight universities built a way to mass-produce training tasks for command-line agents. The method, called RST, starts from verified example tasks and rewrites them into harder ones. Each round extends the reference solution, then realigns the checker and the written instructions. Every candidate is re-run in a fresh sandbox and discarded if the solution fails. Accepted tasks become seeds for the next round. Fifteen rounds produced tens of thousands of tasks with no human authoring. Difficulty rose sharply: leading models solved most early tasks and almost none by round fifteen. Training smaller Qwen3.5 models on the resulting traces improved scores on three terminal benchmarks. The team reports no ceiling, with yield and validation rates stable through the final round.
What changed
Human-written terminal-agent tasks cost hundreds to thousands of dollars each.
What it unlocks
Generating large pools of verified, progressively harder command-line tasks for agent training.
- 37,484 tasks from 639 seeds
- ~$0.05 per verified task
- median solution 67 to 374 lines
- pass@4 fell 90% to 2.5%
What you need to act on it
- a pool of verified seed tasks
- sandboxed execution infrastructure
- access to a strong generator model
Sources