Researchers proposed a self-training framework for large language models. It uses self-generated question-answer pairs and reasoning demonstrations. The model creates its own curriculum from high-level topic descriptions alone. It improves across text and multimodal tasks without human-curated prompts. Capabilities on untrained domains remain intact. The approach works on standard language and visual question-answering benchmarks. The paper shows foundation models can amplify latent abilities without external supervision.
What changed
Existing on-policy self-distillation required ground-truth question-answer pairs or human-curated problem sets.
What it unlocks
Models can now improve themselves using only coarse topic descriptions and no external data.
Sources