Rethinking On-Policy Distillation of Large Language Models II: One Training Example
Training AI assistants on a single example works better than expected
Researchers trained smaller AI models to mimic larger ones using just one example, and found the smaller model kept improving for hundreds of training steps and recovered most of the performance gains from using thousands of examples. The breakthrough comes from understanding what matters: a single example exposes the student model to 71.5% of the different situations (called "states") that training on full datasets reaches, but the student absorbs that information slowly—the real bottleneck isn't data scarcity but the model's learning speed.
AI labs spend enormous resources generating training data to improve large language models. This work shows that data efficiency matters far less than we thought—adding more examples helps slightly, but the real constraint is giving the student model enough training time to actually learn from whatever data it sees. Labs could cut data collection costs significantly while achieving nearly identical results, and should instead focus on making the learning process itself faster.