DemoPSD: Disagreement-Modulated Policy Self-Distillation
Teaching AI to learn from itself without cheating on test day
A new training method called DemoPSD helps large language models learn from their own outputs without picking up bad habits that fall apart when the training wheels come off. The method works by letting the model selectively ignore its teacher's guidance when doing so would help it think better on its own, rather than blindly copying everything it's told. On scientific reasoning tasks, it outperformed existing methods while maintaining the model's ability to explore different reasoning paths.
Current AI training often makes models dependent on information or shortcuts only available during training, causing them to fail on new problems. DemoPSD prevents this by keeping models honest — they learn genuine reasoning skills rather than surface patterns. This matters because it makes AI systems more reliable in the real world, where they won't have access to the training setup that created them.