Knowledge Acquisition During Pre-training? Large Language Models Learn Better With Auxiliary Views
Why showing AI the same idea multiple ways helps it learn better
Large language models learn facts more effectively when trained on different rewordings of the same knowledge, even when the total number of training examples stays constant. Surprisingly, this benefit holds across different sizes of training batches and types of knowledge, and works regardless of whether the rewordings come from a strong or weak source model.
This explains a real puzzle about why diverse training data improves AI performance—variety in how information is presented teaches the model more than simple repetition alone. The finding could guide how companies prepare training data for language models, suggesting that investing in diverse reformulations of key facts and concepts produces smarter AI systems with the same computational budget.