Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passing
A training method that predicts test performance without wasting data on validation
Machine learning models trained on data gradually become overfit, causing their performance on training data to look better than it actually is on new data. Researchers developed a new training algorithm called decoupled descent that cancels out this bias as it trains, allowing the training error to accurately predict test performance without setting aside data for validation—using 100% of available data while still knowing how well the model will perform.
Current machine learning practice forces a choice: either waste 10–20% of your data on a validation set to estimate real performance, or train blindly and risk deploying an overfit model. This algorithm could eliminate that trade-off, letting practitioners use all their data while still getting reliable estimates of how their model will perform in the real world. The method was tested on image classification tasks and consistently narrowed the gap between training and test performance compared to standard training approaches.