Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction
Combining fast training with accurate predictions in machine learning
Researchers created a new loss function called Linear-Core Surrogates that solves a longstanding trade-off in machine learning: smooth functions train quickly but learn slowly, while sharp functions learn efficiently but are hard to optimize. The new approach combines both benefits—it's smooth enough to train fast, yet produces predictions as accurate as harder-to-optimize functions. In structured prediction tasks like language processing, the smoothness enables a 23-fold speedup over existing methods.
Training machine learning models is expensive in both time and computational energy. This approach cuts training time dramatically—by 23× on large text tasks—without sacrificing accuracy. It also handles messy real-world data better: when labels contain errors, the method outperforms standard approaches by 2.6% on standard benchmarks, making it immediately useful for practitioners working with imperfect datasets.