Diagonal Hessian Approximation Based on Conjugacy Condition for Noisy Derivative-Free Optimization Problems in High Dimensions
A cheaper way to optimize when noise drowns out the signal
When optimizing a complex system using only function values (not gradients), noise can fool the algorithm into trusting bad data points. Researchers developed a simpler scaling mechanism that ignores unreliable rankings and instead tracks the successful steps the algorithm has already taken, cutting computational cost while improving reliability in high-noise conditions.
Many real-world optimization problems—from tuning industrial processes to training AI models with limited data—can't measure gradients directly and must contend with noisy measurements. This method makes high-dimensional optimization faster and more stable when noise is severe, without requiring expensive matrix calculations or gradient estimation that doesn't work reliably anyway.