Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Sampling
Popular sampling algorithms struggle more with multiple peaks than their theory suggests
Two widely used sampling methods—Wasserstein gradient flows and forward-only diffusion processes—come with theoretical guarantees of fast convergence, but researchers show these promises don't hold for distributions with multiple distinct peaks. The problem is fundamental: moving probability mass between well-separated modes requires exponentially long times, no matter how you adjust the algorithm's schedule.
These methods are increasingly used in machine learning and statistics to sample from complex distributions, with researchers often citing convergence guarantees as evidence they work well. This paper reveals those guarantees can be misleading—the algorithms may get stuck slowly transitioning between different modes for impractically long periods. Practitioners relying on these methods for multimodal problems may need fundamentally different approaches, not just parameter tweaks.