Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
Why AI language models give different answers from the same starting point
When AI language models restart from a saved checkpoint, they sometimes produce different outputs even though they're starting from identical information. Researchers found that these differences stem from how the model's internal memory (the key-value cache) is reconstructed, not from the actual tokens themselves—and that using higher-precision math can eliminate the divergence entirely. By transplanting the memory from one model run into another, they proved the cache is the actual cause of the problem.
Language models are increasingly used for high-stakes decisions in medicine, law, and finance, where consistency matters. Right now, restarting a model from a checkpoint can silently produce different answers, creating a reproducibility problem that engineers can't easily detect. This work pinpoints exactly where the instability comes from, making it possible to either fix it (use higher precision math) or account for it (know when your model might diverge).