Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability
Why AI agents forget when their underlying models change
When an AI system gets a software upgrade, its stored memories often become unreliable—sometimes losing 13 percentage points of accuracy overnight. Researchers tested four ways of storing agent memories and found that rigid, structured formats (like knowledge graphs with fixed fields) survive model upgrades nearly intact, while flexible, compressed notes become deeply dependent on the specific model that created them and fail unpredictably when that model is replaced.
Companies regularly upgrade their AI models to improve performance, but this study shows those upgrades can silently corrupt an agent's ability to recall past conversations and learned information. The findings provide concrete guidance: organizations upgrading AI systems should either use rigid data structures that don't degrade, keep backup copies of raw conversation history for recovery, or accept that some agent memories will need to be rebuilt from scratch—not just hoped for.