Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
Teaching AI to learn new skills without forgetting old ones
Large language models typically lose knowledge of earlier tasks when learning new ones—a problem called catastrophic forgetting. Researchers created SETA, a system that assigns different parts of the AI's brain to different tasks while keeping some parts shared, so the model can accumulate new abilities without erasing what it already knows. On two popular language models, SETA retained 15–25% more early knowledge than existing methods while staying competitive on new tasks.
AI systems that learn continuously are critical for real-world deployment—think chatbots that adapt to new industries or domains without retraining from scratch. Current systems force developers to choose between forgetting old capabilities or staying stuck in the past. SETA removes that tradeoff, making it possible to deploy language models that grow smarter and more versatile over time without expensive retraining cycles.