Task Context Shapes Component Roles in Language Models
How language models reuse the same parts for different tasks
Language models reuse the same internal components across different tasks, but researchers discovered these components don't just work the same way each time—they change how they process information depending on context. By editing internal activation states in five large language models, the team showed that the same attention heads can switch which facts they read and how they influence later processing, with prediction accuracy improving by up to 53.6% when accounting for these task-dependent shifts.
Understanding how shared components adapt to different tasks reveals how language models achieve efficiency by reusing parts rather than building separate circuits for each job. This insight could help AI researchers design better interventions to change unwanted behaviors—like reducing hallucinations or fixing reasoning errors—without accidentally breaking the model's ability to handle other tasks, making it possible to steer large language models more precisely and reliably.