LLM Agents as Static Level-k Players in Behavioural Games
Why AI agents don't think strategically like humans in economic games
When researchers tested large language models in two classic economic games—a guessing game and a cooperation game—the AI agents behaved nothing like human players, despite producing similar-looking choices on the surface. The models act as fixed strategic thinkers based purely on their size, never adjusting their strategy mid-game or thinking several moves ahead the way humans do.
As companies and researchers increasingly use LLMs to simulate human behavior in economic or social experiments, this work reveals a critical flaw: matching surface-level choice distributions isn't enough. An AI might pick the same numbers as humans, but for entirely different reasons—which means using LLMs as stand-ins for human subjects in behavioral research could lead to false conclusions about how people actually make decisions under pressure.