Does Multi-Agent Debate Improve AI Feedback on Research Papers?
Having AI debate itself doesn't improve feedback on economics research papers
When economists evaluated three AI-generated reports on their own meta-analysis papers, they preferred a straightforward single AI review over two more elaborate multi-agent debate systems—even though one debate tool used 30 times more computational tokens. The finding challenges the assumption that having AI systems argue with each other produces better analysis, at least for research feedback in economics.
As research institutions consider using AI to supplement or replace peer review, this suggests that more complex AI methods don't automatically produce more useful critique. The result is a cautionary finding for anyone designing AI feedback systems: computational sophistication alone doesn't guarantee better quality, and the people whose work is being evaluated remain the most reliable judges of what actually helps them improve their research.