MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant
Why voice assistants fail at joining group conversations
Voice assistants can handle one-on-one chats reasonably well, but they stumble badly in group settings. When tested on a new benchmark called MP-Bench, 12 popular voice agents scored at or below 22% on understanding group conversations and performed near random chance at knowing when to speak—exposing a major gap between lab performance and real-world group dynamics.
Voice assistants are increasingly built into homes, cars, and offices where they'll encounter multiple people talking at once. Until they can understand who's speaking, follow conversational flow, and know when it's appropriate to jump in, they'll either stay silent when useful or interrupt awkwardly. This benchmark provides the first standard way to measure and improve that capability, which is essential before these agents can actually work well in family dinners, meetings, or shared spaces.