JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
Testing AI agents that must coordinate tasks across phones, computers, and tablets
Researchers created JarvisGUI, a benchmark that tests whether AI agents can handle workflows spanning multiple devices—like transferring data from a phone to a laptop and then to a desktop. When tested on Android, Windows, and Ubuntu machines simultaneously, even the best existing agents failed to keep track of shared information, understand context across platforms, or manage complex chains of dependent steps—capabilities essential for real-world use.
Most AI agent testing today happens on isolated single-device tasks, giving an inflated sense of how ready these systems are for actual work. JarvisGUI reveals a fundamental weakness: agents can't reliably move data between your phone and computer or maintain consistency across platforms. Until this gap closes, GUI agents won't be trustworthy for common workflows like research projects, file organization, or data processing that naturally span multiple devices.