World Models in Pieces: Structural Certification for General Agents
Testing AI agents by checking what they actually understand, not everything they could fail at
AI agents designed to handle many different tasks are inherently specialists—good at some things, weak at others. Standard safety tests treat all failures equally, missing where an agent truly understands its world and where it's just guessing. This paper introduces a new testing method that maps an agent's actual performance on specific tasks directly to measurable reliability of its internal understanding, with proven error bounds.
Current safety certification for general AI agents is too blunt: a single worst-case failure in any scenario can block deployment, even if the agent works reliably in the scenarios that matter. This work makes it possible to certify when an agent is safe to deploy on specific tasks by proving exactly where its planning is trustworthy and where it isn't. This could enable practical deployment of capable AI systems while maintaining verifiable safety guarantees.