Learning When to Trust via Selective Context Preference Optimization
Teaching AI when to believe new information and when to ignore it
Language models often get confused by misleading external information—they either blindly trust it and produce wrong answers, or ignore all context and become useless. Researchers created a training method called SCOPE that teaches models to selectively trust context: accept it when it's correct or irrelevant, but reject it when it conflicts with what they already know correctly. The method reduced errors from misleading signals across multiple popular AI models while keeping their accuracy intact when the context was actually helpful.
AI systems increasingly pull information from external sources—search results, databases, user inputs—to answer questions. If they can't distinguish good information from bad, they'll either confidently give wrong answers or refuse to use helpful sources. This work offers a practical way to make AI systems more reliable in the real world, where context is often messy and mixed in quality.