SciFigQual-Bench: A Benchmark for Scientific Figure Quality Assessment with Full-Manuscript Context
Teaching AI to judge whether scientific figures actually support their claims
Researchers created a benchmark of over 6,300 scientific figures from top computer-science papers, each scored by experts on five qualities: clarity, layout, how well captions match images, relevance to the text, and whether the figure misleads. They then built an AI system that learned to evaluate figures the same way humans do, achieving 93% consistency with expert judgment—significantly better than existing image-quality tools, which weren't designed for scientific content.
Scientific papers rely on figures to communicate results, but no automated system has existed to catch figures that are unclear, mislabeled, or misleading. This benchmark and AI tool could help journal editors and researchers spot problematic figures before publication, reducing the spread of poorly presented—or even deceptive—research findings.