The Identity Trap in EEG Foundation Models: A Diagnostic Audit
When brain-reading AI learns to spot the person instead of the disease
Popular AI models trained on EEG brain scans achieve high accuracy on clinical tasks, but a new diagnostic reveals they often rely on subject-identity features rather than genuine disease markers. Researchers identified this "Identity Trap" across three major foundation models and four datasets, showing that subject-specific patterns were 13–89 times stronger than random noise—and only grew stronger after fine-tuning. They developed FMScope, a toolkit that separates real biological signals from identity shortcuts, improving accuracy by 6–27 percentage points when true clinical markers exist.
EEG-based AI models are moving into clinical use for diagnosing epilepsy, sleep disorders, and brain injuries, where misplaced confidence in false accuracy could harm patients. Without tools like FMScope, hospitals might deploy models that perform well on test data but fail on new patients they've never seen—because the model learned to recognize individual brains rather than disease patterns. This work provides a concrete method to audit foundation models before clinical deployment and shows which features are genuinely tied to disease versus which are diagnostic dead ends.