Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries
How AI models leak secrets hidden in their attention patterns
Tabular foundation models — AI systems trained to work with spreadsheet-like data — can be tricked into revealing whether specific records were used during their processing, even when the model was pre-trained on synthetic data. Researchers created a new attack that reads the model's attention patterns to detect membership in context examples with 7.7% higher accuracy than existing methods, then developed a defence that reduces this privacy leakage by half without significantly harming the model's performance.
As businesses deploy these models to make decisions using sensitive customer or patient data, the privacy risk is concrete: attackers can determine whether a specific person's medical record or financial information was part of the data fed to the model. The proposed defence offers a practical fix that works at inference time without retraining, making it immediately applicable to systems already in production.