The Attribution Impossibility: No Feature Ranking Is Faithful, Stable, and Complete Under Collinearity
New research proves that feature ranking in machine learning models is inherently unstable and unreliable when input features are collinear.
Researchers have demonstrated that no feature ranking method can be simultaneously faithful, stable, and complete under collinearity. For collinear features, rankings are essentially random. The study identifies only two valid design families: unstable faithful-complete methods and stable ensemble methods like DASH. The findings were verified using 305 Lean 4 theorems.