OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuit
OMP-MoE is a training-free compression framework that prunes redundant experts in Mixture-of-Experts models.
The method treats expert pruning as a sparse signal reconstruction task using Orthogonal Matching Pursuit. By analyzing expert contribution patterns, it reduces memory requirements without the high search costs associated with traditional pruning techniques.