Layer-wise Curriculum Learning for Efficient LLM Compression
Layer-wise curriculum learning improves LLM compression by partitioning models into segments to optimize knowledge transfer from teacher to student.
This approach addresses cumulative error by breaking the model into layer segments. By starting with simpler optimization tasks and progressing to harder ones, the method accelerates convergence during the compression process.