DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
DAMP introduces post-training quantization for recurrent states in Gated DeltaNet and Kimi Delta Attention to reduce memory usage and decoding latency.
Recurrent states in models like GDN and KDA are typically stored in FP32, leading to high memory consumption and bandwidth bottlenecks. DAMP (Decay-Aware Mixed-Precision) addresses this by applying post-training quantization to these states, optimizing memory footprint without significantly sacrificing performance.