Trust but Verify: Designing Region-Inhomogeneous Forward Dynamics for Discrete Diffusion
Researchers propose a region-inhomogeneous forward process for discrete diffusion to improve context handling.
Standard discrete diffusion models often treat context as binary, leading to error propagation. This new approach combines absorbing and uniform-mixing kernels with region-dependent rates, allowing models to treat context as uncertain during training. A heuristic sampler re-injects noise at inference to improve generation quality.
