  Single image denoising (SID) has achieved significant breakthroughs with the
development of deep learning. However, the proposed methods are often
accompanied by plenty of parameters, which greatly limits their application
scenarios. Different from previous works that blindly increase the depth of the
network, we explore the degradation mechanism of the noisy image and propose a
lightweight Multiple Degradation and Reconstruction Network (MDRN) to
progressively remove noise. Meanwhile, we propose two novel Heterogeneous
Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer and
more accurate features from heterogeneous models, which make it possible to
reconstruct higher-quality denoised images under extreme conditions. Extensive
experiments show that our MDRN achieves favorable performance against other SID
models with fewer parameters. Meanwhile, plenty of ablation studies demonstrate
that the introduced HMDS can improve the performance of tiny models or the
model under high noise levels, which is extremely useful for related
applications.
