  Deep Image Prior (DIP) shows that some network architectures naturally bias
towards smooth images and resist noises, a phenomenon known as spectral bias.
Image denoising is an immediate application of this property. Although DIP has
removed the requirement of large training sets, it still presents two practical
challenges for denoising: architectural design and noise-fitting, which are
often intertwined. Existing methods mostly handcraft or search for the
architecture from a large design space, due to the lack of understanding on how
the architectural choice corresponds to the image. In this study, we analyze
from a frequency perspective to demonstrate that the unlearnt upsampling is the
main driving force behind the denoising phenomenon in DIP. This finding then
leads to strategies for estimating a suitable architecture for every image
without a laborious search. Extensive experiments show that the estimated
architectures denoise and preserve the textural details better than current
methods with up to 95% fewer parameters. The under-parameterized nature also
makes them especially robust to a higher level of noise.
