  Recent advances in single image super-resolution (SISR) explored the power of
convolutional neural network (CNN) to achieve a better performance. Despite the
great success of CNN-based methods, it is not easy to apply these methods to
edge devices due to the requirement of heavy computation. To solve this
problem, various fast and lightweight CNN models have been proposed. The
information distillation network is one of the state-of-the-art methods, which
adopts the channel splitting operation to extract distilled features. However,
it is not clear enough how this operation helps in the design of efficient SISR
models. In this paper, we propose the feature distillation connection (FDC)
that is functionally equivalent to the channel splitting operation while being
more lightweight and flexible. Thanks to FDC, we can rethink the information
multi-distillation network (IMDN) and propose a lightweight and accurate SISR
model called residual feature distillation network (RFDN). RFDN uses multiple
feature distillation connections to learn more discriminative feature
representations. We also propose a shallow residual block (SRB) as the main
building block of RFDN so that the network can benefit most from residual
learning while still being lightweight enough. Extensive experimental results
show that the proposed RFDN achieve a better trade-off against the
state-of-the-art methods in terms of performance and model complexity.
Moreover, we propose an enhanced RFDN (E-RFDN) and won the first place in the
AIM 2020 efficient super-resolution challenge. Code will be available at
https://github.com/njulj/RFDN.
