  With the development of Deep Neural Networks (DNNs), plenty of methods based
on DNNs have been proposed for Single Image Super-Resolution (SISR). However,
existing methods mostly train the DNNs on uniformly sampled LR-HR patch pairs,
which makes them fail to fully exploit informative patches within the image. In
this paper, we present a simple yet effective data augmentation method. We
first devise a heuristic metric to evaluate the informative importance of each
patch pair. In order to reduce the computational cost for all patch pairs, we
further propose to optimize the calculation of our metric by integral image,
achieving about two orders of magnitude speedup. The training patch pairs are
sampled according to their informative importance with our method. Extensive
experiments show our sampling augmentation can consistently improve the
convergence and boost the performance of various SISR architectures, including
EDSR, RCAN, RDN, SRCNN and ESPCN across different scaling factors (x2, x3, x4).
Code is available at https://github.com/littlepure2333/SamplingAug
