  This paper proposes an Any-time super-Resolution Method (ARM) to tackle the
over-parameterized single image super-resolution (SISR) models. Our ARM is
motivated by three observations: (1) The performance of different image patches
varies with SISR networks of different sizes. (2) There is a tradeoff between
computation overhead and performance of the reconstructed image. (3) Given an
input image, its edge information can be an effective option to estimate its
PSNR. Subsequently, we train an ARM supernet containing SISR subnets of
different sizes to deal with image patches of various complexity. To that
effect, we construct an Edge-to-PSNR lookup table that maps the edge score of
an image patch to the PSNR performance for each subnet, together with a set of
computation costs for the subnets. In the inference, the image patches are
individually distributed to different subnets for a better
computation-performance tradeoff. Moreover, each SISR subnet shares weights of
the ARM supernet, thus no extra parameters are introduced. The setting of
multiple subnets can well adapt the computational cost of SISR model to the
dynamically available hardware resources, allowing the SISR task to be in
service at any time. Extensive experiments on resolution datasets of different
sizes with popular SISR networks as backbones verify the effectiveness and the
versatility of our ARM. The source code is available at
https://github.com/chenbong/ARM-Net.
