  In recent times, the need for effective super-resolution (SR) techniques has
surged, especially for large-scale images ranging 2K to 8K resolutions. For
DNN-based SISR, decomposing images into overlapping patches is typically
necessary due to computational constraints. In such patch-decomposing scheme,
one can allocate computational resources differently based on each patch's
difficulty to further improve efficiency while maintaining SR performance.
However, this approach has a limitation: computational resources is uniformly
allocated within a patch, leading to lower efficiency when the patch contain
pixels with varying levels of restoration difficulty. To address the issue, we
propose the Pixel-level Classifier for Single Image Super-Resolution (PCSR), a
novel method designed to distribute computational resources adaptively at the
pixel level. A PCSR model comprises a backbone, a pixel-level classifier, and a
set of pixel-level upsamplers with varying capacities. The pixel-level
classifier assigns each pixel to an appropriate upsampler based on its
restoration difficulty, thereby optimizing computational resource usage. Our
method allows for performance and computational cost balance during inference
without re-training. Our experiments demonstrate PCSR's advantage over existing
patch-distributing methods in PSNR-FLOP trade-offs across different backbone
models and benchmarks. The code is available at
https://github.com/3587jjh/PCSR.
