  The accuracy of deep convolutional neural networks (CNNs) generally improves
when fueled with high resolution images. However, this often comes at a high
computational cost and high memory footprint. Inspired by the fact that not all
regions in an image are task-relevant, we propose a novel framework that
performs efficient image classification by processing a sequence of relatively
small inputs, which are strategically selected from the original image with
reinforcement learning. Such a dynamic decision process naturally facilitates
adaptive inference at test time, i.e., it can be terminated once the model is
sufficiently confident about its prediction and thus avoids further redundant
computation. Notably, our framework is general and flexible as it is compatible
with most of the state-of-the-art light-weighted CNNs (such as MobileNets,
EfficientNets and RegNets), which can be conveniently deployed as the backbone
feature extractor. Experiments on ImageNet show that our method consistently
improves the computational efficiency of a wide variety of deep models. For
example, it further reduces the average latency of the highly efficient
MobileNet-V3 on an iPhone XS Max by 20% without sacrificing accuracy. Code and
pre-trained models are available at
https://github.com/blackfeather-wang/GFNet-Pytorch.
