  Non-Local (NL) blocks have been widely studied in various vision tasks.
However, it has been rarely explored to embed the NL blocks in mobile neural
networks, mainly due to the following challenges: 1) NL blocks generally have
heavy computation cost which makes it difficult to be applied in applications
where computational resources are limited, and 2) it is an open problem to
discover an optimal configuration to embed NL blocks into mobile neural
networks. We propose AutoNL to overcome the above two obstacles. Firstly, we
propose a Lightweight Non-Local (LightNL) block by squeezing the transformation
operations and incorporating compact features. With the novel design choices,
the proposed LightNL block is 400x computationally cheaper} than its
conventional counterpart without sacrificing the performance. Secondly, by
relaxing the structure of the LightNL block to be differentiable during
training, we propose an efficient neural architecture search algorithm to learn
an optimal configuration of LightNL blocks in an end-to-end manner. Notably,
using only 32 GPU hours, the searched AutoNL model achieves 77.7% top-1
accuracy on ImageNet under a typical mobile setting (350M FLOPs), significantly
outperforming previous mobile models including MobileNetV2 (+5.7%), FBNet
(+2.8%) and MnasNet (+2.1%). Code and models are available at
https://github.com/LiYingwei/AutoNL.
