  Can we automatically design a Convolutional Network (ConvNet) with the
highest image classification accuracy under the latency constraint of a mobile
device? Neural Architecture Search (NAS) for ConvNet design is a challenging
problem due to the combinatorially large design space and search time (at least
200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a
novel differentiable NAS method for designing device-efficient ConvNets in less
than 4 hours. 1. Novel NAS formulation: our method introduces a single-path,
over-parameterized ConvNet to encode all architectural decisions with shared
convolutional kernel parameters. 2. NAS efficiency: Our method decreases the
NAS search cost down to 8 epochs (30 TPU-hours), i.e., up to 5,000x faster
compared to prior work. 3. On-device image classification: Single-Path NAS
achieves 74.96% top-1 accuracy on ImageNet with 79ms inference latency on a
Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with
similar latency (<80ms).
