  Neural architecture search (NAS) enables researchers to automatically explore
broad design spaces in order to improve efficiency of neural networks. This
efficiency is especially important in the case of on-device deployment, where
improvements in accuracy should be balanced out with computational demands of a
model. In practice, performance metrics of model are computationally expensive
to obtain. Previous work uses a proxy (e.g., number of operations) or a
layer-wise measurement of neural network layers to estimate end-to-end hardware
performance but the imprecise prediction diminishes the quality of NAS. To
address this problem, we propose BRP-NAS, an efficient hardware-aware NAS
enabled by an accurate performance predictor-based on graph convolutional
network (GCN). What is more, we investigate prediction quality on different
metrics and show that sample efficiency of the predictor-based NAS can be
improved by considering binary relations of models and an iterative data
selection strategy. We show that our proposed method outperforms all prior
methods on NAS-Bench-101 and NAS-Bench-201, and that our predictor can
consistently learn to extract useful features from the DARTS search space,
improving upon the second-order baseline. Finally, to raise awareness of the
fact that accurate latency estimation is not a trivial task, we release
LatBench -- a latency dataset of NAS-Bench-201 models running on a broad range
of devices.
