  Neural architecture search (NAS) has become a common approach to developing
and discovering new neural architectures for different target platforms and
purposes. However, scanning the search space is comprised of long training
processes of many candidate architectures, which is costly in terms of
computational resources and time. Regression algorithms are a common tool to
predicting a candidate architecture's accuracy, which can dramatically
accelerate the search procedure. We aim at proposing a new baseline that will
support the development of regression algorithms that can predict an
architecture's accuracy just from its scheme, or by only training it for a
minimal number of epochs. Therefore, we introduce the NAAP-440 dataset of 440
neural architectures, which were trained on CIFAR10 using a fixed recipe. Our
experiments indicate that by using off-the-shelf regression algorithms and
running up to 10% of the training process, not only is it possible to predict
an architecture's accuracy rather precisely, but that the values predicted for
the architectures also maintain their accuracy order with a minimal number of
monotonicity violations. This approach may serve as a powerful tool for
accelerating NAS-based studies and thus dramatically increase their efficiency.
The dataset and code used in the study have been made public.
