  Neural architecture search (NAS) has shown great success in the automatic
design of deep neural networks (DNNs). However, the best way to use data to
search network architectures is still unclear and under exploration. Previous
work has analyzed the necessity of having ground-truth labels in NAS and
inspired broad interest. In this work, we take a further step to question
whether real data is necessary for NAS to be effective. The answer to this
question is important for applications with limited amount of accessible data,
and can help people improve NAS by leveraging the extra flexibility of data
generation. To explore if NAS needs real data, we construct three types of
unreal datasets using: 1) randomly labeled real images; 2) generated images and
labels; and 3) generated Gaussian noise with random labels. These datasets
facilitate to analyze the generalization and expressivity of the searched
architectures. We study the performance of architectures searched on these
constructed datasets using popular differentiable NAS methods. Extensive
experiments on CIFAR, ImageNet and CheXpert show that the searched
architectures can achieve promising results compared with those derived from
the conventional NAS pipeline with real labeled data, suggesting the
feasibility of performing NAS with unreal data.
