  Due to the recent advances on Neural Architecture Search (NAS), it gains
popularity in designing best networks for specific tasks. Although it shows
promising results on many benchmarks and competitions, NAS still suffers from
its demanding computation cost for searching high dimensional architectural
design space, and this problem becomes even worse when we want to use a
large-scale dataset. If we can make a reliable data proxy for NAS, the
efficiency of NAS approaches increase accordingly. Our basic observation for
making a data proxy is that each example in a specific dataset has a different
impact on NAS process and most of examples are redundant from a relative
accuracy ranking perspective, which we should preserve when making a data
proxy. We propose a systematic approach to measure the importance of each
example from this relative accuracy ranking point of view, and make a reliable
data proxy based on the statistics of training and testing examples. Our
experiment shows that we can preserve the almost same relative accuracy ranking
between all possible network configurations even with 10-20$\times$ smaller
data proxy.
