  One-shot Neural Architecture Search (NAS) aims to minimize the computational
expense of discovering state-of-the-art models. However, in the past year
attention has been drawn to the comparable performance of naive random search
across the same search spaces used by leading NAS algorithms. To address this,
we explore the effects of drastically relaxing the NAS search space, and we
present Bonsai-Net, an efficient one-shot NAS method to explore our relaxed
search space. Bonsai-Net is built around a modified differential pruner and can
consistently discover state-of-the-art architectures that are significantly
better than random search with fewer parameters than other state-of-the-art
methods. Additionally, Bonsai-Net performs simultaneous model search and
training, dramatically reducing the total time it takes to generate
fully-trained models from scratch.
