  The growing interest in both the automation of machine learning and deep
learning has inevitably led to the development of a wide variety of automated
methods for neural architecture search. The choice of the network architecture
has proven to be critical, and many advances in deep learning spring from its
immediate improvements. However, deep learning techniques are computationally
intensive and their application requires a high level of domain knowledge.
Therefore, even partial automation of this process helps to make deep learning
more accessible to both researchers and practitioners. With this survey, we
provide a formalism which unifies and categorizes the landscape of existing
methods along with a detailed analysis that compares and contrasts the
different approaches. We achieve this via a comprehensive discussion of the
commonly adopted architecture search spaces and architecture optimization
algorithms based on principles of reinforcement learning and evolutionary
algorithms along with approaches that incorporate surrogate and one-shot
models. Additionally, we address the new research directions which include
constrained and multi-objective architecture search as well as automated data
augmentation, optimizer and activation function search.
