  In recent years, neural architecture search (NAS) has received intensive
scientific and industrial interest due to its capability of finding a neural
architecture with high accuracy for various artificial intelligence tasks such
as image classification or object detection. In particular, gradient-based NAS
approaches have become one of the more popular approaches thanks to their
computational efficiency during the search. However, these methods often
experience a mode collapse, where the quality of the found architectures is
poor due to the algorithm resorting to choosing a single operation type for the
entire network, or stagnating at a local minima for various datasets or search
spaces.
  To address these defects, we present a differentiable variational
inference-based NAS method for searching sparse convolutional neural networks.
Our approach finds the optimal neural architecture by dropping out candidate
operations in an over-parameterised supergraph using variational dropout with
automatic relevance determination prior, which makes the algorithm gradually
remove unnecessary operations and connections without risking mode collapse.
The evaluation is conducted through searching two types of convolutional cells
that shape the neural network for classifying different image datasets. Our
method finds diverse network cells, while showing state-of-the-art accuracy
with up to almost 2 times fewer non-zero parameters.
