  Neural Architecture Search (NAS) enabled the discovery of state-of-the-art
architectures in many domains. However, the success of NAS depends on the
definition of the search space. Current search spaces are defined as a static
sequence of decisions and a set of available actions for each decision. Each
possible sequence of actions defines an architecture. We propose a more
expressive class of search space: directed graphs. In our formalism, each
decision is a vertex and each action is an edge. This allows us to model
iterative and branching architecture design decisions. We demonstrate in
simulation, and on image classification experiments, basic iterative and
branching search structures, and show that the graph representation improves
sample efficiency.
