  Due to their high computational efficiency on a continuous space, gradient
optimization methods have shown great potential in the neural architecture
search (NAS) domain. The mapping of network representation from the discrete
space to a latent space is the key to discovering novel architectures, however,
existing gradient-based methods fail to fully characterize the networks. In
this paper, we propose an efficient NAS approach to optimize network
architectures in a continuous space, where the latent space is built upon
variational autoencoder (VAE) and graph neural networks (GNN). The framework
jointly learns four components: the encoder, the performance predictor, the
complexity predictor and the decoder in an end-to-end manner. The encoder and
the decoder belong to a graph VAE, mapping architectures between continuous
representations and network architectures. The predictors are two regression
models, fitting the performance and computational complexity, respectively.
Those predictors ensure the discovered architectures characterize both
excellent performance and high computational efficiency. Extensive experiments
demonstrate our framework not only generates appropriate continuous
representations but also discovers powerful neural architectures.
