  In this paper, we propose an algorithmic framework to automatically generate
efficient deep neural networks and optimize their associated hyperparameters.
The framework is based on evolving directed acyclic graphs (DAGs), defining a
more flexible search space than the existing ones in the literature. It allows
mixtures of different classical operations: convolutions, recurrences and dense
layers, but also more newfangled operations such as self-attention. Based on
this search space we propose neighbourhood and evolution search operators to
optimize both the architecture and hyper-parameters of our networks. These
search operators can be used with any metaheuristic capable of handling mixed
search spaces. We tested our algorithmic framework with an evolutionary
algorithm on a time series prediction benchmark. The results demonstrate that
our framework was able to find models outperforming the established baseline on
numerous datasets.
