  Evolutionary computation methods have been successfully applied to neural
networks since two decades ago, while those methods cannot scale well to the
modern deep neural networks due to the complicated architectures and large
quantities of connection weights. In this paper, we propose a new method using
genetic algorithms for evolving the architectures and connection weight
initialization values of a deep convolutional neural network to address image
classification problems. In the proposed algorithm, an efficient
variable-length gene encoding strategy is designed to represent the different
building blocks and the unpredictable optimal depth in convolutional neural
networks. In addition, a new representation scheme is developed for effectively
initializing connection weights of deep convolutional neural networks, which is
expected to avoid networks getting stuck into local minima which is typically a
major issue in the backward gradient-based optimization. Furthermore, a novel
fitness evaluation method is proposed to speed up the heuristic search with
substantially less computational resource. The proposed algorithm is examined
and compared with 22 existing algorithms on nine widely used image
classification tasks, including the state-of-the-art methods. The experimental
results demonstrate the remarkable superiority of the proposed algorithm over
the state-of-the-art algorithms in terms of classification error rate and the
number of parameters (weights).
