  Due to the nonlinearity of artificial neural networks, designing topologies
for deep convolutional neural networks (CNN) is a challenging task and often
only heuristic approach, such as trial and error, can be applied. An
evolutionary algorithm can solve optimization problems where the fitness
landscape is unknown. However, evolutionary algorithms are computing resource
intensive, which makes it difficult for problems when deep CNNs are involved.
In this paper, we propose an evolutionary strategy to find better topologies
for deep CNNs. Incorporating the concept of knowledge inheritance and knowledge
learning, our evolutionary algorithm can be executed with limited computing
resources. We applied the proposed algorithm in finding effective topologies of
deep CNNs for the image classification task using CIFAR-10 dataset. After the
evolution, we analyzed the topologies that performed well for this task. Our
studies verify the techniques that have been commonly used in human designed
deep CNNs. We also discovered that some of the graph properties greatly affect
the system performance. We applied the guidelines learned from the evolution
and designed new network topologies that outperform Residual Net with less
layers on CIFAR-10, CIFAR-100, and SVHN dataset.
