  This paper examines three generic strategies for improving the performance of
neuro-evolution techniques aimed at evolving convolutional neural networks
(CNNs). These were implemented as part of the Evolutionary eXploration of
Augmenting Convolutional Topologies (EXACT) algorithm. EXACT evolves arbitrary
convolutional neural networks (CNNs) with goals of better discovering and
understanding new effective architectures of CNNs for machine learning tasks
and to potentially automate the process of network design and selection. The
strategies examined are node-level mutation operations, epigenetic weight
initialization and pooling connections. Results were gathered over the period
of a month using a volunteer computing project, where over 225,000 CNNs were
trained and evaluated across 16 different EXACT searches. The node mutation
operations where shown to dramatically improve evolution rates over traditional
edge mutation operations (as used by the NEAT algorithm), and epigenetic weight
initialization was shown to further increase the accuracy and generalizability
of the trained CNNs. As a negative but interesting result, allowing for pooling
connections was shown to degrade the evolution progress. The best trained CNNs
reached 99.46% accuracy on the MNIST test data in under 13,500 CNN evaluations
-- accuracy comparable with some of the best human designed CNNs.
