  Evolving Neural Networks (NNs) has recently seen an increasing interest as an
alternative path that might be more successful. It has many advantages compared
to other approaches, such as learning the architecture of the NNs. However, the
extremely large search space and the existence of many complex interacting
parts still represent a major obstacle. Many criteria were recently
investigated to help guide the algorithm and to cut down the large search
space. Recently there has been growing research bringing insights from network
science to improve the design of NNs. In this paper, we investigate evolving
NNs architectures that have one of the most fundamental characteristics of
real-world networks, namely the optimal balance between connections cost and
information flow. The performance of different metrics that represent this
balance is evaluated and the improvement in the accuracy of putting more
selection pressure toward this balance is demonstrated on three datasets.
