  In this study, we explore the impact of network topology on the approximation
capabilities of artificial neural networks (ANNs), with a particular focus on
complex topologies. We propose a novel methodology for constructing complex
ANNs based on various topologies, including Barab\'asi-Albert,
Erd\H{o}s-R\'enyi, Watts-Strogatz, and multilayer perceptrons (MLPs). The
constructed networks are evaluated on synthetic datasets generated from
manifold learning generators, with varying levels of task difficulty and noise,
and on real-world datasets from the UCI suite. Our findings reveal that complex
topologies lead to superior performance in high-difficulty regimes compared to
traditional MLPs. This performance advantage is attributed to the ability of
complex networks to exploit the compositionality of the underlying target
function. However, this benefit comes at the cost of increased forward-pass
computation time and reduced robustness to graph damage. Additionally, we
investigate the relationship between various topological attributes and model
performance. Our analysis shows that no single attribute can account for the
observed performance differences, suggesting that the influence of network
topology on approximation capabilities may be more intricate than a simple
correlation with individual topological attributes. Our study sheds light on
the potential of complex topologies for enhancing the performance of ANNs and
provides a foundation for future research exploring the interplay between
multiple topological attributes and their impact on model performance.
