  The empirical results suggest that the learnability of a neural network is
directly related to its size. To mathematically prove this, we borrow a tool in
topological algebra: Betti numbers to measure the topological geometric
complexity of input data and the neural network. By characterizing the
expressive capacity of a neural network with its topological complexity, we
conduct a thorough analysis and show that the network's expressive capacity is
limited by the scale of its layers. Further, we derive the upper bounds of the
Betti numbers on each layer within the network. As a result, the problem of
architecture selection of a neural network is transformed to determining the
scale of the network that can represent the input data complexity. With the
presented results, the architecture selection of a fully connected network
boils down to choosing a suitable size of the network such that it equips the
Betti numbers that are not smaller than the Betti numbers of the input data. We
perform the experiments on a real-world dataset MNIST and the results verify
our analysis and conclusion. The code is publicly available.
