  There is a large variety of machine learning methodologies that are based on
the extraction of spectral geometric information from data. However, the
implementations of many of these methods often depend on traditional
eigensolvers, which present limitations when applied in practical online big
data scenarios. To address some of these challenges, researchers have proposed
different strategies for training neural networks as alternatives to
traditional eigensolvers, with one such approach known as Spectral Neural
Network (SNN). In this paper, we investigate key theoretical aspects of SNN.
First, we present quantitative insights into the tradeoff between the number of
neurons and the amount of spectral geometric information a neural network
learns. Second, we initiate a theoretical exploration of the optimization
landscape of SNN's objective to shed light on the training dynamics of SNN.
Unlike typical studies of convergence to global solutions of NN training
dynamics, SNN presents an additional complexity due to its non-convex ambient
loss function.
