  The use of artificial neural networks as models of chaotic dynamics has been
rapidly expanding. Still, a theoretical understanding of how neural networks
learn chaos is lacking. Here, we employ a geometric perspective to show that
neural networks can efficiently model chaotic dynamics by becoming structurally
chaotic themselves. We first confirm neural network's efficiency in emulating
chaos by showing that a parsimonious neural network trained only on few data
points can reconstruct strange attractors, extrapolate outside training data
boundaries, and accurately predict local divergence rates. We then posit that
the trained network's map comprises sequential geometric stretching, rotation,
and compression operations. These geometric operations indicate topological
mixing and chaos, explaining why neural networks are naturally suitable to
emulate chaotic dynamics.
