  A large number of current machine learning methods rely upon deep neural
networks. Yet, viewing neural networks as nonlinear dynamical systems, it
becomes quickly apparent that mathematically rigorously establishing certain
patterns generated by the nodes in the network is extremely difficult. Indeed,
it is well-understood in the nonlinear dynamics of complex systems that, even
in low-dimensional models, analytical techniques rooted in pencil-and-paper
approaches frequently reach their limits. In this work, we propose a completely
different perspective via the paradigm of validated numerical methods of
nonlinear dynamics. The idea is to use computer-assisted proofs to validate
mathematically the existence of nonlinear patterns in neural networks. As a
case study, we consider a class of recurrent neural networks, where we prove
via computer assistance the existence of several hundred Hopf bifurcation
points, their non-degeneracy, and hence also the existence of several hundred
periodic orbits. Our paradigm has the capability to rigorously verify complex
nonlinear behaviour of neural networks, which provides a first step to explain
the full abilities, as well as potential sensitivities, of machine learning
methods via computer-assisted proofs. We showcase how validated numerical
techniques can shed light on the internal working of recurrent neural networks
(RNNs). For this, proofs of Hopf bifurcations are a first step towards an
integration of dynamical system theory in practical application of RNNs, by
proving the existence of periodic orbits in a variety of settings.
