  In numerous robotics and mechanical engineering applications, among others,
data is often constrained on smooth manifolds due to the presence of rotational
degrees of freedom. Common datadriven and learning-based methods such as neural
ordinary differential equations (ODEs), however, typically fail to satisfy
these manifold constraints and perform poorly for these applications. To
address this shortcoming, in this paper we study a class of neural ordinary
differential equations that, by design, leave a given manifold invariant, and
characterize their properties by leveraging the controllability properties of
control affine systems. In particular, using a result due to Agrachev and
Caponigro on approximating diffeomorphisms with flows of feedback control
systems, we show that any map that can be represented as the flow of a
manifold-constrained dynamical system can also be approximated using the flow
of manifold-constrained neural ODE, whenever a certain controllability
condition is satisfied. Additionally, we show that this universal approximation
property holds when the neural ODE has limited width in each layer, thus
leveraging the depth of network instead for approximation. We verify our
theoretical findings using numerical experiments on PyTorch for the manifolds
S2 and the 3-dimensional orthogonal group SO(3), which are model manifolds for
mechanical systems such as spacecrafts and satellites. We also compare the
performance of the manifold invariant neural ODE with classical neural ODEs
that ignore the manifold invariant properties and show the superiority of our
approach in terms of accuracy and sample complexity.
