  This work introduces the concept of tangent space regularization for
neural-network models of dynamical systems. The tangent space to the dynamics
function of many physical systems of interest in control applications exhibits
useful properties, e.g., smoothness, motivating regularization of the model
Jacobian along system trajectories using assumptions on the tangent space of
the dynamics. Without assumptions, large amounts of training data are required
for a neural network to learn the full non-linear dynamics without overfitting.
We compare different network architectures on one-step prediction and
simulation performance and investigate the propensity of different
architectures to learn models with correct input-output Jacobian. Furthermore,
the influence of $L_2$ weight regularization on the learned Jacobian eigenvalue
spectrum, and hence system stability, is investigated.
