  Neural ordinary differential equations (ODEs) are an emerging class of deep
learning models for dynamical systems. They are particularly useful for
learning an ODE vector field from observed trajectories (i.e., inverse
problems). We here consider aspects of these models relevant for their
application in science and engineering. Scientific predictions generally
require structured uncertainty estimates. As a first contribution, we show that
basic and lightweight Bayesian deep learning techniques like the Laplace
approximation can be applied to neural ODEs to yield structured and meaningful
uncertainty quantification. But, in the scientific domain, available
information often goes beyond raw trajectories, and also includes mechanistic
knowledge, e.g., in the form of conservation laws. We explore how mechanistic
knowledge and uncertainty quantification interact on two recently proposed
neural ODE frameworks - symplectic neural ODEs and physical models augmented
with neural ODEs. In particular, uncertainty reflects the effect of mechanistic
information more directly than the predictive power of the trained model could.
And vice versa, structure can improve the extrapolation abilities of neural
ODEs, a fact that can be best assessed in practice through uncertainty
estimates. Our experimental analysis demonstrates the effectiveness of the
Laplace approach on both low dimensional ODE problems and a high dimensional
partial differential equation.
