  Ordinary differential equations (ODEs), commonly used to characterize the
dynamic systems, are difficult to propose in closed-form for many complicated
scientific applications, even with the help of domain expert. We propose a fast
and accurate data-driven method, MAGI-X, to learn the unknown dynamic from the
observation data in a non-parametric fashion, without the need of any domain
knowledge. Unlike the existing methods that mainly rely on the costly numerical
integration, MAGI-X utilizes the powerful functional approximator of neural
network to learn the unknown nonlinear dynamic within the MAnifold-constrained
Gaussian process Inference (MAGI) framework that completely circumvents the
numerical integration. Comparing against the state-of-the-art methods on three
realistic examples, MAGI-X achieves competitive accuracy in both fitting and
forecasting while only taking a fraction of computational time. Moreover,
MAGI-X provides practical solution for the inference of partial observed
systems, which no previous method is able to handle.
