  The Path-dependent Neural Jump ODE (PD-NJ-ODE) is a model for online
prediction of generic (possibly non-Markovian) stochastic processes with
irregular (in time) and potentially incomplete (with respect to coordinates)
observations. It is a model for which convergence to the $L^2$-optimal
predictor, which is given by the conditional expectation, is established
theoretically. Thereby, the training of the model is solely based on a dataset
of realizations of the underlying stochastic process, without the need of
knowledge of the law of the process. In the case where the underlying process
is deterministic, the conditional expectation coincides with the process
itself. Therefore, this framework can equivalently be used to learn the
dynamics of ODE or PDE systems solely from realizations of the dynamical system
with different initial conditions. We showcase the potential of our method by
applying it to the chaotic system of a double pendulum. When training the
standard PD-NJ-ODE method, we see that the prediction starts to diverge from
the true path after about half of the evaluation time. In this work we enhance
the model with two novel ideas, which independently of each other improve the
performance of our modelling setup. The resulting dynamics match the true
dynamics of the chaotic system very closely. The same enhancements can be used
to provably enable the PD-NJ-ODE to learn long-term predictions for general
stochastic datasets, where the standard model fails. This is verified in
several experiments.
