  Neural Stochastic Differential Equations (NSDEs) model the drift and
diffusion functions of a stochastic process as neural networks. While NSDEs are
known to make accurate predictions, their uncertainty quantification properties
have been remained unexplored so far. We report the empirical finding that
obtaining well-calibrated uncertainty estimations from NSDEs is computationally
prohibitive. As a remedy, we develop a computationally affordable deterministic
scheme which accurately approximates the transition kernel, when dynamics is
governed by a NSDE. Our method introduces a bidimensional moment matching
algorithm: vertical along the neural net layers and horizontal along the time
direction, which benefits from an original combination of effective
approximations. Our deterministic approximation of the transition kernel is
applicable to both training and prediction. We observe in multiple experiments
that the uncertainty calibration quality of our method can be matched by Monte
Carlo sampling only after introducing high computational cost. Thanks to the
numerical stability of deterministic training, our method also improves
prediction accuracy.
