  Training dynamic models, such as neural ODEs, on long trajectories is a hard
problem that requires using various tricks, such as trajectory splitting, to
make model training work in practice. These methods are often heuristics with
poor theoretical justifications, and require iterative manual tuning. We
propose a principled multiple shooting technique for neural ODEs that splits
the trajectories into manageable short segments, which are optimised in
parallel, while ensuring probabilistic control on continuity over consecutive
segments. We derive variational inference for our shooting-based latent neural
ODE models and propose amortized encodings of irregularly sampled trajectories
with a transformer-based recognition network with temporal attention and
relative positional encoding. We demonstrate efficient and stable training, and
state-of-the-art performance on multiple large-scale benchmark datasets.
