  We detail a novel class of implicit neural models. Leveraging time-parallel
methods for differential equations, Multiple Shooting Layers (MSLs) seek
solutions of initial value problems via parallelizable root-finding algorithms.
MSLs broadly serve as drop-in replacements for neural ordinary differential
equations (Neural ODEs) with improved efficiency in number of function
evaluations (NFEs) and wall-clock inference time. We develop the algorithmic
framework of MSLs, analyzing the different choices of solution methods from a
theoretical and computational perspective. MSLs are showcased in long horizon
optimal control of ODEs and PDEs and as latent models for sequence generation.
Finally, we investigate the speedups obtained through application of MSL
inference in neural controlled differential equations (Neural CDEs) for time
series classification of medical data.
