  Continuous-depth learning has recently emerged as a novel perspective on deep
learning, improving performance in tasks related to dynamical systems and
density estimation. Core to these approaches is the neural differential
equation, whose forward passes are the solutions of an initial value problem
parametrized by a neural network. Unlocking the full potential of
continuous-depth models requires a different set of software tools, due to
peculiar differences compared to standard discrete neural networks, e.g
inference must be carried out via numerical solvers. We introduce TorchDyn, a
PyTorch library dedicated to continuous-depth learning, designed to elevate
neural differential equations to be as accessible as regular plug-and-play deep
learning primitives. This objective is achieved by identifying and subdividing
different variants into common essential components, which can be combined and
freely repurposed to obtain complex compositional architectures. TorchDyn
further offers step-by-step tutorials and benchmarks designed to guide
researchers and contributors.
