  To better conform to data geometry, recent deep generative modelling
techniques adapt Euclidean constructions to non-Euclidean spaces. In this
paper, we study normalizing flows on manifolds. Previous work has developed
flow models for specific cases; however, these advancements hand craft layers
on a manifold-by-manifold basis, restricting generality and inducing cumbersome
design constraints. We overcome these issues by introducing Neural Manifold
Ordinary Differential Equations, a manifold generalization of Neural ODEs,
which enables the construction of Manifold Continuous Normalizing Flows
(MCNFs). MCNFs require only local geometry (therefore generalizing to arbitrary
manifolds) and compute probabilities with continuous change of variables
(allowing for a simple and expressive flow construction). We find that
leveraging continuous manifold dynamics produces a marked improvement for both
density estimation and downstream tasks.
