  Continuous Normalizing Flows (CNFs) are a class of generative models that
transform a prior distribution to a model distribution by solving an ordinary
differential equation (ODE). We propose to train CNFs on manifolds by
minimizing probability path divergence (PPD), a novel family of divergences
between the probability density path generated by the CNF and a target
probability density path. PPD is formulated using a logarithmic mass
conservation formula which is a linear first order partial differential
equation relating the log target probabilities and the CNF's defining vector
field. PPD has several key benefits over existing methods: it sidesteps the
need to solve an ODE per iteration, readily applies to manifold data, scales to
high dimensions, and is compatible with a large family of target paths
interpolating pure noise and data in finite time. Theoretically, PPD is shown
to bound classical probability divergences. Empirically, we show that CNFs
learned by minimizing PPD achieve state-of-the-art results in likelihoods and
sample quality on existing low-dimensional manifold benchmarks, and is the
first example of a generative model to scale to moderately high dimensional
manifolds.
