  Flow-based models typically define a latent space with dimensionality
identical to the observational space. In many problems, however, the data does
not populate the full ambient data space that they natively reside in, rather
inhabiting a lower-dimensional manifold. In such scenarios, flow-based models
are unable to represent data structures exactly as their densities will always
have support off the data manifold, potentially resulting in degradation of
model performance. To address this issue, we propose to learn a manifold prior
for flow models that leverage the recently proposed spread divergence towards
fixing the crucial problem; the KL divergence and maximum likelihood estimation
are ill-defined for manifold learning. In addition to improving both sample
quality and representation quality, an auxiliary benefit enabled by our
approach is the ability to identify the intrinsic dimension of the manifold
distribution.
