  We develop a framework for comparing data manifolds, aimed, in particular,
towards the evaluation of deep generative models. We describe a novel tool,
Cross-Barcode(P,Q), that, given a pair of distributions in a high-dimensional
space, tracks multiscale topology spacial discrepancies between manifolds on
which the distributions are concentrated. Based on the Cross-Barcode, we
introduce the Manifold Topology Divergence score (MTop-Divergence) and apply it
to assess the performance of deep generative models in various domains: images,
3D-shapes, time-series, and on different datasets: MNIST, Fashion MNIST, SVHN,
CIFAR10, FFHQ, chest X-ray images, market stock data, ShapeNet. We demonstrate
that the MTop-Divergence accurately detects various degrees of mode-dropping,
intra-mode collapse, mode invention, and image disturbance. Our algorithm
scales well (essentially linearly) with the increase of the dimension of the
ambient high-dimensional space. It is one of the first TDA-based practical
methodologies that can be applied universally to datasets of different sizes
and dimensions, including the ones on which the most recent GANs in the visual
domain are trained. The proposed method is domain agnostic and does not rely on
pre-trained networks.
