  We investigate statistical properties of a likelihood approach to
nonparametric estimation of a singular distribution using deep generative
models. More specifically, a deep generative model is used to model
high-dimensional data that are assumed to concentrate around some
low-dimensional structure. Estimating the distribution supported on this
low-dimensional structure, such as a low-dimensional manifold, is challenging
due to its singularity with respect to the Lebesgue measure in the ambient
space. In the considered model, a usual likelihood approach can fail to
estimate the target distribution consistently due to the singularity. We prove
that a novel and effective solution exists by perturbing the data with an
instance noise, which leads to consistent estimation of the underlying
distribution with desirable convergence rates. We also characterize the class
of distributions that can be efficiently estimated via deep generative models.
This class is sufficiently general to contain various structured distributions
such as product distributions, classically smooth distributions and
distributions supported on a low-dimensional manifold. Our analysis provides
some insights on how deep generative models can avoid the curse of
dimensionality for nonparametric distribution estimation. We conduct a thorough
simulation study and real data analysis to empirically demonstrate that the
proposed data perturbation technique improves the estimation performance
significantly.
