  Generative models can be categorized into two types: explicit generative
models that define explicit density forms and allow exact likelihood inference,
such as score-based diffusion models (SDMs) and normalizing flows; implicit
generative models that directly learn a transformation from the prior to the
data distribution, such as generative adversarial nets (GANs). While these two
types of models have shown great success, they suffer from respective
limitations that hinder them from achieving fast sampling and high sample
quality simultaneously. In this paper, we propose a unified theoretic framework
for SDMs and GANs. We shown that: i) the learning dynamics of both SDMs and
GANs can be described as a novel SDE named Discriminator Denoising Diffusion
Flow (DiffFlow) where the drift can be determined by some weighted combinations
of scores of the real data and the generated data; ii) By adjusting the
relative weights between different score terms, we can obtain a smooth
transition between SDMs and GANs while the marginal distribution of the SDE
remains invariant to the change of the weights; iii) we prove the asymptotic
optimality and maximal likelihood training scheme of the DiffFlow dynamics; iv)
under our unified theoretic framework, we introduce several instantiations of
the DiffFLow that provide new algorithms beyond GANs and SDMs with exact
likelihood inference and have potential to achieve flexible trade-off between
high sample quality and fast sampling speed.
