  Implicit generative models are difficult to train as no explicit density
functions are defined. Generative adversarial nets (GANs) present a minimax
framework to train such models, which however can suffer from mode collapse due
to the nature of the JS-divergence. This paper presents a learning by teaching
(LBT) approach to learning implicit models, which intrinsically avoids the mode
collapse problem by optimizing a KL-divergence rather than the JS-divergence in
GANs. In LBT, an auxiliary density estimator is introduced to fit the implicit
model's distribution while the implicit model teaches the density estimator to
match the data distribution. LBT is formulated as a bilevel optimization
problem, whose optimal generator matches the true data distribution. LBT can be
naturally integrated with GANs to derive a hybrid LBT-GAN that enjoys
complimentary benefits. Finally, we present a stochastic gradient ascent
algorithm with unrolling to solve the challenging learning problems.
Experimental results demonstrate the effectiveness of our method.
