  In past years, the minimax type single-level optimization formulation and its
variations have been widely utilized to address Generative Adversarial Networks
(GANs). Unfortunately, it has been proved that these alternating learning
strategies cannot exactly reveal the intrinsic relationship between the
generator and discriminator, thus easily result in a series of issues,
including mode collapse, vanishing gradients and oscillations in the training
phase, etc. In this work, by investigating the fundamental mechanism of GANs
from the perspective of hierarchical optimization, we propose Best-Response
Constraint (BRC), a general learning framework, that can explicitly formulate
the potential dependency of the generator on the discriminator. Rather than
adopting these existing time-consuming bilevel iterations, we design an
implicit gradient scheme with outer-product Hessian approximation as our fast
solution strategy. \emph{Noteworthy, we demonstrate that even with different
motivations and formulations, a variety of existing GANs ALL can be uniformly
improved by our flexible BRC methodology.} Extensive quantitative and
qualitative experimental results verify the effectiveness, flexibility and
stability of our proposed framework.
