  We propose a principled method for gradient-based regularization of the
critic of GAN-like models trained by adversarially optimizing the kernel of a
Maximum Mean Discrepancy (MMD). We show that controlling the gradient of the
critic is vital to having a sensible loss function, and devise a method to
enforce exact, analytical gradient constraints at no additional cost compared
to existing approximate techniques based on additive regularizers. The new loss
function is provably continuous, and experiments show that it stabilizes and
accelerates training, giving image generation models that outperform
state-of-the art methods on $160 \times 160$ CelebA and $64 \times 64$
unconditional ImageNet.
