  Existing models for unsupervised image translation with Generative
Adversarial Networks (GANs) can learn the mapping from the source domain to the
target domain using a cycle-consistency loss. However, these methods always
adopt a symmetric network architecture to learn both forward and backward
cycles. Because of the task complexity and cycle input difference between the
source and target domains, the inequality in bidirectional forward-backward
cycle translations is significant and the amount of information between two
domains is different. In this paper, we analyze the limitation of existing
symmetric GANs in asymmetric translation tasks, and propose an AsymmetricGAN
model with both translation and reconstruction generators of unequal sizes and
different parameter-sharing strategy to adapt to the asymmetric need in both
unsupervised and supervised image translation tasks. Moreover, the training
stage of existing methods has the common problem of model collapse that
degrades the quality of the generated images, thus we explore different
optimization losses for better training of AsymmetricGAN, making image
translation with higher consistency and better stability. Extensive experiments
on both supervised and unsupervised generative tasks with 8 datasets show that
AsymmetricGAN achieves superior model capacity and better generation
performance compared with existing GANs. To the best of our knowledge, we are
the first to investigate the asymmetric GAN structure on both unsupervised and
supervised image translation tasks.
