  State-of-the-art image-to-image translation methods tend to struggle in an
imbalanced domain setting, where one image domain lacks richness and diversity.
We introduce a new unsupervised translation network, BalaGAN, specifically
designed to tackle the domain imbalance problem. We leverage the latent
modalities of the richer domain to turn the image-to-image translation problem,
between two imbalanced domains, into a balanced, multi-class, and conditional
translation problem, more resembling the style transfer setting. Specifically,
we analyze the source domain and learn a decomposition of it into a set of
latent modes or classes, without any supervision. This leaves us with a
multitude of balanced cross-domain translation tasks, between all pairs of
classes, including the target domain. During inference, the trained network
takes as input a source image, as well as a reference or style image from one
of the modes as a condition, and produces an image which resembles the source
on the pixel-wise level, but shares the same mode as the reference. We show
that employing modalities within the dataset improves the quality of the
translated images, and that BalaGAN outperforms strong baselines of both
unconditioned and style-transfer-based image-to-image translation methods, in
terms of image quality and diversity.
