  Image to image translation is the problem of transferring an image from a
source domain to a different (but related) target domain. We present a new
unsupervised image to image translation technique that leverages the underlying
semantic information for object transfiguration and domain transfer tasks.
Specifically, we present a generative adversarial learning approach that
jointly translates images and labels from a source domain to a target domain.
Our main technical contribution is an encoder-decoder based network
architecture that jointly encodes the image and its underlying semantics and
translates both individually to the target domain. Additionally, we propose
object transfiguration and cross-domain semantic consistency losses that
preserve semantic labels. Through extensive experimental evaluation, we
demonstrate the effectiveness of our approach as compared to the
state-of-the-art methods on unsupervised image-to-image translation, domain
adaptation, and object transfiguration.
