  Systems that perform image manipulation using deep convolutional networks
have achieved remarkable realism. Perceptual losses and losses based on
adversarial discriminators are the two main classes of learning objectives
behind these advances. In this work, we show how these two ideas can be
combined in a principled and non-additive manner for unaligned image
translation tasks. This is accomplished through a special architecture of the
discriminator network inside generative adversarial learning framework. The new
architecture, that we call a perceptual discriminator, embeds the convolutional
parts of a pre-trained deep classification network inside the discriminator
network. The resulting architecture can be trained on unaligned image datasets
while benefiting from the robustness and efficiency of perceptual losses. We
demonstrate the merits of the new architecture in a series of qualitative and
quantitative comparisons with baseline approaches and state-of-the-art
frameworks for unaligned image translation.
