  Recently, manipulation of real-world images has been highly elaborated along
with the development of Generative Adversarial Networks (GANs) and
corresponding encoders, which embed real-world images into the latent space.
However, designing encoders of GAN still remains a challenging task due to the
trade-off between distortion and perception. In this paper, we point out that
the existing encoders try to lower the distortion not only on the interest
region, e.g., human facial region but also on the uninterest region, e.g.,
background patterns and obstacles. However, most uninterest regions in
real-world images are located at out-of-distribution (OOD), which are
infeasible to be ideally reconstructed by generative models. Moreover, we
empirically find that the uninterest region overlapped with the interest region
can mangle the original feature of the interest region, e.g., a microphone
overlapped with a facial region is inverted into the white beard. As a result,
lowering the distortion of the whole image while maintaining the perceptual
quality is very challenging. To overcome this trade-off, we propose a simple
yet effective encoder training scheme, coined IntereStyle, which facilitates
encoding by focusing on the interest region. IntereStyle steers the encoder to
disentangle the encodings of the interest and uninterest regions. To this end,
we filter the information of the uninterest region iteratively to regulate the
negative impact of the uninterest region. We demonstrate that IntereStyle
achieves both lower distortion and higher perceptual quality compared to the
existing state-of-the-art encoders. Especially, our model robustly conserves
features of the original images, which shows the robust image editing and style
mixing results. We will release our code with the pre-trained model after the
review.
