  Example-guided image synthesis has been recently attempted to synthesize an
image from a semantic label map and an exemplary image. In the task, the
additional exemplary image serves to provide style guidance that controls the
appearance of the synthesized output. Despite the controllability advantage,
the previous models are designed on datasets with specific and roughly aligned
objects. In this paper, we tackle a more challenging and general task, where
the exemplar is an arbitrary scene image that is semantically unaligned to the
given label map. To this end, we first propose a new Masked Spatial-Channel
Attention (MSCA) module which models the correspondence between two
unstructured scenes via cross-attention. Next, we propose an end-to-end network
for joint global and local feature alignment and synthesis. In addition, we
propose a novel patch-based self-supervision scheme to enable training.
Experiments on the large-scale CCOO-stuff dataset show significant improvements
over existing methods. Moreover, our approach provides interpretability and can
be readily extended to other tasks including style and spatial interpolation or
extrapolation, as well as other content manipulation.
