  Many image processing tasks can be formulated as translating images between
two image domains, such as colorization, super resolution and conditional image
synthesis. In most of these tasks, an input image may correspond to multiple
outputs. However, current existing approaches only show very minor diversity of
the outputs. In this paper, we present a novel approach to synthesize diverse
realistic images corresponding to a semantic layout. We introduce a diversity
loss objective, which maximizes the distance between synthesized image pairs
and links the input noise to the semantic segments in the synthesized images.
Thus, our approach can not only produce diverse images, but also allow users to
manipulate the output images by adjusting the noise manually. Experimental
results show that images synthesized by our approach are significantly more
diverse than that of the current existing works and equipping our diversity
loss does not degrade the reality of the base networks.
