  Conditional image synthesis for generating photorealistic images serves
various applications for content editing to content generation. Previous
conditional image synthesis algorithms mostly rely on semantic maps, and often
fail in complex environments where multiple instances occlude each other. We
propose a panoptic aware image synthesis network to generate high fidelity and
photorealistic images conditioned on panoptic maps which unify semantic and
instance information. To achieve this, we efficiently use panoptic maps in
convolution and upsampling layers. We show that with the proposed changes to
the generator, we can improve on the previous state-of-the-art methods by
generating images in complex instance interaction environments in higher
fidelity and tiny objects in more details. Furthermore, our proposed method
also outperforms the previous state-of-the-art methods in metrics of mean IoU
(Intersection over Union), and detAP (Detection Average Precision).
