  In recent years, text-guided image manipulation has gained increasing
attention in the multimedia and computer vision community. The input to
conditional image generation has evolved from image-only to multimodality. In
this paper, we study a setting that allows users to edit an image with multiple
objects using complex text instructions to add, remove, or change the objects.
The inputs of the task are multimodal including (1) a reference image and (2)
an instruction in natural language that describes desired modifications to the
image. We propose a GAN-based method to tackle this problem. The key idea is to
treat text as neural operators to locally modify the image feature. We show
that the proposed model performs favorably against recent strong baselines on
three public datasets. Specifically, it generates images of greater fidelity
and semantic relevance, and when used as a image query, leads to better
retrieval performance.
