  Conditional image generation is an active research topic including text2image
and image translation.
  Recently image manipulation with linguistic instruction brings new challenges
of multimodal conditional generation.
  However, traditional conditional image generation models mainly focus on
generating high-quality and visually realistic images, and lack resolving the
partial consistency between image and instruction.
  To address this issue, we propose an Increment Reasoning Generative
Adversarial Network (IR-GAN), which aims to reason the consistency between
visual increment in images and semantic increment in instructions.
  First, we introduce the word-level and instruction-level instruction encoders
to learn user's intention from history-correlated instructions as semantic
increment.
  Second, we embed the representation of semantic increment into that of source
image for generating target image, where source image plays the role of
referring auxiliary.
  Finally, we propose a reasoning discriminator to measure the consistency
between visual increment and semantic increment, which purifies user's
intention and guarantees the good logic of generated target image.
  Extensive experiments and visualization conducted on two datasets show the
effectiveness of IR-GAN.
