  Recently, text-guided image manipulation has received increasing attention in
the research field of multimedia processing and computer vision due to its high
flexibility and controllability. Its goal is to semantically manipulate parts
of an input reference image according to the text descriptions. However, most
of the existing works have the following problems: (1) text-irrelevant content
cannot always be maintained but randomly changed, (2) the performance of image
manipulation still needs to be further improved, (3) only can manipulate
descriptive attributes. To solve these problems, we propose a novel image
manipulation method that interactively edits an image using complex text
instructions. It allows users to not only improve the accuracy of image
manipulation but also achieve complex tasks such as enlarging, dwindling, or
removing objects and replacing the background with the input image. To make
these tasks possible, we apply three strategies. First, the given image is
divided into text-relevant content and text-irrelevant content. Only the
text-relevant content is manipulated and the text-irrelevant content can be
maintained. Second, a super-resolution method is used to enlarge the
manipulation region to further improve the operability and to help manipulate
the object itself. Third, a user interface is introduced for editing the
segmentation map interactively to re-modify the generated image according to
the user's desires. Extensive experiments on the Caltech-UCSD Birds-200-2011
(CUB) dataset and Microsoft Common Objects in Context (MS COCO) datasets
demonstrate our proposed method can enable interactive, flexible, and accurate
image manipulation in real-time. Through qualitative and quantitative
evaluations, we show that the proposed model outperforms other state-of-the-art
methods.
