  The goal of this paper is to embed controllable factors, i.e., natural
language descriptions, into image-to-image translation with generative
adversarial networks, which allows text descriptions to determine the visual
attributes of synthetic images. We propose four key components: (1) the
implementation of part-of-speech tagging to filter out non-semantic words in
the given description, (2) the adoption of an affine combination module to
effectively fuse different modality text and image features, (3) a novel
refined multi-stage architecture to strengthen the differential ability of
discriminators and the rectification ability of generators, and (4) a new
structure loss to further improve discriminators to better distinguish real and
synthetic images. Extensive experiments on the COCO dataset demonstrate that
our method has a superior performance on both visual realism and semantic
consistency with given descriptions.
