  We introduce a memory-driven semi-parametric approach to text-to-image
generation, which is based on both parametric and non-parametric techniques.
The non-parametric component is a memory bank of image features constructed
from a training set of images. The parametric component is a generative
adversarial network. Given a new text description at inference time, the memory
bank is used to selectively retrieve image features that are provided as basic
information of target images, which enables the generator to produce realistic
synthetic results. We also incorporate the content information into the
discriminator, together with semantic features, allowing the discriminator to
make a more reliable prediction. Experimental results demonstrate that the
proposed memory-driven semi-parametric approach produces more realistic images
than purely parametric approaches, in terms of both visual fidelity and
text-image semantic consistency.
