  Image memes have become a widespread tool used by people for interacting and
exchanging ideas over social media, blogs, and open messengers. This work
proposes to treat automatic image meme generation as a translation process, and
further present an end to end neural and probabilistic approach to generate an
image-based meme for any given sentence using an encoder-decoder architecture.
For a given input sentence, an image meme is generated by combining a meme
template image and a text caption where the meme template image is selected
from a set of popular candidates using a selection module, and the meme caption
is generated by an encoder-decoder model. An encoder is used to map the
selected meme template and the input sentence into a meme embedding and a
decoder is used to decode the meme caption from the meme embedding. The
generated natural language meme caption is conditioned on the input sentence
and the selected meme template. The model learns the dependencies between the
meme captions and the meme template images and generates new memes using the
learned dependencies. The quality of the generated captions and the generated
memes is evaluated through both automated and human evaluation. An experiment
is designed to score how well the generated memes can represent the tweets from
Twitter conversations. Experiments on Twitter data show the efficacy of the
model in generating memes for sentences in online social interaction.
