  Online conversations include more than just text. Increasingly, image-based
responses such as memes and animated gifs serve as culturally recognized and
often humorous responses in conversation. However, while NLP has broadened to
multimodal models, conversational dialog systems have largely focused only on
generating text replies. Here, we introduce a new dataset of 1.56M text-gif
conversation turns and introduce a new multimodal conversational model Pepe the
King Prawn for selecting gif-based replies. We demonstrate that our model
produces relevant and high-quality gif responses and, in a large randomized
control trial of multiple models replying to real users, we show that our model
replies with gifs that are significantly better received by the community.
