  The popularity of image sharing on social media and the engagement it creates
between users reflects the important role that visual context plays in everyday
conversations. We present a novel task, Image-Grounded Conversations (IGC), in
which natural-sounding conversations are generated about a shared image. To
benchmark progress, we introduce a new multiple-reference dataset of
crowd-sourced, event-centric conversations on images. IGC falls on the
continuum between chit-chat and goal-directed conversation models, where visual
grounding constrains the topic of conversation to event-driven utterances.
Experiments with models trained on social media data show that the combination
of visual and textual context enhances the quality of generated conversational
turns. In human evaluation, the gap between human performance and that of both
neural and retrieval architectures suggests that multi-modal IGC presents an
interesting challenge for dialogue research.
