  Arguably, the visual perception of conversational agents to the physical
world is a key way for them to exhibit the human-like intelligence.
Image-grounded conversation is thus proposed to address this challenge.
Existing works focus on exploring the multimodal dialog models that ground the
conversation on a given image. In this paper, we take a step further to study
image-grounded conversation under a fully open-ended setting where no paired
dialog and image are assumed available. Specifically, we present Maria, a
neural conversation agent powered by the visual world experiences which are
retrieved from a large-scale image index. Maria consists of three flexible
components, i.e., text-to-image retriever, visual concept detector and
visual-knowledge-grounded response generator. The retriever aims to retrieve a
correlated image to the dialog from an image index, while the visual concept
detector extracts rich visual knowledge from the image. Then, the response
generator is grounded on the extracted visual knowledge and dialog context to
generate the target response. Extensive experiments demonstrate Maria
outperforms previous state-of-the-art methods on automatic metrics and human
evaluation, and can generate informative responses that have some visual
commonsense of the physical world.
