  Dialogue participants often refer to entities or situations repeatedly within
a conversation, which contributes to its cohesiveness. Subsequent references
exploit the common ground accumulated by the interlocutors and hence have
several interesting properties, namely, they tend to be shorter and reuse
expressions that were effective in previous mentions. In this paper, we tackle
the generation of first and subsequent references in visually grounded
dialogue. We propose a generation model that produces referring utterances
grounded in both the visual and the conversational context. To assess the
referring effectiveness of its output, we also implement a reference resolution
system. Our experiments and analyses show that the model produces better, more
effective referring utterances than a model not grounded in the dialogue
context, and generates subsequent references that exhibit linguistic patterns
akin to humans.
