  Despite recent improvements in open-domain dialogue models, state of the art
models are trained and evaluated on short conversations with little context. In
contrast, the long-term conversation setting has hardly been studied. In this
work we collect and release a human-human dataset consisting of multiple chat
sessions whereby the speaking partners learn about each other's interests and
discuss the things they have learnt from past sessions. We show how existing
models trained on existing datasets perform poorly in this long-term
conversation setting in both automatic and human evaluations, and we study
long-context models that can perform much better. In particular, we find
retrieval-augmented methods and methods with an ability to summarize and recall
previous conversations outperform the standard encoder-decoder architectures
currently considered state of the art.
