  With the evolution of pre-trained language models, current open-domain
dialogue systems have achieved great progress in conducting one-session
conversations. In contrast, Multi-Session Conversation (MSC), which consists of
multiple sessions over a long term with the same user, is under-investigated.
In this paper, we propose History-Aware Hierarchical Transformer (HAHT) for
multi-session open-domain dialogue. HAHT maintains a long-term memory of
history conversations and utilizes history information to understand current
conversation context and generate well-informed and context-relevant responses.
Specifically, HAHT first encodes history conversation sessions hierarchically
into a history memory. Then, HAHT leverages historical information to
facilitate the understanding of the current conversation context by encoding
the history memory together with the current context with attention-based
mechanisms. Finally, to explicitly utilize historical information, HAHT uses a
history-aware response generator that switches between a generic vocabulary and
a history-aware vocabulary. Experimental results on a large-scale MSC dataset
suggest that the proposed HAHT model consistently outperforms baseline models.
Human evaluation results support that HAHT generates more human-like,
context-relevant and history-relevant responses than baseline models.
