  Long-range context modeling is crucial to both dialogue understanding and
generation. The most popular method for dialogue context representation is to
concatenate the last-$k$ utterances in chronological order. However, this
method may not be ideal for conversations containing long-range dependencies,
i.e., when there is a need to look beyond last-$k$ utterances to generate a
meaningful response. In this work, we propose DialoGen, a novel encoder-decoder
based framework for dialogue generation with a generalized context
representation that can look beyond the last-$k$ utterances. The main idea of
our approach is to identify and utilize the most relevant historical utterances
instead of last-$k$, which also enables the compact representation of dialogue
history with fewer tokens. We study the effectiveness of our proposed method on
both dialogue generation (open-domain) and understanding (DST). Even with a
compact context representation, DialoGen performs comparably to the
state-of-the-art models on the open-domain DailyDialog dataset. We observe a
similar behavior on the DST task of the MultiWOZ dataset when the proposed
context representation is applied to existing DST models. We also discuss the
generalizability and interpretability of DialoGen and show that the relevance
score of previous utterances agrees well with human cognition.
