  Dialogue summarization has recently garnered significant attention due to its
wide range of applications. However, existing methods for summarizing dialogues
have limitations because they do not take into account the inherent structure
of dialogue and rely heavily on labeled data, which can lead to poor
performance in new domains. In this work, we propose DIONYSUS (dynamic input
optimization in pre-training for dialogue summarization), a pre-trained
encoder-decoder model for summarizing dialogues in any new domain. To pre-train
DIONYSUS, we create two pseudo summaries for each dialogue example: one is
produced by a fine-tuned summarization model, and the other is a collection of
dialogue turns that convey important information. We then choose one of these
pseudo summaries based on the difference in information distribution across
different types of dialogues. This selected pseudo summary serves as the
objective for pre-training DIONYSUS using a self-supervised approach on a large
dialogue corpus. Our experiments show that DIONYSUS outperforms existing
methods on six datasets, as demonstrated by its ROUGE scores in zero-shot and
few-shot settings.
