  Stylistic headline generation is the task to generate a headline that not
only summarizes the content of an article, but also reflects a desired style
that attracts users. As style-specific article-headline pairs are scarce,
previous researches focus on unsupervised approaches with a standard headline
generation dataset and mono-style corpora. In this work, we follow this line
and propose StyleBART, an unsupervised approach for stylistic headline
generation. Our method decorates the pretrained BART model with adapters that
are responsible for different styles and allows the generation of headlines
with diverse styles by simply switching the adapters. Different from previous
works, StyleBART separates the task of style learning and headline generation,
making it possible to freely combine the base model and the style adapters
during inference. We further propose an inverse paraphrasing task to enhance
the style adapters. Extensive automatic and human evaluations show that
StyleBART achieves new state-of-the-art performance in the unsupervised
stylistic headline generation task, producing high-quality headlines with the
desired style.
