  Millions of news articles are published online every day, which can be
overwhelming for readers to follow. Grouping articles that are reporting the
same event into news stories is a common way of assisting readers in their news
consumption. However, it remains a challenging research problem to efficiently
and effectively generate a representative headline for each story. Automatic
summarization of a document set has been studied for decades, while few studies
have focused on generating representative headlines for a set of articles.
Unlike summaries, which aim to capture most information with least redundancy,
headlines aim to capture information jointly shared by the story articles in
short length, and exclude information that is too specific to each individual
article. In this work, we study the problem of generating representative
headlines for news stories. We develop a distant supervision approach to train
large-scale generation models without any human annotation. This approach
centers on two technical components. First, we propose a multi-level
pre-training framework that incorporates massive unlabeled corpus with
different quality-vs.-quantity balance at different levels. We show that models
trained within this framework outperform those trained with pure human curated
corpus. Second, we propose a novel self-voting-based article attention layer to
extract salient information shared by multiple articles. We show that models
that incorporate this layer are robust to potential noises in news stories and
outperform existing baselines with or without noises. We can further enhance
our model by incorporating human labels, and we show our distant supervision
approach significantly reduces the demand on labeled data.
