  This paper introduces a novel pipeline for summarising timelines of events
reported by multiple news sources. Transformer-based models for abstractive
summarisation generate coherent and concise summaries of long documents but can
fail to outperform established extractive methods on specialised tasks such as
timeline summarisation (TLS). While extractive summaries are more faithful to
their sources, they may be less readable and contain redundant or unnecessary
information. This paper proposes a preference-based reinforcement learning
(PBRL) method for adapting pretrained abstractive summarisers to TLS, which can
overcome the drawbacks of extractive timeline summaries. We define a compound
reward function that learns from keywords of interest and pairwise preference
labels, which we use to fine-tune a pretrained abstractive summariser via
offline reinforcement learning. We carry out both automated and human
evaluation on three datasets, finding that our method outperforms a comparable
extractive TLS method on two of the three benchmark datasets, and participants
prefer our method's summaries to those of both the extractive TLS method and
the pretrained abstractive model. The method does not require expensive
reference summaries and needs only a small number of preferences to align the
generated summaries with human preferences.
