  Multi-document summarization (MDS) has made significant progress in recent
years, in part facilitated by the availability of new, dedicated datasets and
capacious language models. However, a standing limitation of these models is
that they are trained against limited references and with plain
maximum-likelihood objectives. As for many other generative tasks,
reinforcement learning (RL) offers the potential to improve the training of MDS
models; yet, it requires a carefully-designed reward that can ensure
appropriate leverage of both the reference summaries and the input documents.
For this reason, in this paper we propose fine-tuning an MDS baseline with a
reward that balances a reference-based metric such as ROUGE with coverage of
the input documents. To implement the approach, we utilize RELAX (Grathwohl et
al., 2018), a contemporary gradient estimator which is both low-variance and
unbiased, and we fine-tune the baseline in a few-shot style for both stability
and computational efficiency. Experimental results over the Multi-News and WCEP
MDS datasets show significant improvements of up to +0.95 pp average ROUGE
score and +3.17 pp METEOR score over the baseline, and competitive results with
the literature. In addition, they show that the coverage of the input documents
is increased, and evenly across all documents.
