  We study multi-answer retrieval, an under-explored problem that requires
retrieving passages to cover multiple distinct answers for a given question.
This task requires joint modeling of retrieved passages, as models should not
repeatedly retrieve passages containing the same answer at the cost of missing
a different valid answer. In this paper, we introduce JPR, the first joint
passage retrieval model for multi-answer retrieval. JPR makes use of an
autoregressive reranker that selects a sequence of passages, each conditioned
on previously selected passages. JPR is trained to select passages that cover
new answers at each timestep and uses a tree-decoding algorithm to enable
flexibility in the degree of diversity. Compared to prior approaches, JPR
achieves significantly better answer coverage on three multi-answer datasets.
When combined with downstream question answering, the improved retrieval
enables larger answer generation models since they need to consider fewer
passages, establishing a new state-of-the-art.
