  Query-focused summarization (QFS) requires generating a summary given a query
using a set of relevant documents. However, such relevant documents should be
annotated manually and thus are not readily available in realistic scenarios.
To address this limitation, we tackle the QFS task as a knowledge-intensive
(KI) task without access to any relevant documents. Instead, we assume that
these documents are present in a large-scale knowledge corpus and should be
retrieved first. To explore this new setting, we build a new dataset (KI-QFS)
by adapting existing QFS datasets. In this dataset, answering the query
requires document retrieval from a knowledge corpus. We construct three
different knowledge corpora, and we further provide relevance annotations to
enable retrieval evaluation. Finally, we benchmark the dataset with
state-of-the-art QFS models and retrieval-enhanced models. The experimental
results demonstrate that QFS models perform significantly worse on KI-QFS
compared to the original QFS task, indicating that the knowledge-intensive
setting is much more challenging and offers substantial room for improvement.
We believe that our investigation will inspire further research into addressing
QFS in more realistic scenarios.
