  How well can large language models (LLMs) generate summaries? We develop new
datasets and conduct human evaluation experiments to evaluate the zero-shot
generation capability of LLMs across five distinct summarization tasks. Our
findings indicate a clear preference among human evaluators for LLM-generated
summaries over human-written summaries and summaries generated by fine-tuned
models. Specifically, LLM-generated summaries exhibit better factual
consistency and fewer instances of extrinsic hallucinations. Due to the
satisfactory performance of LLMs in summarization tasks (even surpassing the
benchmark of reference summaries), we believe that most conventional works in
the field of text summarization are no longer necessary in the era of LLMs.
However, we recognize that there are still some directions worth exploring,
such as the creation of novel datasets with higher quality and more reliable
evaluation methods.
