  Opinions in scientific research papers can be divergent, leading to
controversies among reviewers. However, most existing datasets for opinion
summarization are centered around product reviews and assume that the analyzed
opinions are non-controversial, failing to account for the variability seen in
other contexts such as academic papers, political debates, or social media
discussions. To address this gap, we propose the task of scientific opinion
summarization, where research paper reviews are synthesized into meta-reviews.
To facilitate this task, we introduce the ORSUM dataset covering 15,062 paper
meta-reviews and 57,536 paper reviews from 47 conferences. Furthermore, we
propose the Checklist-guided Iterative Introspection approach, which breaks
down scientific opinion summarization into several stages, iteratively refining
the summary under the guidance of questions from a checklist. Our experiments
show that (1) human-written summaries do not always satisfy all necessary
criteria such as depth of discussion, and identifying consensus and controversy
for the specific domain, and (2) the combination of task decomposition and
iterative self-refinement shows strong potential for enhancing the opinions and
can be applied to other complex text generation using black-box LLMs.
