  An emerging area of research in situated and multimodal interactive
conversations (SIMMC) includes interactions in scientific papers. Since
scientific papers are primarily composed of text, equations, figures, and
tables, SIMMC methods must be developed specifically for each component to
support the depth of inquiry and interactions required by research scientists.
This work introduces Conversational Papers (cPAPERS), a dataset of
conversational question-answer pairs from reviews of academic papers grounded
in these paper components and their associated references from scientific
documents available on arXiv. We present a data collection strategy to collect
these question-answer pairs from OpenReview and associate them with contextual
information from LaTeX source files. Additionally, we present a series of
baseline approaches utilizing Large Language Models (LLMs) in both zero-shot
and fine-tuned configurations to address the cPAPERS dataset.
