  The same real-life questions posed to different individuals may lead to
different answers based on their unique situations. For instance, whether a
student is eligible for a scholarship depends on eligibility conditions, such
as major or degree required. ConditionalQA was proposed to evaluate models'
capability of reading a document and answering eligibility questions,
considering unmentioned conditions. However, it is limited to questions on
single documents, neglecting harder cases that may require cross-document
reasoning and optimization, for example, "What is the maximum number of
scholarships attainable?" Such questions over multiple documents are not only
more challenging due to more context having to understand, but also because the
model has to (1) explore all possible combinations of unmentioned conditions
and (2) understand the relationship between conditions across documents, to
reason about the optimal outcome. To evaluate models' capability of answering
such questions, we propose a new dataset MDCR, which can reflect real-world
challenges and serve as a new test bed for complex conditional reasoning that
requires optimization. We evaluate this dataset using the most recent LLMs and
demonstrate their limitations in solving this task. We believe this dataset
will facilitate future research in answering optimization questions with
unknown conditions.
