  High-stakes exams play a large role in determining an introductory physics
student's final grade. These exams have been shown to be inequitable, often to
the detriment of students identifying with groups historically marginalized in
physics. Given that exams are made up of individual questions, it is plausible
that certain types of questions may be contributing to the observed equity
gaps. In this paper, we examine whether that is the case for one type of
forced-choice question, the complex multiple-choice (CMC) question. In a CMC
question, students must select an answer choice that includes all correct
responses and no incorrect responses from a list. To conduct our study, we used
four years of data from Problem Roulette, an online program at our university
that allows students to prepare for exams with actual questions from previous
years' exams in a not-for-credit format. We categorized the 951 Physics II
(Electricity and Magnetism) questions in the database as CMC or non-CMC. We
found that students performed 8.0 percentage points worse on CMC questions than
they did on non-CMC questions. In addition, we found differential impacts for
low income students and domestic students relative to medium and high income
students and international students. Regression models supported these
descriptive findings. The results suggest that complex multiple-choice
questions may be contributing to the equity gaps observed on physics exams.
Considering, however, that CMC questions are more difficult for everyone,
future research should examine the source of this difficulty and whether that
source is functionally related to learning and assessment. For example, our
data does not support using CMC questions instead of non-CMC as a way to
differentiate top-performing students from everyone else.
