  Rubric-based admissions are claimed to help make the graduate admissions
process more equitable, possibly helping to address the historical and ongoing
inequities in the U.S. physics graduate school admissions process that have
often excluded applicants from minoritized races, ethnicities, genders, and
backgrounds. Yet, no studies have examined whether rubric-based admissions
methods represent a fundamental change of the admissions process or simply
represent a new tool that achieves the same outcome. To address that, we
developed supervised machine learning models of graduate admissions data
collected from our department over a seven-year period. During the first four
years, our department used a traditional admissions process and switched to a
rubric-based process for the following three years, allowing us to compare
which parts of the applications were used to drive admissions decisions. We
find that faculty focused on applicants' physics GRE scores and grade point
averages when making admissions decisions before the implementation of the
rubric. While we were able to develop a sufficiently good model whose results
we could trust for the data before the implementation of the rubric, we were
unable to do so for the data collected after the implementation of the rubric,
despite multiple modifications to the algorithms and data such as implementing
Tomek Links. Our inability to model the second data set despite being able to
model the first combined with model comparison analyses suggests that
rubric-based admissions does change the underlying process. These results
suggest that rubric-based holistic review is a method that could make the
graduate admissions process in physics more equitable.
