  Each year, selective American colleges sort through tens of thousands of
applications to identify a first-year class that displays both academic merit
and diversity. In the 2023-2024 admissions cycle, these colleges faced
unprecedented challenges. First, the number of applications has been steadily
growing. Second, test-optional policies that have remained in place since the
COVID-19 pandemic limit access to key information historically predictive of
academic success. Most recently, longstanding debates over affirmative action
culminated in the Supreme Court banning race-conscious admissions. Colleges
have explored machine learning (ML) models to address the issues of scale and
missing test scores, often via ranking algorithms intended to focus on 'top'
applicants. However, the Court's ruling will force changes to these models,
which were able to consider race as a factor in ranking. There is currently a
poor understanding of how these mandated changes will shape applicant ranking
algorithms, and, by extension, admitted classes. We seek to address this by
quantifying the impact of different admission policies on the applications
prioritized for review. We show that removing race data from a developed
applicant ranking algorithm reduces the diversity of the top-ranked pool
without meaningfully increasing the academic merit of that pool. We
contextualize this impact by showing that excluding data on applicant race has
a greater impact than excluding other potentially informative variables like
intended majors. Finally, we measure the impact of policy change on individuals
by comparing the arbitrariness in applicant rank attributable to policy change
to the arbitrariness attributable to randomness. We find that any given policy
has a high degree of arbitrariness and that removing race data from the ranking
algorithm increases arbitrariness in outcomes for most applicants.
