  Purpose: When a learner fails to reach a milestone, educators often wonder if
there had been any warning signs that could have allowed them to intervene
sooner. Machine learning can predict which students are at risk of failing a
high-stakes certification exam. If predictions can be made well in advance of
the exam, then educators can meaningfully intervene before students take the
exam to reduce the chances of a failing score.
  Methods: Using already-collected, first-year student assessment data from
five cohorts in a Master of Physician Assistant Studies program, the authors
implement an "adaptive minimum match" version of the k-nearest neighbors
algorithm (AMMKNN), using changing numbers of neighbors to predict each
student's future exam scores on the Physician Assistant National Certifying
Examination (PANCE). Validation occurred in two ways: Leave-one-out
cross-validation (LOOCV) and evaluating the predictions in a new cohort.
  Results: AMMKNN achieved an accuracy of 93% in LOOCV. AMMKNN generates a
predicted PANCE score for each student, one year before they are scheduled to
take the exam. Students can then be classified into extra support, optional
extra support, or no extra support groups. The educator then has one year to
provide the appropriate customized support to each category of student.
  Conclusions: Predictive analytics can identify at-risk students, so they can
receive additional support or remediation when preparing for high-stakes
certification exams. Educators can use the included methods and code to
generate predicted test outcomes for students. The authors recommend that
educators use this or similar predictive methods responsibly and transparently,
as one of many tools used to support students.
