  Automatic grading models are valued for the time and effort saved during the
instruction of large student bodies. Especially with the increasing
digitization of education and interest in large-scale standardized testing, the
popularity of automatic grading has risen to the point where commercial
solutions are widely available and used. However, for short answer formats,
automatic grading is challenging due to natural language ambiguity and
versatility. While automatic short answer grading models are beginning to
compare to human performance on some datasets, their robustness, especially to
adversarially manipulated data, is questionable. Exploitable vulnerabilities in
grading models can have far-reaching consequences ranging from cheating
students receiving undeserved credit to undermining automatic grading
altogether - even when most predictions are valid. In this paper, we devise a
black-box adversarial attack tailored to the educational short answer grading
scenario to investigate the grading models' robustness. In our attack, we
insert adjectives and adverbs into natural places of incorrect student answers,
fooling the model into predicting them as correct. We observed a loss of
prediction accuracy between 10 and 22 percentage points using the
state-of-the-art models BERT and T5. While our attack made answers appear less
natural to humans in our experiments, it did not significantly increase the
graders' suspicions of cheating. Based on our experiments, we provide
recommendations for utilizing automatic grading systems more safely in
practice.
