  This paper presents an experiment of automatically scoring handwritten
descriptive answers in the trial tests for the new Japanese university entrance
examination, which were made for about 120,000 examinees in 2017 and 2018.
There are about 400,000 answers with more than 20 million characters. Although
all answers have been scored by human examiners, handwritten characters are not
labeled. We present our attempt to adapt deep neural network-based handwriting
recognizers trained on a labeled handwriting dataset into this unlabeled answer
set. Our proposed method combines different training strategies, ensembles
multiple recognizers, and uses a language model built from a large general
corpus to avoid overfitting into specific data. In our experiment, the proposed
method records character accuracy of over 97% using about 2,000 verified
labeled answers that account for less than 0.5% of the dataset. Then, the
recognized answers are fed into a pre-trained automatic scoring system based on
the BERT model without correcting misrecognized characters and providing rubric
annotations. The automatic scoring system achieves from 0.84 to 0.98 of
Quadratic Weighted Kappa (QWK). As QWK is over 0.8, it represents an acceptable
similarity of scoring between the automatic scoring system and the human
examiners. These results are promising for further research on end-to-end
automatic scoring of descriptive answers.
