  Open cloze questions have been attracting attention for both measuring the
ability and facilitating the learning of L2 English learners. In spite of its
benefits, the open cloze test has been introduced only sporadically on the
educational front, largely because it is burdensome for teachers to manually
create the questions. Unlike the more commonly used multiple choice questions
(MCQ), open cloze questions are in free form and thus teachers have to ensure
that only a ground truth answer and no additional words will be accepted in the
blank. To help ease this burden, we developed CLOZER, an automatic open cloze
question generator. In this work, we evaluate CLOZER through quantitative
experiments on 1,600 answers and show statistically that it can successfully
generate open cloze questions that only accept the ground truth answer. A
comparative experiment with human-generated questions also reveals that CLOZER
can generate OCQs better than the average non-native English teacher.
Additionally, we conduct a field study at a local high school to clarify the
benefits and hurdles when introducing CLOZER. The results demonstrate that
while students found the application useful for their language learning.
Finally, on the basis of our findings, we proposed several design improvements.
