  Grammatical error correction (GEC) is one of the areas in natural language
processing in which purely neural models have not yet superseded more
traditional symbolic models. Hybrid systems combining phrase-based statistical
machine translation (SMT) and neural sequence models are currently among the
most effective approaches to GEC. However, both SMT and neural
sequence-to-sequence models require large amounts of annotated data. Language
model based GEC (LM-GEC) is a promising alternative which does not rely on
annotated training data. We show how to improve LM-GEC by applying modelling
techniques based on finite state transducers. We report further gains by
rescoring with neural language models. We show that our methods developed for
LM-GEC can also be used with SMT systems if annotated training data is
available. Our best system outperforms the best published result on the
CoNLL-2014 test set, and achieves far better relative improvements over the SMT
baselines than previous hybrid systems.
