  We improve automatic correction of grammatical, orthographic, and collocation
errors in text using a multilayer convolutional encoder-decoder neural network.
The network is initialized with embeddings that make use of character N-gram
information to better suit this task. When evaluated on common benchmark test
data sets (CoNLL-2014 and JFLEG), our model substantially outperforms all prior
neural approaches on this task as well as strong statistical machine
translation-based systems with neural and task-specific features trained on the
same data. Our analysis shows the superiority of convolutional neural networks
over recurrent neural networks such as long short-term memory (LSTM) networks
in capturing the local context via attention, and thereby improving the
coverage in correcting grammatical errors. By ensembling multiple models, and
incorporating an N-gram language model and edit features via rescoring, our
novel method becomes the first neural approach to outperform the current
state-of-the-art statistical machine translation-based approach, both in terms
of grammaticality and fluency.
