  Grammatical error correction (GEC) is the task of correcting typos, spelling,
punctuation and grammatical issues in text. Approaching the problem as a
sequence-to-sequence task, we compare the use of a common subword unit
vocabulary and byte-level encoding. Initial synthetic training data is created
using an error-generating pipeline, and used for finetuning two subword-level
models and one byte-level model. Models are then finetuned further on
hand-corrected error corpora, including texts written by children, university
students, dyslexic and second-language writers, and evaluated over different
error types and origins. We show that a byte-level model enables higher
correction quality than a subword approach, not only for simple spelling
errors, but also for more complex semantic, stylistic and grammatical issues.
In particular, initial training on synthetic corpora followed by finetuning on
a relatively small parallel corpus of real-world errors helps the byte-level
model correct a wide range of commonly occurring errors. Our experiments are
run for the Icelandic language but should hold for other similar languages,
particularly morphologically rich ones.
