  Spelling error correction is one of topics which have a long history in
natural language processing. Although previous studies have achieved remarkable
results, challenges still exist. In the Vietnamese language, a state-of-the-art
method for the task infers a syllable's context from its adjacent syllables.
The method's accuracy can be unsatisfactory, however, because the model may
lose the context if two (or more) spelling mistakes stand near each other. In
this paper, we propose a novel method to correct Vietnamese spelling errors. We
tackle the problems of mistyped errors and misspelled errors by using a deep
learning model. The embedding layer, in particular, is powered by the byte pair
encoding technique. The sequence to sequence model based on the Transformer
architecture makes our approach different from the previous works on the same
problem. In the experiment, we train the model with a large synthetic dataset,
which is randomly introduced spelling errors. We test the performance of the
proposed method using a realistic dataset. This dataset contains 11,202
human-made misspellings in 9,341 different Vietnamese sentences. The
experimental results show that our method achieves encouraging performance with
86.8% errors detected and 81.5% errors corrected, which improves the
state-of-the-art approach 5.6% and 2.2%, respectively.
