  Grammatical error correction (GEC) is a challenging task of natural language
processing techniques. While more attempts are being made in this approach for
universal languages like English or Chinese, relatively little work has been
done for low-resource languages for the lack of large annotated corpora. In
low-resource languages, the current unsupervised GEC based on language model
scoring performs well. However, the pre-trained language model is still to be
explored in this context. This study proposes a BERT-based unsupervised GEC
framework, where GEC is viewed as multi-class classification task. The
framework contains three modules: data flow construction module, sentence
perplexity scoring module, and error detecting and correcting module. We
propose a novel scoring method for pseudo-perplexity to evaluate a sentence's
probable correctness and construct a Tagalog corpus for Tagalog GEC research.
It obtains competitive performance on the Tagalog corpus we construct and
open-source Indonesian corpus and it demonstrates that our framework is
complementary to baseline method for low-resource GEC task.
