  Recently, with the help of deep learning models, significant advances have
been made in different Natural Language Processing (NLP) tasks. Unfortunately,
state-of-the-art models are vulnerable to noisy texts. We propose a new
contextual text denoising algorithm based on the ready-to-use masked language
model. The proposed algorithm does not require retraining of the model and can
be integrated into any NLP system without additional training on paired
cleaning training data. We evaluate our method under synthetic noise and
natural noise and show that the proposed algorithm can use context information
to correct noise text and improve the performance of noisy inputs in several
downstream tasks.
