  Large-scale pre-trained language models have shown outstanding performance in
a variety of NLP tasks. However, they are also known to be significantly
brittle against specifically crafted adversarial examples, leading to
increasing interest in probing the adversarial robustness of NLP systems. We
introduce RSMI, a novel two-stage framework that combines randomized smoothing
(RS) with masked inference (MI) to improve the adversarial robustness of NLP
systems. RS transforms a classifier into a smoothed classifier to obtain robust
representations, whereas MI forces a model to exploit the surrounding context
of a masked token in an input sequence. RSMI improves adversarial robustness by
2 to 3 times over existing state-of-the-art methods on benchmark datasets. We
also perform in-depth qualitative analysis to validate the effectiveness of the
different stages of RSMI and probe the impact of its components through
extensive ablations. By empirically proving the stability of RSMI, we put it
forward as a practical method to robustly train large-scale NLP models. Our
code and datasets are available at https://github.com/Han8931/rsmi_nlp
