  Consistency training regularizes a model by enforcing predictions of original
and perturbed inputs to be similar. Previous studies have proposed various
augmentation methods for the perturbation but are limited in that they are
agnostic to the training model. Thus, the perturbed samples may not aid in
regularization due to their ease of classification from the model. In this
context, we propose an augmentation method of adding a discrete noise that
would incur the highest divergence between predictions. This virtual
adversarial discrete noise obtained by replacing a small portion of tokens
while keeping original semantics as much as possible efficiently pushes a
training model's decision boundary. Experimental results show that our proposed
method outperforms other consistency training baselines with text editing,
paraphrasing, or a continuous noise on semi-supervised text classification
tasks and a robustness benchmark
