  Generating high-quality textual adversarial examples is critical for
investigating the pitfalls of natural language processing (NLP) models and
further promoting their robustness. Existing attacks are usually realized
through word-level or sentence-level perturbations, which either limit the
perturbation space or sacrifice fluency and textual quality, both affecting the
attack effectiveness. In this paper, we propose Phrase-Level Textual
Adversarial aTtack (PLAT) that generates adversarial samples through
phrase-level perturbations. PLAT first extracts the vulnerable phrases as
attack targets by a syntactic parser, and then perturbs them by a pre-trained
blank-infilling model. Such flexible perturbation design substantially expands
the search space for more effective attacks without introducing too many
modifications, and meanwhile maintaining the textual fluency and grammaticality
via contextualized generation using surrounding texts. Moreover, we develop a
label-preservation filter leveraging the likelihoods of language models
fine-tuned on each class, rather than textual similarity, to rule out those
perturbations that potentially alter the original class label for humans.
Extensive experiments and human evaluation demonstrate that PLAT has a superior
attack effectiveness as well as a better label consistency than strong
baselines.
