  Machine learning models can reach high performance on benchmark natural
language processing (NLP) datasets but fail in more challenging settings. We
study this issue when a pre-trained model learns dataset artifacts in natural
language inference (NLI), the topic of studying the logical relationship
between a pair of text sequences. We provide a variety of techniques for
analyzing and locating dataset artifacts inside the crowdsourced Stanford
Natural Language Inference (SNLI) corpus. We study the stylistic pattern of
dataset artifacts in the SNLI. To mitigate dataset artifacts, we employ a
unique multi-scale data augmentation technique with two distinct frameworks: a
behavioral testing checklist at the sentence level and lexical synonym criteria
at the word level. Specifically, our combination method enhances our model's
resistance to perturbation testing, enabling it to continuously outperform the
pre-trained baseline.
