  Transformer-based pretrained models like BERT, GPT-2 and T5 have been
finetuned for a large number of natural language processing (NLP) tasks, and
have been shown to be very effective. However, while finetuning, what changes
across layers in these models with respect to pretrained checkpoints is
under-studied. Further, how robust are these models to perturbations in input
text? Does the robustness vary depending on the NLP task for which the models
have been finetuned? While there exists some work on studying the robustness of
BERT finetuned for a few NLP tasks, there is no rigorous study that compares
this robustness across encoder only, decoder only and encoder-decoder models.
In this paper, we characterize changes between pretrained and finetuned
language model representations across layers using two metrics: CKA and STIR.
Further, we study the robustness of three language models (BERT, GPT-2 and T5)
with eight different text perturbations on classification tasks from the
General Language Understanding Evaluation (GLUE) benchmark, and generation
tasks like summarization, free-form generation and question generation. GPT-2
representations are more robust than BERT and T5 across multiple types of input
perturbation. Although models exhibit good robustness broadly, dropping nouns,
verbs or changing characters are the most impactful. Overall, this study
provides valuable insights into perturbation-specific weaknesses of popular
Transformer-based models, which should be kept in mind when passing inputs. We
make the code and models publicly available
[https://github.com/PavanNeerudu/Robustness-of-Transformers-models].
