  As the number of fine tuning of pretrained models increased, understanding
the bias of pretrained model is essential. However, there is little tool to
analyse transformer architecture and the interpretation of the attention maps
is still challenging. To tackle the interpretability, we propose
Input-Attribution and Attention Score Vector (IAV) which measures the
similarity between attention map and input-attribution and shows the general
trend of interpretable attention patterns. We empirically explain the
pretraining bias of supervised and unsupervised pretrained ViT models, and show
that each head in ViT has a specific range of agreement on the decision of the
classification. We show that generalization, robustness and entropy of
attention maps are not property of pretraining types. On the other hand, IAV
trend can separate the pretraining types.
