  Due to the superior performance, large-scale pre-trained language models
(PLMs) have been widely adopted in many aspects of human society. However, we
still lack effective tools to understand the potential bias embedded in the
black-box models. Recent advances in prompt tuning show the possibility to
explore the internal mechanism of the PLMs. In this work, we propose two
token-level sentiment tests: Sentiment Association Test (SAT) and Sentiment
Shift Test (SST) which utilize the prompt as a probe to detect the latent bias
in the PLMs. Our experiments on the collection of sentiment datasets show that
both SAT and SST can identify sentiment bias in PLMs and SST is able to
quantify the bias. The results also suggest that fine-tuning can possibly
augment the existing bias in PLMs.
