  The Stereotype Content model (SCM) states that we tend to perceive minority
groups as cold, incompetent or both. In this paper we adapt existing work to
demonstrate that the Stereotype Content model holds for contextualised word
embeddings, then use these results to evaluate a fine-tuning process designed
to drive a language model away from stereotyped portrayals of minority groups.
We find the SCM terms are better able to capture bias than demographic agnostic
terms related to pleasantness. Further, we were able to reduce the presence of
stereotypes in the model through a simple fine-tuning procedure that required
minimal human and computer resources, without harming downstream performance.
We present this work as a prototype of a debiasing procedure that aims to
remove the need for a priori knowledge of the specifics of bias in the model.
