  In the current landscape of language model research, larger models, larger
datasets and more compute seems to be the only way to advance towards
intelligence. While there have been extensive studies of scaling laws and
models' scaling behaviors, the effect of scale on a model's social biases and
stereotyping tendencies has received less attention. In this study, we explore
the influence of model scale and pre-training data on its learnt social biases.
We focus on BERT -- an extremely popular language model -- and investigate
biases as they show up during language modeling (upstream), as well as during
classification applications after fine-tuning (downstream). Our experiments on
four architecture sizes of BERT demonstrate that pre-training data
substantially influences how upstream biases evolve with model scale. With
increasing scale, models pre-trained on large internet scrapes like Common
Crawl exhibit higher toxicity, whereas models pre-trained on moderated data
sources like Wikipedia show greater gender stereotypes. However, downstream
biases generally decrease with increasing model scale, irrespective of the
pre-training data. Our results highlight the qualitative role of pre-training
data in the biased behavior of language models, an often overlooked aspect in
the study of scale. Through a detailed case study of BERT, we shed light on the
complex interplay of data and model scale, and investigate how it translates to
concrete biases.
