  This paper describes a method for using Transformer-based Language Models
(TLMs) to understand public opinion from social media posts. In this approach,
we train a set of GPT models on several COVID-19 tweet corpora that reflect
populations of users with distinctive views. We then use prompt-based queries
to probe these models to reveal insights into the biases and opinions of the
users. We demonstrate how this approach can be used to produce results which
resemble polling the public on diverse social, political and public health
issues. The results on the COVID-19 tweet data show that transformer language
models are promising tools that can help us understand public opinions on
social media at scale.
