  Although the recent rise and uptake of COVID-19 vaccines in the United States
has been encouraging, there continues to be significant vaccine hesitancy in
various geographic and demographic clusters of the adult population. Surveys,
such as the one conducted by Gallup over the past year, can be useful in
determining vaccine hesitancy, but can be expensive to conduct and do not
provide real-time data. At the same time, the advent of social media suggests
that it may be possible to get vaccine hesitancy signals at an aggregate level
(such as at the level of zip codes) by using machine learning models and
socioeconomic (and other) features from publicly available sources. It is an
open question at present whether such an endeavor is feasible, and how it
compares to baselines that only use constant priors. To our knowledge, a proper
methodology and evaluation results using real data has also not been presented.
In this article, we present such a methodology and experimental study, using
publicly available Twitter data collected over the last year. Our goal is not
to devise novel machine learning algorithms, but to evaluate existing and
established models in a comparative framework. We show that the best models
significantly outperform constant priors, and can be set up using open-source
tools.
