  Traditional quantitative content analysis approach (human coding method) has
weaknesses, such as assuming all human coders are equally accurate once the
intercoder reliability for training reaches a threshold score. We applied the
Biased-Annotator Competence Estimation (BACE) model (Tyler, 2021), which draws
on Bayesian modeling to improve human coding. An important contribution of this
model is it takes each coder's potential biases and reliability into
consideration and treats the "true" label of each message as a latent
parameter, with quantifiable estimation uncertainties. In contrast, in
conventional human coding, each message will receive a fixed label without
estimates for measurement uncertainties. In this extended abstract, we first
summarize the weaknesses of conventional human coding; and then apply the BACE
model to COVID-19 vaccine Twitter data and compare BACE with other statistical
models; finally, we discuss how the BACE model can be applied to improve human
coding of latent message features.
