  Social media is increasingly used for large-scale population predictions,
such as estimating community health statistics. However, social media users are
not typically a representative sample of the intended population -- a
"selection bias". Within the social sciences, such a bias is typically
addressed with restratification techniques, where observations are reweighted
according to how under- or over-sampled their socio-demographic groups are.
Yet, restratifaction is rarely evaluated for improving prediction. In this
two-part study, we first evaluate standard, "out-of-the-box" restratification
techniques, finding they provide no improvement and often even degraded
prediction accuracies across four tasks of esimating U.S. county population
health statistics from Twitter. The core reasons for degraded performance seem
to be tied to their reliance on either sparse or shrunken estimates of each
population's socio-demographics. In the second part of our study, we develop
and evaluate Robust Poststratification, which consists of three methods to
address these problems: (1) estimator redistribution to account for shrinking,
as well as (2) adaptive binning and (3) informed smoothing to handle sparse
socio-demographic estimates. We show that each of these methods leads to
significant improvement in prediction accuracies over the standard
restratification approaches. Taken together, Robust Poststratification enables
state-of-the-art prediction accuracies, yielding a 53.0% increase in variance
explained (R^2) in the case of surveyed life satisfaction, and a 17.8% average
increase across all tasks.
