  Social biases on Wikipedia, a widely-read global platform, could greatly
influence public opinion. While prior research has examined man/woman gender
bias in biography articles, possible influences of other demographic attributes
limit conclusions. In this work, we present a methodology for analyzing
Wikipedia pages about people that isolates dimensions of interest (e.g.,
gender), from other attributes (e.g., occupation). Given a target corpus for
analysis (e.g.~biographies about women), we present a method for constructing a
comparison corpus that matches the target corpus in as many attributes as
possible, except the target one. We develop evaluation metrics to measure how
well the comparison corpus aligns with the target corpus and then examine how
articles about gender and racial minorities (cis. women, non-binary people,
transgender women, and transgender men; African American, Asian American, and
Hispanic/Latinx American people) differ from other articles. In addition to
identifying suspect social biases, our results show that failing to control for
covariates can result in different conclusions and veil biases. Our
contributions include methodology that facilitates further analyses of bias in
Wikipedia articles, findings that can aid Wikipedia editors in reducing biases,
and a framework and evaluation metrics to guide future work in this area.
