  In this work, we examine the extent to which embeddings may encode
marginalized populations differently, and how this may lead to a perpetuation
of biases and worsened performance on clinical tasks. We pretrain deep
embedding models (BERT) on medical notes from the MIMIC-III hospital dataset,
and quantify potential disparities using two approaches. First, we identify
dangerous latent relationships that are captured by the contextual word
embeddings using a fill-in-the-blank method with text from real clinical notes
and a log probability bias score quantification. Second, we evaluate
performance gaps across different definitions of fairness on over 50 downstream
clinical prediction tasks that include detection of acute and chronic
conditions. We find that classifiers trained from BERT representations exhibit
statistically significant differences in performance, often favoring the
majority group with regards to gender, language, ethnicity, and insurance
status. Finally, we explore shortcomings of using adversarial debiasing to
obfuscate subgroup information in contextual word embeddings, and recommend
best practices for such deep embedding models in clinical settings.
