  We investigate performance disparities in deep classifiers. We find that the
ability of classifiers to separate individuals into subgroups varies
substantially across medical imaging modalities and protected characteristics;
crucially, we show that this property is predictive of algorithmic bias.
Through theoretical analysis and extensive empirical evaluation, we find a
relationship between subgroup separability, subgroup disparities, and
performance degradation when models are trained on data with systematic bias
such as underdiagnosis. Our findings shed new light on the question of how
models become biased, providing important insights for the development of fair
medical imaging AI.
