  As machine learning algorithms are more widely deployed in healthcare, the
question of algorithmic fairness becomes more critical to examine. Our work
seeks to identify and understand disparities in a deployed model that
classifies doctor-patient conversations into sections of a medical SOAP note.
We employ several metrics to measure disparities in the classifier performance,
and find small differences in a portion of the disadvantaged groups. A deeper
analysis of the language in these conversations and further stratifying the
groups suggests these differences are related to and often attributable to the
type of medical appointment (e.g., psychiatric vs. internist). Our findings
stress the importance of understanding the disparities that may exist in the
data itself and how that affects a model's ability to equally distribute
benefits.
