  To measure bias, we encourage teams to consider using AUC Gap: the absolute
difference between the highest and lowest test AUC for subgroups (e.g., gender,
race, SES, prior knowledge). It is agnostic to the AI/ML algorithm used and it
captures the disparity in model performance for any number of subgroups, which
enables non-binary fairness assessments such as for intersectional identity
groups. The teams use a wide range of AI/ML models in pursuit of a common goal
of doubling math achievement in low-income middle schools. Ensuring that the
models, which are trained on datasets collected in many different contexts, do
not introduce or amplify biases is important for achieving the goal. We offer
here a versatile and easy-to-compute measure of model bias for all the teams in
order to create a common benchmark and an analytical basis for sharing what
strategies have worked for different teams.
