  We present a metric to quantify systematic labeling bias in galaxy morphology
data sets stemming from the quality of the labeled data. This labeling bias is
independent from labeling errors and requires knowledge about the intrinsic
properties of the data with respect to the observed properties. We conduct a
relative comparison of label bias for different low redshift galaxy morphology
data sets. We show our metric is able to recover previous de-biasing procedures
based on redshift as biasing parameter. By using the image resolution instead,
we find biases that have not been addressed. We find that the morphologies
based on supervised machine-learning trained over features such as colors,
shape, and concentration show significantly less bias than morphologies based
on expert or citizen-science classifiers. This result holds even when there is
underlying bias present in the training sets used in the supervised machine
learning process. We use catalog simulations to validate our bias metric, and
show how to bin the multidimensional intrinsic and observed galaxy properties
used in the bias quantification. Our approach is designed to work on any other
labeled multidimensional data sets and the code is publicly available.
