  A long-standing challenge of deep learning models involves how to handle
noisy labels, especially in applications where human lives are at stake.
Adoption of the data Shapley Value (SV), a cooperative game theoretical
approach, is an intelligent valuation solution to tackle the issue of noisy
labels. Data SV can be used together with a learning model and an evaluation
metric to validate each training point's contribution to the model's
performance. The SV of a data point, however, is not unique and depends on the
learning model, the evaluation metric, and other data points collaborating in
the training game. However, effects of utilizing different evaluation metrics
for computation of the SV, detecting the noisy labels, and measuring the data
points' importance has not yet been thoroughly investigated. In this context,
we performed a series of comparative analyses to assess SV's capabilities to
detect noisy input labels when measured by different evaluation metrics. Our
experiments on COVID-19-infected of CT images illustrate that although the data
SV can effectively identify noisy labels, adoption of different evaluation
metric can significantly influence its ability to identify noisy labels from
different data classes. Specifically, we demonstrate that the SV greatly
depends on the associated evaluation metric.
