  Assessing the quality and impact of individual data points is critical for
improving model performance and mitigating undesirable biases within the
training dataset. Several data valuation algorithms have been proposed to
quantify data quality, however, there lacks a systemic and standardized
benchmarking system for data valuation. In this paper, we introduce
OpenDataVal, an easy-to-use and unified benchmark framework that empowers
researchers and practitioners to apply and compare various data valuation
algorithms. OpenDataVal provides an integrated environment that includes (i) a
diverse collection of image, natural language, and tabular datasets, (ii)
implementations of eleven different state-of-the-art data valuation algorithms,
and (iii) a prediction model API that can import any models in scikit-learn.
Furthermore, we propose four downstream machine learning tasks for evaluating
the quality of data values. We perform benchmarking analysis using OpenDataVal,
quantifying and comparing the efficacy of state-of-the-art data valuation
approaches. We find that no single algorithm performs uniformly best across all
tasks, and an appropriate algorithm should be employed for a user's downstream
task. OpenDataVal is publicly available at https://opendataval.github.io with
comprehensive documentation. Furthermore, we provide a leaderboard where
researchers can evaluate the effectiveness of their own data valuation
algorithms.
