  Machine Learning (ML) has seen massive progress in the last decade and as a
result, there is a pressing need for validating ML-based systems. To this end,
we propose, design and evaluate CALLISTO - a novel test generation and data
quality assessment framework. To the best of our knowledge, CALLISTO is the
first blackbox framework to leverage the uncertainty in the prediction and
systematically generate new test cases for ML classifiers. Our evaluation of
CALLISTO on four real world data sets reveals thousands of errors. We also show
that leveraging the uncertainty in prediction can increase the number of
erroneous test cases up to a factor of 20, as compared to when no such
knowledge is used for testing.
  CALLISTO has the capability to detect low quality data in the datasets that
may contain mislabelled data. We conduct and present an extensive user study to
validate the results of CALLISTO on identifying low quality data from four
state-of-the-art real world datasets.
