  Training data plays an essential role in modern applications of machine
learning. However, gathering labeled training data is time-consuming.
Therefore, labeling is often outsourced to less experienced users, or
completely automated. This can introduce errors, which compromise valuable
training data, and lead to suboptimal training results. We thus propose a novel
approach that uses the power of pretrained classifiers to visually guide users
to noisy labels, and let them interactively check error candidates, to
iteratively improve the training data set. To systematically investigate
training data, we propose a categorization of labeling errors into three
different types, based on an analysis of potential pitfalls in label
acquisition processes. For each of these types, we present approaches to
detect, reason about, and resolve error candidates, as we propose measures and
visual guidance techniques to support machine learning users. Our approach has
been used to spot errors in well-known machine learning benchmark data sets,
and we tested its usability during a user evaluation. While initially developed
for images, the techniques presented in this paper are independent of the
classification algorithm, and can also be extended to many other types of
training data.
