  Machine learning models always make a prediction, even when it is likely to
be inaccurate. This behavior should be avoided in many decision support
applications, where mistakes can have severe consequences. Albeit already
studied in 1970, machine learning with rejection recently gained interest. This
machine learning subfield enables machine learning models to abstain from
making a prediction when likely to make a mistake.
  This survey aims to provide an overview on machine learning with rejection.
We introduce the conditions leading to two types of rejection, ambiguity and
novelty rejection, which we carefully formalize. Moreover, we review and
categorize strategies to evaluate a model's predictive and rejective quality.
Additionally, we define the existing architectures for models with rejection
and describe the standard techniques for learning such models. Finally, we
provide examples of relevant application domains and show how machine learning
with rejection relates to other machine learning research areas.
