  Learning with rejection (LWR) allows development of machine learning systems
with the ability to discard low confidence decisions generated by a prediction
model. That is, just like human experts, LWR allows machine models to abstain
from generating a prediction when reliability of the prediction is expected to
be low. Several frameworks for this learning with rejection have been proposed
in the literature. However, most of them work for classification problems only
and regression with rejection has not been studied in much detail. In this
work, we present a neural framework for LWR based on a generalized meta-loss
function that involves simultaneous training of two neural network models: a
predictor model for generating predictions and a rejecter model for deciding
whether the prediction should be accepted or rejected. The proposed framework
can be used for classification as well as regression and other related machine
learning tasks. We have demonstrated the applicability and effectiveness of the
method on synthetically generated data as well as benchmark datasets from UCI
machine learning repository for both classification and regression problems.
Despite being simpler in implementation, the proposed scheme for learning with
rejection has shown to perform at par or better than previously proposed
methods. Furthermore, we have applied the method to the problem of hurricane
intensity prediction from satellite imagery. Significant improvement in
performance as compared to conventional supervised methods shows the
effectiveness of the proposed scheme in real-world regression problems.
