  Learning with rejection is an important framework that can refrain from
making predictions to avoid critical mispredictions by balancing between
prediction and rejection. Previous studies on cost-based rejection only focused
on the classification setting, which cannot handle the continuous and infinite
target space in the regression setting. In this paper, we investigate a novel
regression problem called regression with cost-based rejection, where the model
can reject to make predictions on some examples given certain rejection costs.
To solve this problem, we first formulate the expected risk for this problem
and then derive the Bayes optimal solution, which shows that the optimal model
should reject to make predictions on the examples whose variance is larger than
the rejection cost when the mean squared error is used as the evaluation
metric. Furthermore, we propose to train the model by a surrogate loss function
that considers rejection as binary classification and we provide conditions for
the model consistency, which implies that the Bayes optimal solution can be
recovered by our proposed surrogate loss. Extensive experiments demonstrate the
effectiveness of our proposed method.
