  Successful application of machine learning models to real-world prediction
problems, e.g. financial forecasting and personalized medicine, has proved to
be challenging, because such settings require limiting and quantifying the
uncertainty in the model predictions, i.e. providing valid and accurate
prediction intervals. Conformal Prediction is a distribution-free approach to
construct valid prediction intervals in finite samples. However, the prediction
intervals constructed by Conformal Prediction are often (because of
over-fitting, inappropriate measures of nonconformity, or other issues) overly
conservative and hence inadequate for the application(s) at hand. This paper
proposes an AutoML framework called Automatic Machine Learning for Conformal
Prediction (AutoCP). Unlike the familiar AutoML frameworks that attempt to
select the best prediction model, AutoCP constructs prediction intervals that
achieve the user-specified target coverage rate while optimizing the interval
length to be accurate and less conservative. We tested AutoCP on a variety of
datasets and found that it significantly outperforms benchmark algorithms.
