  Convex regression is a method for estimating an unknown function $f_0$ from a
data set of $n$ noisy observations when $f_0$ is known to be convex. This
method has played an important role in operations research, economics, machine
learning, and many other areas. It has been empirically observed that the
convex regression estimator produces inconsistent estimates of $f_0$ and
extremely large subgradients near the boundary of the domain of $f_0$ as $n$
increases. In this paper, we provide theoretical evidence of this overfitting
behaviour. We also prove that the penalised convex regression estimator, one of
the variants of the convex regression estimator, exhibits overfitting
behaviour. To eliminate this behaviour, we propose two new estimators by
placing a bound on the subgradients of the estimated function. We further show
that our proposed estimators do not exhibit the overfitting behaviour by
proving that (a) they converge to $f_0$ and (b) their subgradients converge to
the gradient of $f_0$, both uniformly over the domain of $f_0$ with probability
one as $n \rightarrow \infty$. We apply the proposed methods to compute the
cost frontier function for Finnish electricity distribution firms and confirm
their superior performance in predictive power over some existing methods.
