  In the context of linear regression, we construct a data-driven convex loss
function with respect to which empirical risk minimisation yields optimal
asymptotic variance in the downstream estimation of the regression
coefficients. Our semiparametric approach targets the best decreasing
approximation of the derivative of the log-density of the noise distribution.
At the population level, this fitting process is a nonparametric extension of
score matching, corresponding to a log-concave projection of the noise
distribution with respect to the Fisher divergence. The procedure is
computationally efficient, and we prove that our procedure attains the minimal
asymptotic covariance among all convex $M$-estimators. As an example of a
non-log-concave setting, for Cauchy errors, the optimal convex loss function is
Huber-like, and our procedure yields an asymptotic efficiency greater than 0.87
relative to the oracle maximum likelihood estimator of the regression
coefficients that uses knowledge of this error distribution; in this sense, we
obtain robustness without sacrificing much efficiency. Numerical experiments
confirm the practical merits of our proposal.
