  We study robust linear regression in high-dimension, when both the dimension
$d$ and the number of data points $n$ diverge with a fixed ratio $\alpha=n/d$,
and study a data model that includes outliers. We provide exact asymptotics for
the performances of the empirical risk minimisation (ERM) using
$\ell_2$-regularised $\ell_2$, $\ell_1$, and Huber losses, which are the
standard approach to such problems. We focus on two metrics for the
performance: the generalisation error to similar datasets with outliers, and
the estimation error of the original, unpolluted function. Our results are
compared with the information theoretic Bayes-optimal estimation bound. For the
generalization error, we find that optimally-regularised ERM is asymptotically
consistent in the large sample complexity limit if one perform a simple
calibration, and compute the rates of convergence. For the estimation error
however, we show that due to a norm calibration mismatch, the consistency of
the estimator requires an oracle estimate of the optimal norm, or the presence
of a cross-validation set not corrupted by the outliers. We examine in detail
how performance depends on the loss function and on the degree of outlier
corruption in the training set and identify a region of parameters where the
optimal performance of the Huber loss is identical to that of the $\ell_2$
loss, offering insights into the use cases of different loss functions.
