  We study the behavior of high-dimensional robust regression estimators in the
asymptotic regime where $p/n$ tends to a finite non-zero limit. More
specifically, we study ridge-regularized estimators, i.e
$\widehat{\beta}=\text{argmin}_{\beta \in \mathbb{R}^p} \frac{1}{n}\sum_{i=1}^n
\rho(\varepsilon_i-X_i' \beta)+\frac{\tau}{2}\lVert\beta\rVert^2$. In a
recently published paper, we had developed with collaborators probabilistic
heuristics to understand the asymptotic behavior of $\widehat{\beta}$. We give
here a rigorous proof, properly justifying all the arguments we had given in
that paper. Our proof is based on the probabilistic heuristics we had
developed, and hence ideas from random matrix theory, measure concentration and
convex analysis. While most the work is done for $\tau>0$, we show that under
some extra assumptions on $\rho$, it is possible to recover the case $\tau=0$
as a limiting case. We require that the $X_i$'s be i.i.d with independent
entries, but our proof handles the case where these entries are not Gaussian. A
2-week old paper of Donoho and Montanari [arXiv:1310.7320] studied a similar
problem by a different method and with a different point of view. At this
point, their interesting approach requires Gaussianity of the design matrix.
