  This article studies the behavior of regularized Tyler estimators (RTEs) of
scatter matrices. The key advantages of these estimators are twofold. First,
they guarantee by construction a good conditioning of the estimate and second,
being a derivative of robust Tyler estimators, they inherit their robustness
properties, notably their resilience to the presence of outliers. Nevertheless,
one major problem that poses the use of RTEs in practice is represented by the
question of setting the regularization parameter $\rho$. While a high value of
$\rho$ is likely to push all the eigenvalues away from zero, it comes at the
cost of a larger bias with respect to the population covariance matrix. A deep
understanding of the statistics of RTEs is essential to come up with
appropriate choices for the regularization parameter. This is not an easy task
and might be out of reach, unless one considers asymptotic regimes wherein the
number of observations $n$ and/or their size $N$ increase together. First
asymptotic results have recently been obtained under the assumption that $N$
and $n$ are large and commensurable. Interestingly, no results concerning the
regime of $n$ going to infinity with $N$ fixed exist, even though the
investigation of this assumption has usually predated the analysis of the most
difficult $N$ and $n$ large case. This motivates our work. In particular, we
prove in the present paper that the RTEs converge to a deterministic matrix
when $n\to\infty$ with $N$ fixed, which is expressed as a function of the
theoretical covariance matrix. We also derive the fluctuations of the RTEs
around this deterministic matrix and establish that these fluctuations converge
in distribution to a multivariate Gaussian distribution with zero mean and a
covariance depending on the population covariance and the parameter $\rho$.
