  In the linear regression model, the minimum l2-norm interpolant estimator has
received much attention since it was proved to be consistent even though it
fits noisy data perfectly under some condition on the covariance matrix
$\Sigma$ of the input vector, known as benign overfitting. Motivated by this
phenomenon, we study the generalization property of this estimator from a
geometrical viewpoint. Our main results extend and improve the convergence
rates as well as the deviation probability from [Tsigler and Bartlett]. Our
proof differs from the classical bias/variance analysis and is based on the
self-induced regularization property introduced in [Bartlett, Montanari and
Rakhlin]: the minimum l2-norm interpolant estimator can be written as a sum of
a ridge estimator and an overfitting component. The two geometrical properties
of random Gaussian matrices at the heart of our analysis are the
Dvoretsky-Milman theorem and isomorphic and restricted isomorphic properties.
In particular, the Dvoretsky dimension appearing naturally in our geometrical
viewpoint, coincides with the effective rank and is the key tool for handling
the behavior of the design matrix restricted to the sub-space where overfitting
happens.
  We extend these results to heavy-tailed scenarii proving the universality of
this phenomenon beyond exponential moment assumptions. This phenomenon is
unknown before and is widely believed to be a significant challenge. This
follows from an anistropic version of the probabilistic Dvoretsky-Milman
theorem that holds for heavy-tailed vectors which is of independent interest.
