  The state-of-the-art methods for estimating high-dimensional covariance
matrices all shrink the eigenvalues of the sample covariance matrix towards a
data-insensitive shrinkage target. The underlying shrinkage transformation is
either chosen heuristically - without compelling theoretical justification - or
optimally in view of restrictive distributional assumptions. In this paper, we
propose a principled approach to construct covariance estimators without
imposing restrictive assumptions. That is, we study distributionally robust
covariance estimation problems that minimize the worst-case Frobenius error
with respect to all data distributions close to a nominal distribution, where
the proximity of distributions is measured via a divergence on the space of
covariance matrices. We identify mild conditions on this divergence under which
the resulting minimizers represent shrinkage estimators. We show that the
corresponding shrinkage transformations are intimately related to the
geometrical properties of the underlying divergence. We also prove that our
robust estimators are efficiently computable and asymptotically consistent and
that they enjoy finite-sample performance guarantees. We exemplify our general
methodology by synthesizing explicit estimators induced by the
Kullback-Leibler, Fisher-Rao, and Wasserstein divergences. Numerical
experiments based on synthetic and real data show that our robust estimators
are competitive with state-of-the-art estimators.
