  Statistical models that possess symmetry arise in diverse settings such as
random fields associated to geophysical phenomena, exchangeable processes in
Bayesian statistics, and cyclostationary processes in engineering. We formalize
the notion of a symmetric model via group invariance. We propose projection
onto a group fixed point subspace as a fundamental way of regularizing
covariance matrices in the high-dimensional regime. In terms of parameters
associated to the group we derive precise rates of convergence of the
regularized covariance matrix and demonstrate that significant statistical
gains may be expected in terms of the sample complexity. We further explore the
consequences of symmetry on related model-selection problems such as the
learning of sparse covariance and inverse covariance matrices. We also verify
our results with simulations.
