  We study the spectra of MANOVA estimators for variance component covariance
matrices in multivariate random effects models. When the dimensionality of the
observations is large and comparable to the number of realizations of each
random effect, we show that the empirical spectra of such estimators are
well-approximated by deterministic laws. The Stieltjes transforms of these laws
are characterized by systems of fixed-point equations, which are numerically
solvable by a simple iterative procedure. Our proof uses operator-valued free
probability theory, and we establish a general asymptotic freeness result for
families of rectangular orthogonally-invariant random matrices, which is of
independent interest. Our work is motivated by the estimation of components of
covariance between multiple phenotypic traits in quantitative genetics, and we
specialize our results to common experimental designs that arise in this
application.
