  Many popular statistical models, such as factor and random effects models,
give arise a certain type of covariance structures that is a summation of low
rank and sparse matrices. This paper introduces a penalized approximation
framework to recover such model structures from large covariance matrix
estimation. We propose an estimator based on minimizing a non-likelihood loss
with separable non-smooth penalty functions. This estimator is shown to recover
exactly the rank and sparsity patterns of these two components, and thus
partially recovers the model structures. Convergence rates under various matrix
norms are also presented. To compute this estimator, we further develop a
first-order iterative algorithm to solve a convex optimization problem that
contains separa- ble non-smooth functions, and the algorithm is shown to
produce a solution within O(1/t^2) of the optimal, after any finite t
iterations. Numerical performance is illustrated using simulated data and stock
portfolio selection on S&P 100.
