  When data is sampled from an unknown subspace, principal component analysis
(PCA) provides an effective way to estimate the subspace and hence reduce the
dimension of the data. At the heart of PCA is the Eckart-Young-Mirsky theorem,
which characterizes the best rank k approximation of a matrix. In this paper,
we prove a generalization of the Eckart-Young-Mirsky theorem under all
unitarily invariant norms. Using this result, we obtain closed-form solutions
for a set of rank/norm regularized problems, and derive closed-form solutions
for a general class of subspace clustering problems (where data is modelled by
unions of unknown subspaces). From these results we obtain new theoretical
insights and promising experimental results.
