  Given a set of vectors (the data) in a Hilbert space H, we prove the
existence of an optimal collection of subspaces minimizing the sum of the
square of the distances between each vector and its closest subspace in the
collection. This collection of subspaces gives the best sparse representation
for the given data, in a sense defined in the paper, and provides an optimal
model for sampling in union of subspaces. The results are proved in a general
setting and then applied to the case of low dimensional subspaces of R^N and to
infinite dimensional shift-invariant spaces in L^2(R^d). We also present an
iterative search algorithm for finding the solution subspaces. These results
are tightly connected to the new emergent theories of compressed sensing and
dictionary design, signal models for signals with finite rate of innovation,
and the subspace segmentation problem.
