  Perturbation bounds for singular spaces, in particular Wedin's $\sin \Theta$
theorem, are a fundamental tool in many fields including high-dimensional
statistics, machine learning, and applied mathematics. In this paper, we
establish separate perturbation bounds, measured in both spectral and Frobenius
$\sin \Theta$ distances, for the left and right singular subspaces. Lower
bounds, which show that the individual perturbation bounds are rate-optimal,
are also given.
  The new perturbation bounds are applicable to a wide range of problems. In
this paper, we consider in detail applications to low-rank matrix denoising and
singular space estimation, high-dimensional clustering, and canonical
correlation analysis (CCA). In particular, separate matching upper and lower
bounds are obtained for estimating the left and right singular spaces. To the
best of our knowledge, this is the first result that gives different optimal
rates for the left and right singular spaces under the same perturbation. In
addition to these problems, applications to other high-dimensional problems
such as community detection in bipartite networks, multidimensional scaling,
and cross-covariance matrix estimation are also discussed.
