  Two common approaches in low-rank optimization problems are either working
directly with a rank constraint on the matrix variable, or optimizing over a
low-rank factorization so that the rank constraint is implicitly ensured. In
this paper, we study the natural connection between the rank-constrained and
factorized approaches. We show that all second-order stationary points of the
factorized objective function correspond to fixed points of projected gradient
descent run on the original problem (where the projection step enforces the
rank constraint). This result allows us to unify many existing optimization
guarantees that have been proved specifically in either the rank-constrained or
the factorized setting, and leads to new results for certain settings of the
problem. We demonstrate application of our results to several concrete low-rank
optimization problems arising in matrix inverse problems.
